JP2021064147A - Moving object system, moving object, control program, and control method - Google Patents

Abstract

To provide a moving object system capable of responding to a detailed judgment and operation by a person while reducing a workload of a person by autonomous operation of a moving object.SOLUTION: This disclosure relates to a mobile system 1 having a mobile object (robot) 10 and a terminal 20 used by an operator OP. The mobile object has an information acquisition unit 14, 15 that acquires information pertaining to the surroundings of the mobile object, a moving unit 13 that moves the mobile object in any direction, and a control unit 11 that performs an autonomous control process for the mobile object using a result acquired by the information acquisition unit in the autonomous control process to identify whether or not to abort the autonomous control process, and that can notify a terminal if the autonomous control identification process determines that the autonomous control process should not be continued.SELECTED DRAWING: Figure 1

Description

The present invention relates to a moving object system, a moving object, a control program, and a control method, and relates to, for example, a robot corresponding to a moving function and a communication function, and a system including a means for remotely controlling the robot by an operator.

Conventionally, there is a technique described in Patent Document 1 as a robot system including a robot corresponding to a movement function and a communication function and a means for remotely controlling the robot by an operator.

In the robot system described in Patent Document 1, a robot that can freely move autonomously in a conference room and a conference in a conference room can be communicated to participants in a remote conference via the robot and a conference in the conference room. It is possible to provide a communication function capable of exchanging information with participants (for example, exchanging information such as presentation of materials and conversation).

JP-A-2002-46088

However, in the conventional robot that can move autonomously as described above, although it can move freely, there is a large gap between the environmental information such as the moving path (running path) stored by the robot and the state of the actual moving path. Abnormalities that cannot be dealt with by the autonomous control of the robot (abnormalities that cannot be dealt with from the beginning) may occur, such as when there is an obstacle or when there is an obstacle on the moving path. When an abnormality that cannot be dealt with by autonomous control as described above occurs, the robot needs to rush to perform maintenance work, in addition to the risk of accidents due to unexpected movements and the decrease in operating rate due to suspension of operation. There was a challenge.

In addition, with conventional robots, due to technical restrictions, when a robot attempts to perform work including processes that cannot be realized by fully automatic operation, it is necessary for humans to operate (remotely control) everything.

Furthermore, with conventional robots, it is necessary for one person to handle one robot in complete remote control that does not operate autonomously, and the effect of labor saving (labor saving of human work) by using the robot is limited. there were.

In view of the above problems, movement that enables the autonomous movement of moving objects (for example, robots) to reduce the workload of humans (for example, operators) while also allowing humans to make detailed judgments and operations. A physical system is desired.

According to the first aspect of the present invention, in a moving object system having a moving object and a terminal used by an operator, (1) the moving object is (1-1) an information acquisition unit for acquiring information related to the moving object. , (1-2) A moving unit that moves the moving object in an arbitrary direction, and (1-3) an autonomous control process for the moving object. In the process of the autonomous control process, the imaging unit And / or, using the result acquired by the information acquisition unit, the autonomous control identification process for identifying whether or not to stop the autonomous control process is performed, and as a result of the autonomous control identification process, the continuation of the autonomous control process is performed. It is characterized by having a control unit capable of notifying the terminal when it is determined not to do so.

In the second aspect of the present invention, there are (1) an information acquisition unit for acquiring the information related to the moving object, (2) a moving unit for moving the moving object in an arbitrary direction, and (3) the moving object. The autonomous control process is performed, and in the process of the autonomous control process, the result acquired by the imaging unit and / or the information acquisition unit is used to identify whether or not to stop the autonomous control process. It is characterized by having a control unit capable of notifying an external terminal used by the operator when the control identification process is performed and, as a result of the autonomous control identification process, it is determined that the autonomous control process is not continued. To do.

According to the present invention, it is possible to realize a moving object system capable of responding to fine judgments and operations by human beings while reducing the workload of human beings by autonomous operation of moving objects.

It is a block diagram which showed the whole structure (including the functional structure of the robot which concerns on embodiment) of the robot system which concerns on 1st Embodiment. It is a flowchart which showed the operation of the robot which concerns on 1st Embodiment. It is a figure which showed the example of the planned movement route which the robot which concerns on 1st Embodiment moves. It is a figure (the 1) which showed the example of the avoidance route (alternative route) acquired by the robot which concerns on 1st Embodiment. It is a figure (No. 2) which showed the example of the avoidance route (alternative route) acquired by the robot which concerns on 1st Embodiment. It is a figure (No. 3) which showed the example of the avoidance route (alternative route) acquired by the robot which concerns on 1st Embodiment. It is a figure which showed the example of the process which the robot which concerns on 1st Embodiment acquires the correction alternative route by route correction. It is a figure which showed the example of the policy when the robot which concerns on 1st Embodiment acquires an avoidance route. It is a figure which showed the example (the 1) of the operation screen which receives the remote control of a robot with the operation desk which concerns on 1st Embodiment. It is a figure which showed the example (the 2) of the operation screen which receives the remote control of a robot with the operation desk which concerns on 1st Embodiment. It is a figure which showed the example of the screen at the time of manually accepting the input of the avoidance route from the operator on the operation console which concerns on 1st Embodiment. It is a figure which showed the example of the monitoring screen which makes an operator monitor the movement state of a robot on the operation console which concerns on 1st Embodiment. It is a block diagram which showed the whole structure (including the functional structure of the robot which concerns on embodiment) of the robot system which concerns on 2nd and 3rd Embodiment. It is a flowchart which showed the operation of the robot which concerns on 2nd Embodiment. It is a flowchart (the 1) which showed the operation of the robot which concerns on 3rd Embodiment. It is a flowchart (No. 2) which showed the operation of the robot which concerns on 3rd Embodiment.

(A) First Embodiment Hereinafter, the first embodiment of the moving object system, the moving object, the control program, and the control method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the moving object system and the moving object of the present invention are applied to the robot system and the robot will be described.

(A-1) Configuration of First Embodiment FIG. 1 is a block diagram showing an overall configuration of the robot system 1 of this embodiment.

The robot system 1 has a robot 10 and an operation console.

Here, a robot generally refers to "an android", "a device that performs actions and tasks similar to humans", "a device that performs actions and tasks based on instructions by computer operation", and "autonomous actions and tasks". "Devices operated by computer", "Machines used in industry that have manipulation functions or movement functions by automatic control and can execute various tasks by programs" (JIS B 0134-1998), "Robots that serve humans" (JIS B 0187: 2005) "Intelligent mechanical system with three elemental technologies of sensor, intelligence / control system, and drive system" (Ministry of Economy, Trade and Industry Robot Policy Study Group), etc.

In this embodiment, the robot 10 is a general-purpose moving object that can be applied to various uses (operations). The robot 10 is used, for example, for security (for example, patrol monitoring at a facility to be guarded or dialogue with a human found at a patrol destination), manufacturing (for example, equipment inspection in a factory, transportation of manufactured parts and materials), construction (for example, for example). Transportation of building parts, confirmation of construction status), passenger business (for example, transportation of passengers and luggage, inspection of tourist spots and luggage, dialogue with passengers), inspection of various infrastructures, logistics business (for example, transportation of cargo, cargo) It can be applied to applications (operations) such as inspection of freight vehicles. Further, the robot 10 may execute a plurality of types of operations at the same time.

Further, the robot 10 can operate autonomously and also cooperate with and cooperate with a human being (operator OP of the operation console 20). For example, the robot 10 performs autonomous notification processing to the console 20, data transmission processing according to the request of the console 20, and operations (for example, movement, etc.) according to control (control signal) from the console 20. ) Is also possible.

The console 20 is a terminal used by the operator OP. The operation console 20 corresponds to a function for realizing a coordinated / coordinated operation between the operator OP and the robot 10. For example, the console 20 executes notification output from the robot 10, data output regarding the state (status) of the robot 10, remote control processing of the robot 10 according to the operation of the operator OP, monitoring processing regarding the state of the operator OP, and the like. It shall be possible.

It is assumed that the robot 10 and the console 20 can communicate with each other via an IP network NW (for example, a network such as the Internet or a LAN). The communication method between the robot 10 and the console 20 is not limited, and various communication methods other than the IP network can be applied. In this embodiment, it is assumed that at least the robot 10 is a communication device capable of wireless LAN communication and can be connected to the IP network NW at an arbitrary position where the operation is executed.

Next, the internal configuration of the robot 10 will be described.

As shown in FIG. 1, the robot 10 has a control unit 11, a communication unit 12, a moving unit 13, a camera unit 14, a sensor unit 15, a display unit 16, a speaker 17, and a microphone 18.

The control unit 11 has a function of controlling the entire robot 10 and a function of various data processing. The control unit 11 can be realized, for example, by installing a program including the control program according to this embodiment on a computer having a processor and a memory (program execution configuration).

The communication unit 12 is a communication IF for connecting to the IP network NW. In this embodiment, it is assumed that the communication unit 12 can wirelessly connect to the IP network NW (a wireless base station that can connect to the IP network NW).

The moving unit 13 is a moving mechanism (moving means) for moving the own device (robot 10) in an arbitrary direction according to the control of the control unit 11. In this embodiment, it is assumed that the moving unit 13 can move in an arbitrary direction within an area for executing an operation (hereinafter, referred to as an “operation execution area”). The movement method supported by the moving unit 13 is not limited. Various moving mechanisms such as wheels, caterpillars, hovercraft, bipedal walking, and quadrupedal walking can be applied to the moving unit 13. Further, the moving portion 13 is a moving mechanism capable of not only two-dimensional movement but also three-dimensional movement (for example, it may be movable in the air by a rotary blade, or it may be movable on or under water by a screw. May be).

The camera unit 14 is an imaging means capable of imaging the periphery of the robot 10 with one or a plurality of cameras (imaging devices) under the control of the control unit 11. The camera unit 14 may be configured to secure a field of view of the entire periphery by, for example, a plurality of cameras.

The sensor unit 15 is a detection means capable of acquiring sensor information (detection result) by performing various sensing processes around the robot 10 by one or a plurality of types of sensors (detection devices). In the present specification, "sensing" means detection of estimated amounts of temperature, humidity, illuminance, atmospheric pressure, vibration (vibration), distance, inclination, etc. (detection of increase / decrease or occurrence), as well as smoke, chemical substances, and the like. It is also used as a term to indicate the detection of the generation of static electricity (detection of increase / decrease or estimated amount). Further, in the present specification, "sensing" is used as a word indicating a process of converting the detected content into a signal in addition to the detection. Further, in the present specification, "sensing" is used as a word indicating measurement / discrimination / analysis of the detected content in addition to detection. The type of sensor included in the sensor unit 15 is not limited. For example, the sensor unit 15 is from a thermometer, a hygrometer, a luminometer, a barometer, a lidar (Laser Imaging Detection and Ranger) sensor for detecting the arrangement status of surrounding objects, a thermal image measuring device (thermography camera), or the like. One or a plurality of types may be provided. For example, the sensor unit 15 acquires sensor information from an external sensor device (a device capable of performing various sensing processes around the sensor device to acquire sensor information (detection result)) by wireless communication. You may. In this case, the sensor unit 15 has a sensing wireless communication unit (for example, a wireless communication unit corresponding to 920 MHz band multi-hop wireless communication), and the sensor unit 15 (sensing wireless communication unit) serves as a master unit and a slave unit. The sensor information is acquired by exchanging with the sensor device by wireless communication (multi-hop wireless communication in which wireless communication is relayed by a plurality of slave units (sensor devices) serving as repeaters).

As described above, in the robot 10 of this embodiment, an information acquisition unit that acquires information related to the surroundings of the robot 10 is configured by the camera unit 14 and the sensor unit 15. If the robot 10 can acquire all the information around the own device required for the processing of the control unit 11 only by the camera unit 14, the sensor unit 15 may be excluded.

The display unit 16 is a display means that displays and outputs various information according to the control of the control unit 11. For example, the display unit 16 can apply a display device such as a display or a lamp.

The speaker 17 is an acoustic output means that outputs an acoustic (for example, an operator OP voice, a synthetic voice, etc.) according to the control of the control unit 11 to the periphery of the robot 10 (acoustic output, phonetic output).

The microphone 18 is a sound capturing means for capturing the sound around the robot 10 (for example, the voice of a person around the robot 10).

Next, the internal configuration of the console 20 will be described.

The operation console 20 includes a control unit 21, a communication unit 22, a display unit 23, an operation unit 24, a camera unit 25, a sensor unit 26, a speaker 27, and a microphone 28.

In the console 20, various computer (for example, smart phones and PCs) configurations can be applied to the configurations other than the sensor unit 26.

The control unit 21 has a function of controlling the entire operation console 20 and a function of various data processing. The control unit 21 can be realized, for example, by installing a program including a console control program (terminal control program) according to this embodiment on a computer having a processor and a memory (program execution configuration).

The communication unit 22 is a communication IF for connecting to the IP network NW.

The display unit 23 is a display means that displays and outputs various information according to the control of the control unit 21. For example, the display unit 23 can apply a display (including a display constituting a touch panel display). In the operation console 20, for example, the display unit 23 can display an operation screen (GUI) including an image captured by the camera unit 14 of the robot 10 (hereinafter, referred to as a “camera image”).

The operation unit 24 is an operation means (operation device) for receiving an operation from the operator OP. For example, the operation unit 24 includes a keyboard, a mouse, a touch panel (including a touch panel constituting the touch panel display), a track pad, a controller (for example, a device capable of accepting operations related to the movement of the robot 10 by a cross key or an operation lever) and the like. Devices can be applied. The operation unit 24 can receive operations (inputs) such as the movement direction and movement speed of the robot 10 from the operator OP, for example.

The camera unit 25 is an imaging means that images the periphery of the console 20 (including the periphery of the operator OP and / or the operator OP).

The sensor unit 26 is a detection means capable of performing various sensing processes on the periphery of the console 20 (including the periphery of the operator OP and / or the periphery of the operator OP) by one or a plurality of types of sensors (detection devices). The sensor unit 26 may apply, for example, a sensor that can detect vital signs of the operator OP (for example, body temperature, heart rate, blood pressure value, respiratory rate, etc.). For example, like the sensor unit 15, the sensor unit 26 may acquire sensor information from an external sensor device by wireless communication.

The speaker 27 is an acoustic output means that outputs sound (acoustic output, phonetic output) to the periphery (including the operator OP) of the console 20 according to the control of the control unit 21.

The microphone 28 is a sound capturing means for capturing the sound around the console 20 (including the voice spoken by the operator OP).

Next, an outline of the autonomous control process performed by the control unit 11 of the robot 10 (hereinafter, referred to as “autonomous control process”) and the cooperative process with the operator OP via the console 20 will be described.

The control unit 11 normally controls each part of the robot 10 by autonomous judgment (hereinafter referred to as "autonomous control"), but the camera image captured by the camera unit 14 and / or the sensor information detected by the sensor unit 15. When a predetermined event is detected by using (detection result), the autonomous control may be stopped. In the following, an event in which the control unit 11 determines that the independent control should be stopped shall be referred to as an "autonomous control stop incident" or simply an "incident". Then, when the control unit 11 detects an incident, the control unit 11 performs processing according to the content of the incident (for example, the type of the incident). For example, when the control unit 11 detects an incident, the control unit 11 notifies the operator OP via the console 20 and accepts a remote control under the control of the console 20 (operator OP) (hereinafter referred to as an operation mode). It may shift to "remote control mode").

Specifically, for example, when the control unit 11 detects an obstacle that hinders the movement (running) of the robot 10 on the moving path and determines that the obstacle cannot be avoided only by autonomous movement. Incidents originating from the obstacle (hereinafter, also referred to as “incident source”) may be detected. In this case, for example, the control unit 11 notifies the operation console 20 (operator OP) that an incident has occurred, shifts to the remote control mode, and takes over the operation to the operation console 20 (operator OP). You can avoid objects and continue moving.

The method by which the control unit 11 detects an obstacle is not limited. For example, the control unit 11 detects an obstacle based on the camera image captured by the camera unit 14 and the sensor information (detection result) detected by the sensor unit 15 (for example, an ultrasonic sensor or a millimeter wave radar (not shown)). It may be detected. The control unit 11 may detect an obstacle based on one or both of the camera image and the sensor information. Further, when the control unit 11 finds an obstacle on the moving path, the control unit 11 may recognize the type and state of the obstacle. For example, the control unit 11 uses AI processing (for example, AI processing using a model in which an image of an incident source (for example, an obstacle) assumed in advance is learned as teacher data) from camera images and sensor information. It is also possible to acquire information that identifies the type and state of the obstacle.

For example, an obstacle, such as a small tree branch, may be an obstacle that can be avoided without changing the road (passage / road) planned by the robot 10. There are patterns such as large fallen trees that completely block the road (passage / road). At this time, the control unit 11 divides the frequency and the level to be taken into consideration in advance, and sets in advance whether or not some action is required for the discovered obstacle by the intervention of the operator OP. You may. For example, in the robot system 1, the operator OP searches for another route (detour) by manually operating the console 20 according to the type (level) of the obstacle detected by the robot 10 (control unit 11). It may be possible to urge people (for example, local guards, etc.) to rush to remove obstacles.

Then, when the control unit 11 determines that the intervention of a person (for example, an operator OP, a local security guard, etc.) is necessary, it determines that an incident (autonomous control cancellation incident) has been detected, and determines that the communication unit 12 has been detected. Notify the console 20 (operator OP) of information about the incident (for example, it may include the need for human intervention and the cause of the decision), and shift to the remote control mode. It may be. Then, the control unit 21 of the operation console 20 outputs the received notification to the operator OP (for example, outputs using the display unit 23 or the speaker 27) so that the operator OP recognizes the detection of the incident and the operation console. 20 (operation unit 24) can be manually operated to check the surrounding conditions of the robot 10 while making decisions such as avoiding obstacles or dispatching people to the site as necessary. it can.

(A-2) Operation of the First Embodiment Next, the operation of the robot system 1 of the first embodiment having the above configuration (control method according to the embodiment) will be described.

FIG. 2 is a flowchart showing the operation of the robot 10 in this embodiment.

FIG. 2 is a flowchart showing an operation (process executed by the control unit 11) when the robot 10 moves from the start position (first position) to the target position (second position) in the process of operation. It has become.

When starting the movement from the start position to the target point, the control unit 11 of the robot 10 first calculates a route for reaching the target position from the start position (hereinafter, referred to as a “planned movement route”) (hereinafter, referred to as a “planned movement route”). S101).

Next, the control unit 11 controls the moving unit 13 (controls the direction and speed of moving so as to move along the planned movement route) so as to move according to the calculated movement schedule route (S102).

At this time, the control unit 11 confirms whether or not the target position has been reached (S103) while moving, and an obstacle on the planned movement route (an object or fear of hindering the movement of the robot 10 on the planned movement route). The presence or absence of (an object with) is detected (S104). At this time, when the moving unit 13 confirms that the target point has been reached, the processing of the flowchart is terminated, and when an obstacle is detected on the planned movement route, the moving unit 13 proceeds to step S105, which will be described later, to reach the target point. If it has not arrived and there are no obstacles on the planned movement route, the process returns to step S102 described above to operate and continue the movement on the planned movement route.

On the other hand, when an obstacle is detected on the planned movement route, the control unit 11 determines whether or not the obstacle can be avoided by changing the planned movement route (presence or absence of the avoidance route) (S105). If it is determined that it can be avoided, the process proceeds to step S109 described later, and if not, the process proceeds to step S106 described later.

When it is determined in step S105 that the obstacle can be avoided by changing the planned movement route, the control unit 11 sets a new planned movement route avoiding the obstacle (S109), and in step S102 described above. Go back and resume moving to the target position.

On the other hand, in step S105 described above, when it is determined that the obstacle cannot be avoided due to the change of the planned movement route, the control unit 11 determines that an incident has occurred, and determines to the operation console 20 (operator OP). A notification is given (S106), and the presence or absence of a response (response to the notification) from the console 20 is confirmed (S107). At this time, the control unit 11 may control the moving unit 13 to temporarily stop the movement.

Then, when there is a response to the notification from the console 20, the control unit 11 proceeds to step S108, which will be described later, and when there is no response to the notification (for example, when there is no response within a certain time from the immediately preceding notification), the control unit 11 proceeds to step S108, which will be described later. Returning to step S106 described above, the notification process is performed again.

When the response from the operation console 20 is received in step S107 described above, the control unit 11 switches to the remote control mode in which the movement unit 13 is operated in response to the control signal from the operation console 20 (S208). The operation right of the above is transferred to the operation console 20, and the processing of the flowchart is completed.

Next, an example of a specific scenario in which the robot 10 (moving unit 13) operates according to the flowchart of FIG. 2 described above will be described.

Here, as shown in FIG. 3, a scenario in which the robot 10 moves from the position PA to the position PB in the process of operation will be described.

FIG. 3 illustrates an example of a map of a planned movement route on which the robot 10 moves. In the map shown in FIG. 3, the area where the robot 10 cannot move (hereinafter, referred to as “immovable area”) is shaded with diagonal lines (hatch) so that the robot 10 can move (hereinafter, “moving”). It is divided into "possible areas"). In FIG. 3, the line of the first route R1 is shown as a route that can be reached from the position PA to the position PB. In the present specification, when the map is illustrated, it will be described in the same format as in FIG.

Here, it is assumed that the robot 10 autonomously moves (moves by autonomous control) with the first route R1 as the planned movement route. That is, it can be said that FIG. 3 visualizes the information required for autonomous movement (information necessary for determining the planned movement route; hereinafter referred to as “movement route information”) in the control unit 11 of the robot 10. .. In other words, here, it is assumed that the control unit 11 of the robot 10 holds the movement route information as shown in FIG. That is, the movement route information includes map information of the entire area where the robot 10 can move (hereinafter, the map based on this map information is referred to as "overall map") and initial position information on the entire map (for example, in FIG. 3). Information such as coordinates that specify the position PA), target position information on the overall map (for example, information such as coordinates that specify the position PB in FIG. 3), and information on the planned movement route on the overall map (for example, FIG. 3). Information that identifies the curve corresponding to the first route R1 in the above) is included. The map information of the entire map may be, for example, two-dimensional information (map information represented by vertical and horizontal directions) as shown in FIG. 3, or three-dimensional information report (map information represented by vertical and horizontal heights). .. The control unit 11 of the robot 10 may store all the map information of the entire map in advance, or download it from the outside (for example, a server of map information (not shown) on the IP network NW) as needed. You may do so.

In step S101 described above, the algorithm in which the control unit 11 of the robot 10 calculates (determines) the planned movement route (for example, the first route R1 in FIG. 3) is not limited, and various algorithms are applied. be able to.

Then, when the control unit 11 of the robot 10 detects an obstacle on the planned movement route as in steps S104, S105, and S109 described above, the control unit 11 avoids the obstacle and moves to the target position based on the movement route information. The reachable avoidance route is identified, and if the avoidance route can be acquired, the avoidance route is set as a new planned movement route. The method by which the control unit 11 acquires (identifies) the avoidance route is not limited.

4 and 5 are diagrams showing an example of an avoidance route when an obstacle occurs on the first route R1.

FIG. 4 shows an example in which the control unit 11 of the robot 10 detects an obstacle OB at a position PD on the first route R1 at a point (position PA) where the movement is started with the first route R1 as the planned movement route. Is shown. In the example of FIG. 4, as avoidance routes that can be reached from the position PA to the position PB without passing through the position PD, for example, there are a second route R2 and a third route R3.

FIG. 5 shows an example in which the control unit 11 of the robot 10 starts moving with the first route R1 as the planned movement route, and detects an obstacle OB at the position PD when it reaches the position PC. In this case, as shown in FIG. 5, as an avoidance route that can reach the position PB from the position PC without passing through the position PD, for example, there is a fourth route R4.

Next, a specific example of the process of identifying (acquiring) the avoidance route when the control unit 11 detects an obstacle will be described.

For example, when the information of the alternative route corresponding to the planned travel route (the route that reaches the target position by a route different from the planned travel route) is preset in the travel route information (hereinafter, the preset alternative route). Is called a "static alternative route"). At this time, if the static alternative route can avoid obstacles and reach the target position, the control unit 11 can acquire the static alternative route as a candidate for the avoidance route. For example, when the control unit 11 of the robot 10 sets the second route R2 as a static alternative route corresponding to the first route R1, the second route R2 is acquired as a candidate for the avoidance route. become.

Further, the control unit 11 substitutes according to a predetermined algorithm (for example, various route determination algorithms using AI or the like) based on the movement route information, the current position of the own device, the position of the obstacle, and the target position. A route is searched (hereinafter, the alternative route detected at this time is referred to as a "dynamic alternative route"), and if a dynamic alternative route is detected, the dynamic alternative route is acquired as a candidate for an avoidance route. You may. For example, in the control unit 11 of the robot 10, the third route R3 shown in FIG. 4 is set as a dynamic alternative route based on the movement route information, the current position PA of the own device, the position PD of the obstacle, and the target position PB. If detected, the third route R3 will be acquired as a candidate for the avoidance route.

The control unit 11 is not limited to a specific algorithm for detecting a dynamic alternative route. For example, the control unit 11 can avoid the detected obstacle based on the current position of the own device, the position of the obstacle, and the target position, and sets the shortest route to reach the target position as a dynamic alternative route. It may be detected. For example, the control unit 11 may derive a pure shortest route leading to the target position from the point where the obstacle is found based on the RRT (Rapidly exploring random tree) algorithm or the like by AI, or find the obstacle. It is also possible to return to the current planned travel route and derive the shortest route leading to the target position while avoiding obstacles from the point. For example, in the example of FIG. 5, the fourth route R4 is a pure shortest route leading from the point position PC where the obstacle is found to the target position PB, while avoiding the obstacle from the position PC where the obstacle is found. It is also the shortest route that returns to the current route and leads to the target position PB.

As shown in FIGS. 4 and 5, the control unit 11 of the robot 10 may detect only a route whose road (passage / road) itself is different from the current planned movement route as a candidate for an avoidance route. By correcting (changing) the position of the route (passage / road) that is the same as the planned travel route, it may be possible to detect a route that avoids obstacles. For example, the control unit 11 identifies the positional relationship between the road (passage / road) of the current planned travel route and the obstacle based on the image captured by the camera unit 14 and the detection result of the sensor unit 15, and currently By correcting the position where the own device passes on the same road as the planned movement route (hereinafter referred to as "route correction"), it is possible to pass while avoiding obstacles (identify the width and height to identify the passability). ) May be determined. Then, when the obstacle can be avoided by the route correction, the control unit 11 sets the route (a route obtained by correcting the current planned movement route; hereinafter referred to as a "correction alternative route") as a candidate for the avoidance route. It may be obtained as.

A specific example of route correction (method of detecting a correction alternative route) performed by the control unit 11 of the robot 10 will be described with reference to FIGS. 6 and 7.

Here, as shown in FIG. 6, the control unit 11 of the robot 10 detects (discovers) an obstacle OB existing at the position PD at the position of the position PC while moving with the first route R1 as the planned movement route. It is assumed that

FIG. 7 is a diagram showing an example of a camera image in the traveling direction (moving direction) captured by the camera unit 25 of the robot 10.

FIG. 7 is a diagram showing a camera image when the camera unit 25 of the robot 10 captures the traveling direction of the first route R1 from the position PC. In this case, the control unit 11 of the robot 10 is based on the camera image as shown in FIG. 7A and the sensor information of the sensor unit 15 (for example, the detection result using a lidar sensor (not shown)). As shown in the above, in the passage of the position PD, it can be detected that a gap exists on the right side and the left side of the obstacle OB when viewed from the robot 10. At this time, the control unit 11 of the robot 10 has the size of the gap formed on the right side and the left side of the obstacle OB based on the camera image and the sensor information of the sensor unit 15 as shown in FIG. 7A. (Width) can be detected. That is, the control unit 11 of the robot 10 has its own device (the gaps formed on the right side and the left side of the obstacle OB, respectively, based on the camera image as shown in FIG. 7A and the sensor information of the sensor unit 15. It is possible to determine whether or not the robot 10) can pass through without touching the obstacle OB. Here, as shown in FIG. 7B, the control unit 11 of the robot 10 allows the robot 10 to pass through the gap formed on the right side of the obstacle OB, and can avoid the obstacle OB by route correction. It shall be judged. Then, as shown in FIGS. 7C and 6, the control unit 11 corrects the route passing through the right side of the obstacle OB with respect to the first route R1 which is the current planned movement route when viewed from the robot 10. The fifth route R5 (correction alternative route) performed can be acquired as a candidate for the avoidance route.

As described above, the control unit 11 of the robot 10 can acquire the static alternative route, the dynamic alternative route, and the correction alternative route as candidates for the avoidance route. The control unit 11 of the robot 10 may have a configuration in which all of the static alternative route, the dynamic alternative route, and the correction alternative route can be acquired as candidates for the avoidance route, or only a part of them can be acquired as candidates for the avoidance route. May be. Further, as shown in steps S105 to S108, the control unit 11 of the robot 10 also changes actions such as processing linked with the operation console 20 (operator OP) depending on whether or not the avoidance route can be acquired.

Here, the control unit 11 of the robot 10 determines an action (action according to the presence or absence of an incident (autonomous control stop incident)) according to the acquisition process of the avoidance route by the control unit 11 and the acquisition result of the avoidance route. A specific example will be described. Specifically, here, the control unit 11 will be described as determining an action according to the acquisition of the avoidance route and the acquisition result of the avoidance route in the policy shown by the matrix as shown in FIG.

In the policy shown in FIG. 8, the control unit 11 acquires the corrected alternative route, the static alternative route, and the dynamic alternative route in this order, and acquires the first acquired avoidable alternative route as the avoidance route. It is a policy to judge that it is unavoidable if the alternative route of is also unavoidable. In other words, in the policy shown in FIG. 8, the control unit 11 acquires the avoidance route candidates in the priority order of the correction alternative route, the static alternative route, and the dynamic alternative route.

In the policy shown in FIG. 8, in order from the upper line, if it can be avoided by the corrected alternative route, it cannot be avoided by the corrected alternative route, but if it can be avoided by the static alternative route, it cannot be avoided by the corrected alternative route and the static alternative route. Shows the "identification result" (whether or not avoidance is possible) when can be avoided by the dynamic alternative route and the case where it cannot be avoided by any alternative route, and the subsequent "action".

In the policy shown in FIG. 8 (policy on the first line), when the control unit 11 determines that the correction alternative route can be avoided, the control unit 11 moves to the avoidance position as an action and returns to the original movement route (for example, the correction alternative route). For example, a process of acquiring the fifth route R5) shown in FIG. 6 as an avoidance route and setting it as a planned movement route is performed. At this time, the control unit 11 notifies the operation console 20 (operator OP) of the detected obstacle information (hereinafter referred to as "obstacle information") and the fact that the user is moving on the correction alternative route. You may do it. The obstacle information includes, for example, a camera image of an obstacle or an obstacle name (for example, a cardboard box, a fallen tree, a trolley, a dangerous object, etc.) identified from the camera image of the obstacle (for example, identified by AI or the like). ) And information on the size of the obstacle (for example, width, height, depth, etc.) acquired from the measurement result of the sensor unit 15.

Further, in the policy shown in FIG. 8 (policy on the second line), when the control unit 11 determines that it cannot be avoided by the corrected alternative route but can be avoided by the static alternative route, the static alternative route (for example, for example) is used as an action. , The second route R2) shown in FIG. 4 is acquired as an avoidance route, and a process of setting it as a planned movement route is performed. At this time, the control unit 11 may notify the operation console 20 (operator OP) of obstacle information or the fact that the user is moving by a static alternative route.

Further, in the policy shown in FIG. 8 (policy on the third line), when the control unit 11 determines that it cannot be avoided by the corrected alternative route and the static alternative route but can be avoided by the dynamic alternative route, the control unit 11 dynamically substitutes as an action. A process of acquiring a route (for example, the fourth route R4 shown in FIG. 5) as an avoidance route and setting it as a planned movement route is performed. At this time, the control unit 11 may notify the operation console 20 (operator OP) of the obstacle information or the fact that the vehicle is moving by the dynamic alternative route.

Furthermore, in the policy shown in FIG. 8 (policy on the 4th line), when the control unit 11 determines that it cannot be avoided by any of the alternative routes, it determines that it cannot be avoided (determines that it is an incident detection), and takes action as an action. The movement is stopped on the spot, and obstacle information and information that the obstacle is unavoidable (that is unavoidable by any alternative route) are added to the operation console 20 (operator OP). It may be), and a process of requesting the operator OP to manually operate the own device (hereinafter referred to as "manual operation request") is performed. Further, when the control unit 11 receives a response to the manual operation request from the operation console 20, the control unit 11 transmits data on the surrounding situation (for example, data including a camera image and the measurement result of the sensor unit 15) to the operation console 20. It may be.

As described above, when the control unit 11 of the robot 10 determines that the obstacle cannot be avoided, the control unit 11 sends a manual operation request to the operation console 20 (operator OP) to manually operate the operation console 20 (operator OP). Performs the process of shifting to the operation mode (remote operation mode) for accepting the remote operation by (see step S108).

On the other hand, when the control unit 21 of the operation console 20 receives the manual operation request from the robot 10, it performs a process of outputting to that effect to the operator OP (for example, a process of displaying output by the display unit 16 or sound output by the speaker 17). Do). After that, when the control unit 21 of the operation console 20 receives the operation of the response to the manual operation request from the operator OP (when a predetermined operation to the operation unit 24 is received), the control unit 21 transmits the response to the manual operation request to the robot 10. A connection (connection for remote control) is established with the robot 10. As a result, the control unit 21 of the operation console 20 transmits a control signal (for example, a control signal relating to the moving direction and speed) to the robot 10 in response to an operation input (operation on the operation unit 24) from the operator OP. I do. On the other hand, when the robot 10 shifts to the remote operation mode, the control unit 11 of the robot 10 performs a process of controlling the moving unit 13 so as to move in response to a control signal from the operation console 20. At this time, the control unit 11 of the robot 10 acquires the image of the field of view in the moving direction (traveling direction) from the camera image captured by the camera unit 14 in real time and the sensor information acquired by the sensor unit 15 and displays it on the console 20. The image transmitted and acquired by the control unit 21 of the operation console 20 may be displayed on the display unit 23 (operation screen). At this time, the control unit 11 of the operation console 20 may display the screen as shown in FIG. 9 on the display unit 23, for example.

9 and 10 show screens displayed on the console 20 (display unit 23) when the robot 10 (control unit 11) finds an obstacle and switches to the remote control mode (hereinafter, “remote control screen””. It is a figure which showed the example of). 9 and 10 show an example of a remote control screen displayed on the console 20 when the robot 10 finds an obstacle OB existing in the position PD on the position PC and shifts to the remote operation mode as shown in FIG. Is shown.

As shown in FIG. 9, on the remote control screen, along with the camera image (image in the direction in which the robot 10 moves) captured by the robot 10, obstacle information (characters "cardboard length 30 cm x width 200 cm x height 130 cm" in FIG. 9). (Column) and sensor information (character string "temperature: 26 ° C., humidity: 45%, illuminance: 46 lp, atmospheric pressure: 990 hPa" in FIG. 9) may be added. As a result, the operator OP can perform remote control while viewing the camera image (video information) of the robot 10, obstacle information of obstacles found by the robot 10, and environmental information (temperature, etc.) of the position where the robot 10 exists. It can be carried out.

As the video information displayed on the remote control screen, the image captured by the camera unit 14 may be simply displayed as it is, or may be processed and displayed. On the remote control screen, for example, as shown in FIG. 10, a bird's-eye view image of the robot 10 generated (rendered) based on a camera image captured by the robot 10 (a free viewpoint image around the robot 10 and the robot 10). An image (so-called flying view image) obtained by rendering a graphic G101 (so-called avatar image) in the shape of the robot 10 may be displayed on the image (an image viewed from diagonally above). Regarding the process of rendering the Flying View image, for example, Reference 1 (Oki Electric Industry Co., Ltd. Hidetaka Komai, "Flying View", [Online], INTERNET, [Searched on October 8, 2019], <URL: https You may apply the description technology of //www.oki.com/jp/iot/doc/2018/img/ceatec/ceatec_flyingview.pdf>.

Further, the control unit 11 of the robot 10 may accept not only the manual operation reception of the operator OP but also the setting of a new planned movement route by the manual operation of the operator OP in the remote control mode.

Here, as a specific example, an example of an operation in which the robot 10 (control unit 11) accepts the setting of a new planned movement route by the manual operation of the operator OP in the remote control mode will be described with reference to FIG.

In the example of FIG. 11, an example is shown in which the robot 10 finds an obstacle OB existing in the position PD on the position PC while moving with the first route R1 as the planned movement route. In this case, the control unit 21 of the operation console 20 displays an operation screen (for example, an operation screen displaying the movement route information as shown in FIG. 11) capable of accepting the setting of a new movement route on the display unit 23. It may be displayed and a new planned travel route may be accepted. Here, as an example, it is assumed that the sixth route R6 shown in FIG. 11 is set as a new planned movement route by the operator OP. In this case, the control unit 21 of the console 20 transmits the movement route information including the new movement schedule route to the robot. Then, when the control unit 11 of the robot 10 receives the movement route information including the new movement schedule route in the remote control mode, the new movement schedule route is set and the movement control is started. For example, in the case of the example of FIG. 11, the control unit 11 of the robot 10 returns to the initial position PA and then starts moving toward the target position PB on the sixth route R6, which is a new planned movement route. You can do it. While moving along the manually set planned movement route, the control unit 11 of the robot 10 displays the camera image captured by the camera unit 25 (for example, the camera image of the field of view in the direction in which the robot 10 moves) on the console 20. For example, as shown in FIG. 12, the control unit 11 of the operation console 20 may display an operation screen for displaying the camera image transmitted from the robot 10 on the display unit 23. In the screen example of FIG. 12, the camera image when the robot 10 returns from the position PC to the position PA and starts moving on the sixth route R6, which is a new planned movement route (camera image in the direction in which the robot 10 moves). ) Is displayed. In the screen example of FIG. 12, the image captured by the camera unit 14 is simply displayed as it is, but a bird's-eye view image of the robot 10 generated (rendered) based on the camera image captured by the robot 10 (around the robot 10). As an operation screen that displays a rendered image (so-called flying view image) of a graphic (so-called avatar image) in the shape of the robot 10 on a free-viewpoint image of the robot 10 from an obliquely upward view). May be good.

(A-3) Effect of First Embodiment According to the first embodiment, the following effects can be obtained.

The robot 10 (control unit 11) of the first embodiment recognizes an obstacle, and if it cannot be avoided, it requires the intervention of a human (operator OP), so that an incident (autonomous control stop incident) is detected. To detect. Then, when the robot 10 detects an incident, the robot 10 notifies the operation console 20 used by the operator OP to that effect. As a result, in the first embodiment, the operation is taken over by the operator OP only when autonomous control by the robot 10 (control unit 11) is not possible (when human intervention is essential), so that the workload of the operator OP is taken over. It has the effect of reducing the amount of water.

In addition, the robot 10 (control unit 11) of the first embodiment also notifies the operation console 20 of information regarding the cause (source) of the incident (for example, information regarding obstacles). As a result, in the first embodiment, the operator OP can smoothly take over the operation (operation of the robot 10), and further, by confirming the obstacle in real time, it is possible to respond to the incident in detail according to the situation. Can be made to.

(B) Second Embodiment Hereinafter, a second embodiment of the moving object system, the moving object, the control program, and the control method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the moving object system and the moving object of the present invention are applied to the robot system and the robot will be described.

(B-1) Configuration of Second Embodiment FIG. 13 is a block diagram showing an example of the overall configuration of the robot system 1A in the second embodiment. In FIG. 13, the same reference numerals or corresponding reference numerals are given to the same portions or corresponding portions as those in FIG. 1 described above. The reference numerals in parentheses in FIG. 13 are reference numerals used only in the third embodiment described later.

Hereinafter, the second embodiment will be described focusing on the differences from the first embodiment.

In the robot system 1A of the second embodiment, the number of the robot 10 and the operating console 20 is a plurality. In the robot system 1A of the second embodiment shown in FIG. 13, N robots (N is an integer of 2 or more) and M robots 10 (10-1 to 10-N) (M is an integer of 2 or more). The console 20 (20-1 to 20-M) is arranged. In the second embodiment, the number (N) of the robots 10 may be one. Further, FIG. 13 illustrates an example in which M operator OPs (OP-1 to OP-N) use the corresponding consoles 20 (20-1 to 20-M). That is, in this embodiment, the operators OP-1 to OP-M use the consoles 20-1 to 20-M.

Each robot 10-1 to 10-N in the second embodiment is the same as the first embodiment except that a part of the processing of the control unit 11 is different. Further, the configurations of the operating consoles 20-1 to 20-N in the second embodiment are the same as those in the first embodiment except that a part of the processing of the control unit 21 is different.

Further, in the robot system 1 of the first embodiment, the occurrence of an obstacle (an event in which the movement on the planned movement route is hindered by the obstacle) is applied as an incident that the robot 10 should report to the operator OP (operation console 20). An example of doing so was explained. In the robot system 1A of the second embodiment, the robot 10 also detects an incident other than the occurrence of an obstacle (for example, an injured person or the like) and executes processing according to a scenario according to the type of the detected incident. To do. The method by which the control unit 11 detects the injured or sick as an incident is not limited, but for example, a matching person may be detected from the camera image captured by the camera unit 14 by AI processing or the like.

Further, in the robot system 1A of the second embodiment, when an incident is detected by the robot 10, an operator OP (operation console 20) for reporting the incident is selected based on a predetermined rule, and the selected operator is selected. Performs a process of notifying the OP (operation console 20). For example, when an incident is detected by the control unit 11 of the robot 10, the state of each operator OP (hereinafter referred to as "operator state") and preset information about each operator OP (for example, the ability of each operator, etc.) It is assumed that the operator OP for notifying the incident is selected (sorted) in consideration of information such as roles and responsibilities; hereinafter referred to as "operator information"). Each operator OP state is estimated based on, for example, an image of the operator OP captured by the camera unit 25 of the console 20 and a result (sensor information) when the sensor unit 26 is detected for the operator OP. It may be information including the mental and physical state of the operator OP and the task currently being executed by the operator OP (for example, remote control of another robot 10).

The mental and physical state of the operator OP is, for example, the emotional state analyzed from the facial image of the operator OP (for example, the degree of emotions), the state of the pupil (for example, the position and size of the pupil), and the like. The degree of work concentration to be analyzed can be mentioned.

As described above, the control unit 11 may select an appropriate operator OP for the incident (for example, the incident type) based on the operator state, operator information, and the like of each operator OP.

In the robot system 1A, the configuration in which the operator information of each operator OP and the information on the operator state are shared by each robot 10 (control unit 11) is not limited. For example, as operator information regarding each operator OP, the ability of each operator OP (for example, "what kind of abnormality is suitable for dealing with", "proficiency in operation of robot 10, strength in scene, weakness", " Information about "Do you have security knowledge", "Do you have knowledge about life-saving emergency", etc.) is registered in a database (for example, a database server (not shown) that can be connected to the IP network NW) and each is registered. The information may be shared by the robot 10 (control unit 11).

Further, in the robot system 1A, the control unit 21 of each console 20 captures sensor information (sensor information as a result of detecting the operator OP) by the sensor unit 26 and an image captured by the camera unit 25 (image captured by the operator OP). Operator information is acquired based on the above, and the operator information (operator information of each operator OP) acquired by each operation console 20 is transferred to a database (for example, a database server (not shown) that can be connected to the IP network NW) in real time. The information may be updated and registered so that the information can be shared by each robot 10 (control unit 11).

Then, the control unit 11 of each robot 10 may acquire the operator information and the operator state of each operator OP, compare them with the incident (type of detected abnormality, etc.), and select an appropriate operator OP. .. For example, suppose a robot 10 is deployed for security operations. At this time, the robot 10 (control unit 11) possesses security knowledge based on the incident (for example, when an abnormality such as a fallen human being is detected) and the operator information of each operator OP, and the local map information. You can select the operator OP who knows. Further, at this time, the robot 10 (control unit 11) has an operator OP whose emotions are stable (for example, the bias of emotions is less than a predetermined value and the work concentration ratio is less than a predetermined value) based on the operator state of each operator OP. It is possible to select an operator OP) that is greater than or equal to a predetermined value.

In the robot system 1A, the control unit 11 of each robot 10 selects (sorts) the operator OP when an incident is detected (hereinafter, referred to as "operator selection process" or "operator sorting process". ) May be performed on the control unit 11 side of each robot 10, or may be performed on a server (a server connected to the IP network NW) (not shown).

(B-2) Operation of the Second Embodiment Next, the operation of the robot system 1A of the second embodiment having the above configuration (control method according to the embodiment) will be described.

FIG. 14 is a flowchart showing the operation of the robot 10 in this embodiment.

FIG. 14 is a flowchart showing an operation (process executed by the control unit 11) when the robot 10 moves from the start position to the first target position in the process of operation.

When the control unit 11 of the robot 10 starts moving from the start position to the first target point, the control unit 11 first calculates a planned movement route for reaching the first target position from the start position (S201).

Next, the control unit 11 controls the moving unit 13 so as to move according to the calculated movement scheduled route (S202).

At this time, the control unit 11 confirms whether or not the first target position has been reached (S203) and detects the presence or absence of an incident (S204) while moving. At this time, when the moving unit 13 confirms the arrival at the first target point, the processing of the flowchart is terminated, and when an incident is detected, the moving unit 13 proceeds to step S205 described later and moves to the first target point. If it has not arrived and no incident has been detected, the operation is performed from step S202 described above, and the movement on the planned movement route is continued.

On the other hand, when an incident is detected while moving to the first target position, the control unit 11 determines a scenario (action content) according to the type of the detected incident (S205).

For example, it is assumed that the control unit 11 can hold data (hereinafter, referred to as “scenario definition data” or “action content definition data”) that defines a scenario (action content) corresponding to each type of incident. The scenario definition data may be held by the control unit 11 itself, or may be downloaded and held by the control unit 11 from a server (not shown).

Here, it is assumed that the control unit 11 has detected an injured person (for example, a person who has fallen down) as an incident. Then, here, it is assumed that the control unit 11 holds the scenario definition data corresponding to the incident of the injured person. Then, in the scenario definition data corresponding to the incident of the injured person, a process of moving to the vicinity of the incident source (injured person) (for example, a position of about 1 m from the face of the incident source) and notifying the operator OP is defined. It is assumed that there is.

When the control unit 11 starts an action based on the acquired scenario definition data, it first determines whether it can move to the vicinity of the incident source (for example, a distance of about 1 m) (S206), and if it can move, it will be described later. The process proceeds to step S207, and if not, the process proceeds to step S211 described later.

When it is determined in step S206 described above that the vehicle can move to the vicinity of the incident source, the control unit 11 calculates a planned movement route from the current position to reach the vicinity of the incident source (S207), and newly calculates the route to be moved. The neighborhood of the incident source is set as the second target position according to the planned movement route (S208).

Next, the control unit 11 controls the moving unit 13 so as to move toward the second target position according to the calculated movement scheduled route (S209).

At this time, the control unit 11 confirms whether or not the second target position has been reached (S210) while moving. At this time, when the control unit 11 confirms that it has reached the second target point, it proceeds to step S211 described later, and if it has not reached the second target point, it operates from step S209 described above. Continue moving on the planned movement route.

When the process proceeds from step S210 or step S208 to step S211 above, the control unit 11 performs a process of selecting one of the operator OPs (operation console 20) for the notification destination of the incident (S211).

Then, the control unit 11 notifies the operation console 20 used by the selection operator OP about the incident (S212), and confirms whether or not there is a response (response to the notification) from the operation console 20 (S213). At this time, the control unit 11 controls the moving unit 13 to temporarily stop the movement.

Then, when there is a response to the notification from the operation console 20, the control unit 11 proceeds to step S214, which will be described later, and when there is no response to the notification (for example, when there is no response within a certain time from the immediately preceding notification), the control unit 11 proceeds to step S214. Returning to step S212 described above, the notification process is performed again.

When the response from the console 20 is received in step S213 described above, the control unit 11 shifts to the remote control mode that operates in response to the control signal from the console 20, and the response console 20 is displayed. The operation right is transferred, and the processing of the flowchart is terminated.

(B-3) Effect of Second Embodiment According to the second embodiment, the following effects can be obtained.

In the second embodiment, the operator information and the operator status are stored in a database, and the operator OP (operation console 20) of the incident notification destination is selected by comparing with the type of incident detected by the robot 10 (control unit 11) (control unit 11). Changed). As a result, in the second implementation Keita, the switching to the operation of the operator OP becomes smooth, and the time required for the operation of the operator OP after the switching becomes shorter.

(C) Third Embodiment Hereinafter, a third embodiment of the moving object system, the moving object, the control program, and the control method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the moving object system and the moving object of the present invention are applied to the robot system and the robot will be described.

(C-1) Configuration of Third Embodiment The configuration of the robot system 1B of the third embodiment can also be shown with reference to FIG. 13 described above. The reference numerals in parentheses in FIG. 13 are reference numerals used only in the third embodiment.

Hereinafter, the configuration of the robot system 1B of the third embodiment will be described as being different from that of the second embodiment.

Each robot 10-1 to 10-N in the third embodiment is the same as the second embodiment except that a part of the processing of the control unit 11 is different. Further, the configurations of the operating consoles 20-1 to 20-N in the third embodiment are the same as those in the second embodiment except that a part of the processing of the control unit 21 is different.

Specifically, in the control unit 11 of the robot 10 of the third embodiment, the robot 10 (control unit 11) autonomously performs communication processing (for example, operator OP and / or incident) according to the type of incident that has occurred. It differs from the second embodiment in that it performs a process of inducing an incident response (solution) to the source.

For example, when the type of the detected incident is "discovery of a victim (for example, a fallen person)", the control unit 11 of the robot 10 captures a camera image of the incident source (victim) with the camera unit 25. Or, analysis processing may be performed based on the sensor information detected by the sensor unit 26, and information on the analysis result (hereinafter, referred to as “incident source analysis information”) may be acquired. Then, when the robot control unit 11 notifies the operator OP (operation console 20) of the incident, the robot control unit 11 also performs a process of transmitting the acquired incident source analysis information. Specifically, for example, when the incident source (incident type) is an injured person, the sensor unit 15 measures the state of the injured person such as pulse and body temperature and acquires it as incident source analysis information, and the operator OP ( The operation console 20) may be notified. Further, for example, when the control unit 11 detects an abnormality due to a natural disaster such as smoke due to a fire, a falling object due to an earthquake, a rain leak, flooding, a fallen tree, or a landslide, the camera unit 25 takes an image of the incident source. The range, severity, and the like may be sensed based on the camera image and the sensor information detected by the sensor unit 26, acquired as incident source analysis information, and notified to the operator OP (operation console 20).

Further, for example, when the incident type is "discovery of suspicious person", "discovery of lost child", or the like, the control unit 11 calls out to the operator OP to the incident source (for example, suspicious person or lost child) (for example). The process of inducing the conversation) may be performed. For example, when the incident type is the discovery of a lost child or a suspicious person, the control unit 11 autonomously informs the operator OP (operation console 20) of the incident type (for example, information that a lost child or a suspicious person has been found). The communication unit 12, the camera unit 14, the speaker 17, and the microphone 18 are controlled so as to notify and establish a video call with the operator OP (operation console 20), and between the operator OP and the incident source. Video calls may be enabled. As a result, the robot 10 (control unit 11) can guide the operator OP to speak, talk, guide, or the like to the incident source (for example, a lost child or a suspicious person).

The method by which the control unit 11 detects a lost child or a suspicious person as an incident is not limited, but for example, a preset person image (for example, features of a face or clothes) is obtained from a camera image captured by the camera unit 14. Or, a person who matches the behavior) may be detected. In addition, the control unit 11 can be used as an incident source in consideration of the state in which the robot 10 is placed, such as outdoors or indoors, an exclusion zone, a place where valuables are placed, etc., based on map information of the surrounding area. The person may be detected.

In addition, when detecting an incident or identifying an avoidance route (dynamic alternative route, etc.), the control unit 11 has dynamic map information (for example, an escalator / elevator stopped in the facility or under construction) around the planned movement route. Information on impassable areas such as blocked roads) may be considered. For example, the control unit 11 holds dynamic map information from the outside (for example, a server that holds security information such as an impassable part (not shown)) and reflects it in the map information held by its own device. , It may be used for incident detection and avoidance route identification. For example, the control unit 11 may identify the avoidance route (dynamic alternative route, etc.) using only the passages / roads that can be passed at the current time based on the dynamic map information. Further, even if there are roads that are blocked during construction, the control unit 11 autonomously controls them without detecting them as incidents if they match the positions that can be confirmed in advance by the dynamic map information. May be continued. By preventing the robot 10 from generating an incident based on a scheduled construction work or the like, it is possible to promote the efficient work of the operator OP.

(C-2) Operation of the Third Embodiment Next, the operation of the robot system 1B of the third embodiment having the above configuration (control method according to the embodiment) will be described.

FIG. 15 is a flowchart showing a first example of the operation of the robot 10 in the third embodiment. In the flowchart of FIG. 15 (first example), a scenario in which the control unit 11 of the robot 10 performs sensing processing on an incident source according to the type of incident, acquires sensing analysis information, and notifies the operator OP. An example of executing is shown.

The difference between the flowchart of FIG. 15 and the second embodiment (the flowchart of FIG. 14 described above) will be described. In the flowchart of FIG. 15, the same reference numerals (step numbers) are assigned to the operations similar to those of the second embodiment.

The flowchart shown in FIG. 15 is different from the second embodiment (flow chart of FIG. 14) in that steps S205 and S212 are replaced with steps S301 and S302, respectively, and step S302 is inserted before step S211. There is.

When an incident is detected while moving to the first target position in step S301, the control unit 11 determines a scenario (action content) according to the type of the detected incident. Here, it is assumed that the control unit 11 has detected an injured person (for example, a person who has fallen down) as an incident. Then, here, it is assumed that the control unit 11 holds the scenario definition data corresponding to the incident of the injured person. Then, in the scenario definition data corresponding to the incident of the victim, the user moves to the vicinity of the incident source (victim) (for example, about 1 m from the face of the incident source) and senses the incident source (victim). It is assumed that the process of performing the process, acquiring the sensing source analysis information, and notifying the operator OP (notifying together with the sensing source analysis information) is defined.

Then, when the control unit 11 reaches the second target position (near the incident source) according to the scenario definition data in step S302, the control unit 11 controls the camera unit 25 and the sensor unit 26 to perform sensing processing on the incident source. Acquire sensing source analysis information.

Then, in step S303, the control unit 11 performs a process of notifying the operation console 20 of the selected operator OP together with the sensing source analysis information. At this time, the control unit 21 of the console 20 can notify the operator OP that the robot 10 has notified the operator OP together with the received sensing source analysis information. As a result, the operator OP can perform a quick and detailed response (for example, calling out to the injured or sick or requesting an ambulance) based on the sensing source analysis information.

FIG. 16 is a flowchart showing a second example of the operation of the robot 10 in the third embodiment. In the flowchart of FIG. 16 (second example), a scenario in which the control unit 11 of the robot 10 guides a conversation (for example, a video call) between the operator OP and the person at the incident source according to the type of incident. An example of executing it is shown.

The difference between the flowchart of FIG. 16 and the second embodiment (the flowchart of FIG. 14 described above) will be described. In the flowchart of FIG. 16, the same reference numerals (step numbers) are assigned to the operations similar to those of the second embodiment.

The flowchart shown in FIG. 16 is different from the second embodiment (flow chart of FIG. 14) in that steps S205 and subsequent steps (steps S205 to S214) are replaced with steps S401 to S405.

When an incident is detected while moving to the first target position in step S401, the control unit 11 determines a scenario (action content) according to the type of the detected incident. Here, it is assumed that the control unit 11 detects a lost child (for example, a person whose degree of confusion is equal to or higher than a predetermined value and is 6 years old or younger from facial feature analysis or facial expression analysis) as an incident. Then, here, it is assumed that the control unit 11 holds the scenario definition data corresponding to the incident of the lost child. Then, in the scenario definition data corresponding to the incident of the lost child, it is assumed that the process of inducing the conversation (video call) between the operator OP and the incident source (lost child) is defined.

Next, the control unit 11 selects the operator OP for the incident (the same process as in step S211 described above) (S402).

Next, the control unit 11 notifies the selected operator OP (operation console 20) of the incident type (for example, information that a lost child has been found) (S403), and responds (to the notification) from the operation console 20. The presence or absence of (response) is confirmed (S404).

When there is a response from the console 20, the control unit 11 shifts to the remote control mode that operates in response to the control signal from the console 20, transfers the operation right to the responding console 20, and at the same time. The communication unit 12, the camera unit 14, the speaker 17, and the microphone 18 are controlled so that a video call is established between the incident source (lost child) and the operator OP (control console 20) (S405). As a result, it is possible to smoothly guide the operator OP to speak or talk to the incident source (lost child) via the robot 10.

(C-3) Effect of Third Embodiment According to the third embodiment, the following effects can be obtained.

In the third embodiment, the robot 10 (control unit 11) performs communication processing according to the detected incident (for example, processing for inducing the operator OP and / or the incident source to deal with (solve) the incident). As a result, for example, when the operation is transferred from the robot 10 to the operator OP, the time for responding to the incident (for example, calling out to a lost child or a suspicious person or checking the situation) can be shortened, and the efficient operator OP can be used. Work can be encouraged.

(D) Other Embodiments The present invention is not limited to each of the above embodiments, and modified embodiments as illustrated below can also be mentioned.

(D-1) In the third embodiment, a plurality of examples of the operator OP, the robot 10 and the console 20 have been described, but as in the first embodiment, the operator OP, the robot 10 and the console 20 have been described, respectively. It may be singular.

(D-2) In the above embodiment, two or more robots 10 may be connected and operated, and may be separated from each other as needed. For example, a second robot 10 that normally moves on the ground is mounted on a second robot 10 that can move in the air with rotary wings and is moved, and when necessary (for example, the incident source in the first robot 10). The second robot 10 may be separated and operated (for example, moved from the air to the vicinity of the incident source) when the movement path to the robot 10 cannot be secured. At this time, the first robot 10 (control unit 11) autonomously gives an operation to the second robot 10 (control unit 11) (the operation is directly performed on the second robot 10 without going through the operator OP). Give).

1, 1A, 1B ... Robot system, 10, 10-1 to 10-N ... Robot, 11 ... Control unit, 12 ... Communication unit, 13 ... Moving unit, 14 ... Camera unit, 15 ... Sensor unit, 16 ... Display unit , 17 ... speaker, 18 ... microphone, 20, 20-1 to 20-N ... console, 21 ... control unit, 22 ... communication unit, 23 ... display unit, 24 ... operation unit, 25 ... camera unit, 26 ... sensor Department, 27 ... Speaker, 28 ... Microphone, NW ... IP network, OP ... Operator.

Claims (13)

In a moving object system having a moving object and a terminal used by an operator
The moving object is
An information acquisition unit that acquires information related to the surroundings of the moving object,
A moving part that moves the moving object in an arbitrary direction,
Autonomous control for identifying whether or not to stop the autonomous control process by using the result acquired by the information acquisition unit in the process of the autonomous control process. A moving object system, characterized in that it has a control unit capable of notifying the terminal when the identification process is performed and, as a result of the autonomous control identification process, it is determined that the autonomous control process is not continued. When the control unit detects an autonomous control stop incident that should stop the autonomous control process by using the result acquired by the information acquisition unit in the process of the autonomous control process, the moving object is connected to the terminal. Requesting remote control A notification including a remote control request is given, and the moving object is controlled according to a control signal from the terminal.
The terminal outputs an operation unit capable of receiving an operation input from an operator and a control signal corresponding to the content of the operation input of the operation unit when the notification from the moving object includes the remote operation request. The moving object system according to claim 1, further comprising a remote control means for transmitting to a moving object. When there is an obstacle on the first route for reaching the target point while moving to the target point, the control unit acquires an alternative route other than the first route, and the acquired alternative route. When the avoidance route acquisition process of avoiding the obstacle and identifying and acquiring the avoidance route that can reach the target point is performed and the avoidance route cannot be acquired by the avoidance route acquisition process, the obstacle The moving object system according to claim 2, wherein it is determined that an autonomous control discontinuation incident originating from the above is detected. The moving object system includes a plurality of the terminals, each having a different operator.
When an autonomous control stop incident occurs, the control unit holds information about each operator, performs an operator selection process for selecting a destination operator to be notified based on the held information, and is selected as a result of the operator selection process. The moving object system according to any one of claims 1 to 3, wherein the terminal used by the operator is notified of the autonomous control cancellation incident. The moving object according to claim 4, wherein the control unit selects a notification destination operator for notifying an autonomous control stop incident in consideration of the attribute information of each operator in the operator selection process. system. Each of the terminals further comprises an operator detecting means for detecting an operator state indicating the state of the operator to be used.
The moving object system according to claim 4 or 5, wherein the control unit selects an operator to be notified of an autonomous control stop incident in consideration of the operator state of each operator. When the control unit detects an autonomous control stop incident in the process of the autonomous control process, the control unit can perform communication processing with the operator via the terminal according to the type of the autonomous control stop incident. The moving object system according to any one of claims 1 to 6. The control unit moves to the vicinity of the source of the autonomous control stop incident according to the type of the detected autonomous control stop incident, and the incident source information regarding the source is based on the result acquired by the information acquisition unit. The moving object system according to claim 7, wherein the communication process of acquiring the information and notifying the terminal can be executed. The claim is characterized in that the control unit can execute a communication process for inducing a call between the source of the autonomous control cancellation incident and the operator according to the type of the detected autonomous control cancellation incident. Item 7. The moving object system according to item 7. The information acquisition unit includes an imaging unit that images the surroundings of the moving object, and a sensor unit that senses the surroundings of the moving object using one or a plurality of sensors. The moving object system according to any one of 1 to 9. In moving objects
The information acquisition department that acquires information related to the surroundings of the moving object,
A moving part that moves the moving object in any direction,
Autonomous control for identifying whether or not to stop the autonomous control process by using the result acquired by the information acquisition unit in the process of the autonomous control process. It is characterized by having a control unit capable of notifying an external terminal used by the operator when the identification process is performed and, as a result of the autonomous control identification process, it is determined that the autonomous control process is not continued. Moving objects. A computer mounted on a moving object having an information acquisition unit for acquiring information related to the surroundings of the own device and a moving unit for moving the own device in an arbitrary direction.
Autonomous control for identifying whether or not to stop the autonomous control process by using the result acquired by the information acquisition unit in the process of the autonomous control process. When the identification process is performed and, as a result of the autonomous control identification process, it is determined that the autonomous control process is not continued, the control unit functions as a control unit capable of notifying an external terminal used by the operator. Control program. In the method of controlling moving objects
The moving object has an information acquisition unit, a moving unit, and a control unit.
The information acquisition unit acquires information related to the surroundings of the moving object, and obtains information about the surroundings of the moving object.
The moving unit moves the moving object in an arbitrary direction.
The control unit performs autonomous control processing of the moving object, and whether or not to stop the autonomous control processing by using the result acquired by the information acquisition unit in the process of the autonomous control processing. When it is determined that the autonomous control processing is not continued as a result of the autonomous control identification processing, it is possible to notify an external terminal used by the operator. Control method to do.

Images