WO2020203341A1

WO2020203341A1 - Control device, control method, and program

Info

Publication number: WO2020203341A1
Application number: PCT/JP2020/012239
Authority: WO
Inventors: 啓輔前田; 隆盛山口
Original assignee: ソニー株式会社
Priority date: 2019-04-02
Filing date: 2020-03-19
Publication date: 2020-10-08

Abstract

The present technique relates to a control device, a control method, and a program that allow a moving body to move within a range in predetermined specifications when an AI is used to realize the movement of the moving body. The control device to which the present technique is applied uses, as an input, input information including at least information related to a trajectory and information related to an obstacle and adjusts, depending on the situation of a moving body, the input information to be input to an inference model that outputs output information related to movement following the trajectory. The control device inputs the adjusted input information to the inference model to generate output information, and controls movement of the moving body on the basis of the output information generated using the inference model. The present technique is applicable to a robot capable of movement.

Description

Controls, control methods, and programs

The present technology relates to control devices, control methods, and programs, and in particular, when the movement of a moving body is realized by using AI, the control device that allows the movement of the moving body to be performed within a predetermined specification. Regarding control methods and programs.

Conventionally, there are moving objects that move autonomously by creating an environment map by sensing the surrounding environment. Moving objects include automobiles, robots, and airplanes.

Japanese Unexamined Patent Publication No. 2013-31897 Japanese Unexamined Patent Publication No. 2013-22705 Japanese Unexamined Patent Publication No. 2012-236244

Among the moving bodies that move autonomously, there are moving bodies equipped with so-called AI (Artificial Intelligence) that realizes movement using a neural network generated by performing machine learning such as deep learning.

It is conceivable to control the movement of a moving object by using a neural network that inputs information on the surrounding environment and outputs information indicating how to move. In this case, since the internal processing of the neural network can be said to be a black box, there is a risk that the movement as expected by the designer cannot be guaranteed.

This technology was made in view of such a situation, and when the movement of a moving body is realized by using AI, the movement of the moving body is performed within a predetermined specification. ..

The control device of one aspect of the present technology inputs input information including at least information about an orbit and information about an obstacle, and inputs the input information to an inference model that outputs output information about movement following the orbit. An input adjustment unit that adjusts according to the situation of the moving body, a generation unit that generates the output information using the inference model, and the moving body based on the output information generated by using the inference model. It is provided with a movement control unit that controls the movement of the.

In one aspect of the present technology, input information including at least information about an orbit and information about an obstacle is input, and input information input to an inference model that outputs output information about movement following the orbit is a moving body. The output information is generated using the inference model, and the movement of the moving object is controlled based on the output information generated using the inference model.

It is a figure which shows the use state of the customer service system which concerns on one Embodiment of this technique. It is a figure which shows the example of the posture of a customer service robot. It is an enlarged view which shows the customer service robot at the time of interaction. It is a figure which shows by disassembling the housing. It is a figure which shows the example of the trajectory plan. It is a figure which shows the example of the correction of an orbit. It is a figure which shows the example of the trajectory tracking model which a customer service robot has. It is a figure which shows the example of the movement of a customer service robot. It is a figure which shows the adjustment example of the movement of a customer service robot. It is a block diagram which shows the configuration example of the customer service system. It is a block diagram which shows the functional structure example of a control part. It is a figure which shows the example of the area set in the vicinity of a goal position. It is a figure which shows the example of the area set in the vicinity of a goal position. It is a flowchart explaining the movement control processing. It is a flowchart explaining the arrival determination process # 1 performed in step S7 of FIG. It is a figure which shows the state that the customer service robot is in a relaxation area. It is a figure which shows the example of deceleration of a translational speed. It is a flowchart explaining the arrival determination process # 2 performed in step S7 of FIG. It is a flowchart explaining the trajectory follow-up process performed in step S8 of FIG. It is a figure which shows the state that the customer service robot is in the deceleration start area. It is a block diagram which shows the configuration example of a computer.

Hereinafter, modes for implementing the present technology will be described. The explanation will be given in the following order.
1. 1. Applications of customer service system 2. About the movement of the customer service robot 1 using AI 3. Configuration example of customer service system 4. Operation of customer service robot 1 5. Modification example

<Use of customer service system>
FIG. 1 is a diagram showing a usage state of a customer service system according to an embodiment of the present technology.

The customer service system of FIG. 1 is used, for example, indoors. There are people (users) in the space where the customer service system is installed.

As shown in FIG. 1, a plurality of cubic customer service robots are prepared in the room. In the example of FIG. 1, three customer service robots 1-1 to 1-3 are shown. When it is not necessary to distinguish between the customer service robots, they are collectively referred to as the customer service robot 1.

The customer service robot 1 is a moving body that moves on the floor surface. On the bottom surface of the customer service robot 1, a configuration such as a tire used for moving the customer service robot 1 is provided.

The customer service robot 1 has a function of searching for a user in the room based on an image taken by a camera or the like, and approaching the user detected by the search to serve the customer. For example, the customer service robot 1 performs customer service for answering the questionnaire. The customer service system using the customer service robot 1 is used, for example, in an exhibition venue, a concert venue, a movie theater, an amusement facility, or the like.

FIG. 2 is a diagram showing an example of the posture of the customer service robot 1.

The state of the customer service robot 1 shown in A of FIG. 2 is the state at the time of movement. While moving to the destination, the customer service robot 1 moves in a substantially cubic state. Even when the robot 1 is waiting at a predetermined place without serving customers, the state of the customer service robot 1 is a substantially cubic state as shown in A of FIG.

The state of the customer service robot 1 shown in B of FIG. 2 is a state at the time of interaction, that is, when the customer is being served to the target user. When serving customers, the customer service robot 1 controls its own posture with the top plate raised in order to facilitate work on the top plate. The customer service robot 1 is provided with an arm portion for raising and lowering the top plate.

FIG. 3 is an enlarged view of the customer service robot 1 at the time of interaction.

As shown by the broken line, the top plate 12 has a built-in data processing terminal 13 such as a tablet terminal having a display equipped with a touch panel. At the time of interaction, characters and images to be a questionnaire are displayed on the display provided in the range indicated by the broken line. The user inputs data such as an answer to a questionnaire by operating a button displayed on the display of the data processing terminal 13 with a finger.

In this way, the top plate 12 is used as a desk when the user performs work such as answering a questionnaire.

When the questionnaire is completed, the customer service robot 1 closes the upper surface of the housing 11 with the top plate 12 by lowering the top plate 12, and returns to the home position in a state like a simple box shown in A of FIG. ..

In this way, the customer service system of FIG. 1 is a system in which the customer service robot 1 that blends into the space like a mere box approaches the user and changes the posture as if asking the user to take a questionnaire.

The user who saw the top plate 12 of the customer service robot 1 moving near him / herself can intuitively confirm that he / she should answer the questionnaire. In addition, the user can answer the questionnaire in a form of communicating with the customer service robot 1.

FIG. 4 is an exploded view of the housing 11.

As shown in FIG. 4, panels 22-1 to 22-4 are attached to the side surfaces of the box-shaped main body 21. Panels 22-1 to 22-4 are resin panels that serve as, for example, half mirrors.

A depth camera 23 is provided above the front of the main body 21. The shooting by the depth camera 23 is performed through the panel 22-1 attached to the front. LiDAR 24 is provided below the front surface of the main body 21.

A columnar support arm 25 is provided on the upper surface of the main body 21. By expanding and contracting the support arm 25 or moving it in the vertical direction, the raising and lowering of the top plate 12 fixed to the upper end of the support arm 25 is controlled. Inside the main body 21, a drive unit such as a motor or a gear for expanding / contracting or moving the support arm 25 in the vertical direction is provided.

Inside the main body 21, a computer for performing various processes, a moving mechanism for tires, a power supply, and the like are also provided.

Each customer service robot 1 shown in FIG. 1 has the above configuration.

<About the movement of the customer service robot 1 using AI>
FIG. 5 is a diagram showing an example of a track plan.

The movement of the customer service robot 1 is performed according to the trajectory (route) planned by the customer service robot 1 itself. In the trajectory plan by the customer service robot 1, as shown by the broken line arrow in FIG. 5, the position P1 which is the self-position of the customer service robot 1 and the position P2 which is the position of the target user U1 are connected by a straight line to the position P1. It is done so as to set the coordinates and velocity at each position on the orbit between positions P2.

The coordinates are set based on, for example, a predetermined position in the space where the customer service robot 1 is prepared. The speed is the moving speed of the customer service robot 1, and is composed of a translation speed and an angular speed.

Although it is assumed that the trajectory composed of straight lines is planned, the trajectory consisting of a combination of straight lines and curves may be planned, or the trajectory composed only of curves may be planned. ..

The trajectory plan is repeated after the target user U1 is determined based on the image taken by the camera, for example, without changing the target. That is, the customer service robot 1 continues to recognize the user U1 based on an image or the like taken by the camera, and plans the trajectory so as to follow the user U1. In this way, the trajectory once planned is appropriately modified according to the situation at that time.

Here, the space in which the customer service robot 1 is prepared is a space in which there are a plurality of users other than the target user U1. The trajectory correction is also performed when a user other than the target user U1 is detected as an obstacle.

FIG. 6 is a diagram showing an example of orbit correction.

The situation shown in FIG. 6 is a situation in which the user starts moving according to the trajectory shown in FIG. 5 and an obstacle user is detected at the position P11 on the track. The position P11 is a position closer to the position P2, which is the position of the target user, than the position P1.

As shown by the alternate long and short dash line in FIG. 6, when an obstacle exists in the orbit, the orbit is modified so as to avoid the obstacle. In the example of FIG. 6, the vehicle advances to the position P12 on the orbit indicated by the broken line, and the obstacle avoidance is started at the position P12. After avoiding obstacles and moving to the position P13 on the original orbit, the movement is performed according to the original orbit indicated by the broken line.

In this way, the movement of the customer service robot 1 is performed so as to follow the original trajectory after avoiding the obstacle, although it once deviates from the trajectory in order to avoid the obstacle.

The movement of the customer service robot 1 that follows the trajectory is performed using AI (Artificial Intelligence). The customer service robot 1 is a device equipped with AI.

FIG. 7 is a diagram showing an example of a trajectory tracking model of the customer service robot 1.

The AI mounted on the customer service robot 1 is realized by the trajectory tracking model shown in FIG. The trajectory tracking model is an inference model composed of a neural network generated by performing machine learning such as deep learning.

As shown in FIG. 7, the trajectory tracking model is a model that inputs self-position information, obstacle map information, and trajectory information, and outputs velocity information.

The self-position information that is input to the trajectory tracking model is information that represents the self-position of the customer service robot 1 in the space where the customer service robot 1 is prepared.

The obstacle map information is information representing a map of the space where the customer service robot 1 is prepared, including information such as the position, size, and shape of the obstacle. The obstacle map information is generated based on, for example, a distance image generated by the depth camera 23 and distance information to an object measured by LiDAR24.

The trajectory information is information representing the trajectory planned by the customer service robot 1 itself.

On the other hand, the speed information that is the output of the trajectory tracking model is information that represents the speed of the customer service robot 1. The speed information represents the translational speed and the angular velocity at each position of the customer service robot 1.

The learning of the orbit tracking model takes, for example, self-position information, obstacle map information, and orbit information as inputs, and gives information for following the orbit and speed information for avoiding obstacles on the orbit. It is done by. That is, the learning of the trajectory tracking model is performed with trajectory tracking and obstacle avoidance as evaluation items.

By controlling the speed of the customer service robot 1 based on the speed information output from the trajectory tracking model, the customer service robot 1 follows the trajectory while avoiding obstacles as described with reference to FIG. Movement is realized.

It is difficult for a person to design a program that moves while avoiding obstacles in a complicated situation such as a crowded space. By using the orbit-following model, it is possible to move following the orbit while avoiding obstacles even in a complicated situation.

In this way, the movement of the customer service robot 1 is controlled according to the velocity information obtained by planning the trajectory to the position of the target user and inputting the information of the planned trajectory into the trajectory tracking model.

Although the orbit planning itself is not a black box, it can be said that it is a black box because the method of determining the actual movement method including avoidance of obstacles uses a model for orbit tracking.

If the movement is controlled by using the output of the trajectory tracking model as it is, the customer service robot 1 cannot be moved as expected by the designer because it is unknown what kind of output can be obtained from the trajectory tracking model. there is a possibility.

FIG. 8 is a diagram showing an example of movement of the customer service robot 1.

In FIG. 8, the colored user U1 is a target user. On the other hand, the users U11 and U12 are users who are not targets. Since the users U11 and U12 are treated as obstacles, the customer service robot 1 moves so as to avoid the users U11 and U12.

When the movement is controlled by using the output of the trajectory tracking model as it is, the customer service robot 1 treats not only the non-target user but also the target user U1 as an obstacle as shown in A of FIG. To avoid. As described above, the trajectory tracking model uses trajectory tracking and obstacle avoidance as evaluation items.

Further, as shown in B of FIG. 8, when the target user U1 approaches, the customer service robot 1 cannot reach the goal position and strays (wanders) in the vicinity of the goal position. Moving.

Therefore, in the customer service robot 1, as shown in A of FIG. 9, when the target user U1 is approached, the angular velocity is decelerated. In A of FIG. 9, the range indicated by the broken line circle is preset as the range in which the deceleration of the angular velocity is started.

As a result, it is possible to prevent the target user U1 from being avoided as shown in A of FIG.

Further, in the customer service robot 1, as shown in B of FIG. 9, when staying in the area near the target user U1 for a certain period of time, it is determined that the robot 1 has reached the goal. If it is determined that the robot has arrived at the goal, the movement of the customer service robot 1 is terminated, and an operation for interaction is performed. In B of FIG. 9, the range indicated by the broken line circle is set in advance with the goal position as the center as the range for determining that the goal has been reached.

As a result, straying in the vicinity of the goal position as shown in B of FIG. 8 is suppressed.

For the movement as shown in FIG. 9, the velocity information output from the trajectory tracking model is not used as it is, but the velocity information output from the trajectory tracking model is adjusted as appropriate, and the adjusted velocity information is used. It is realized by controlling the movement of the customer service robot 1.

In addition, the input to the trajectory tracking model is adjusted, and the inference by the trajectory tracking model is performed using the adjusted input.

A series of processes for controlling the movement of the customer service robot 1 according to such specifications will be described later with reference to the flowchart.

<Example of customer service system configuration>
FIG. 10 is a block diagram showing a configuration example of a customer service system.

As shown in FIG. 10, the customer service system includes a customer service robot 1 and a control device 71. The customer service robot 1 and the control device 71 are connected via wireless communication.

The customer service robot 1 is composed of a control unit 51, a moving unit 52, an elevating control unit 53, a camera 54, a sensor 55, a communication unit 56, and a power supply unit 57. As described above, the data processing terminal 13 is built in the top plate 12 of the customer service robot 1.

The control unit 51 is composed of a computer. The control unit 51 executes a predetermined program by the CPU and controls the overall operation of the customer service robot 1. The control unit 51 has a function as a control device that controls various operations of the customer service robot 1, including the movement of the customer service robot 1 as a moving body.

The moving unit 52 rotates the tires by driving the motor and gears, and realizes the movement of the customer service robot 1. The moving unit 52 functions as a moving unit that realizes the movement of the customer service robot 1 while controlling the moving speed and the moving direction (translation speed and angular velocity) according to the control by the control unit 51.

The elevating control unit 53 controls the expansion and contraction of the support arm 25 by driving a motor and gears.

The camera 54 is composed of the depth camera 23 of FIG. 4 that captures a distance image, an RGB camera that captures an RGB image, an IR camera that captures an IR image, and the like. The image taken by the camera 54 is output to the control unit 51.

The sensor 55 is composed of various sensors such as an acceleration sensor, a gyro sensor, a motion sensor, an encoder provided in the moving portion 52 for detecting the amount of rotation of the tire, and LiDAR24. Information representing the sensing result by the sensor 55 is output to the control unit 51.

At least one of the camera 54 and the sensor 55 may be provided outside the customer service robot 1. In this case, the image taken by the camera 54 provided outside the customer service robot 1 and the information representing the sensing result by the sensor 55 are transmitted to the customer service robot 1 via wireless communication.

The communication unit 56 performs wireless communication with the control device 71. The communication unit 56 receives, for example, the information transmitted from the control device 71 and outputs the information to the control unit 51.

The power supply unit 57 has a battery. The power supply unit 57 supplies power to each unit of the customer service robot 1.

The control device 71 is composed of a data processing device such as a PC. The control device 71 functions as a higher-level system that controls the behavior of each customer service robot 1. For example, the control device 71 instructs each customer service robot 1 to start searching for a user.

FIG. 11 is a block diagram showing a functional configuration example of the control unit 51.

At least a part of the functional units shown in FIG. 11 is realized by executing a predetermined program by the CPU of the computer constituting the control unit 51.

In the control unit 51, the environment sensing unit 101, the self-position identification unit 102, the obstacle map generation unit 103, the application unit 104, the trajectory planning unit 105, the arrival determination unit 106, the trajectory tracking unit 107, and the movement control unit 108 are realized. Will be done. The orbit following unit 107 is composed of an AI orbit following unit 111 and a monitoring unit 112.

The environment sensing unit 101 senses the environment around the customer service robot 1 by driving the camera 54 and the sensor 55. Environmental information representing the environmental sensing result is supplied to the self-position identification unit 102 and the obstacle map generation unit 103.

The environmental information includes, for example, a distance image taken by the depth camera 23 and distance information to an object measured by LiDAR24. The environmental information may include information representing the measurement results of each sensor constituting the sensor 55, such as information representing the amount of rotation of the tire detected by the encoder.

The self-position identification unit 102 identifies the self-position, which is the current position of the customer service robot 1, based on the environmental information supplied from the environment sensing unit 101. Identification of self-position is performed by dead reckoning or dead reckoning.

Dead reckoning is a method of estimating the self-position using the output of a sensor inside the robot, such as an axle encoder or IMU (Inertial Measurement Unit). On the other hand, star reckoning is a method of estimating the self-position based on the situation of the outside world, such as marker recognition using the camera 54 and recognition using LiDAR SLAM. Dead reckoning and star reckoning may be switched according to the position of the customer service robot 1.

The self-machine information including the self-position information representing the self-position identification result is supplied to the arrival determination unit 106 and the trajectory tracking unit 107. The own machine information also includes information indicating the speed of the customer service robot 1 as appropriate.

The obstacle map generation unit 103 generates an obstacle map, which is a map showing the arrangement status of obstacles existing in the surroundings, based on the environmental information supplied from the environment sensing unit 101. Obstacles in the surroundings include users. The obstacle map information representing the obstacle map generated by the obstacle map generation unit 103 is supplied to the trajectory tracking unit 107.

The application unit 104 searches for users and determines a target user. The application unit 104 outputs the information of the target user to the trajectory planning unit 105 and instructs the start of the movement. The information output to the trajectory planning unit 105 includes, for example, information on the distance and direction to the target user.

The trajectory planning unit 105 sets the position of the target user as the goal position and plans the trajectory to the goal position. As described above, the trajectory plan is performed by connecting the position of the customer service robot 1 and the position of the target user with a straight line and setting the coordinates and the speed at each position on the straight line. The track planning unit 105 outputs track information representing the planned track to the arrival determination unit 106.

The arrival determination unit 106 determines whether or not the goal has been reached based on the self-position represented by the self-position information supplied from the self-position identification unit 102.

FIG. 12 is a diagram showing an example of a region set in the vicinity of the goal position.

As shown in FIG. 12, two areas, a goal area A1 and a relaxation area A2, are set around the position P2 which is a position in front of the target user.

The goal area A1 is an area determined as having arrived at the goal when the customer service robot 1 enters the area. When the customer service robot 1 enters the goal area A1, the arrival determination unit 106 determines that the robot 1 has arrived at the goal.

On the other hand, the relaxation area A2 is an area determined as having reached the goal when the customer service robot 1 stays in the area for a certain period of time or longer. The relaxation region A2 is set as a region wider than the goal region A1 so as to include the goal region A1.

When the customer service robot 1 enters the relaxation area A2, the arrival determination unit 106 starts counting the staying time. When the customer service robot 1 stays in the relaxation area A2 for a certain period of time or longer, the arrival determination unit 106 determines that the robot 1 has arrived at the goal. The relaxation area A2 is an area in which the condition for determining that the goal has been reached is relaxed.

Further, the relaxation area A2 is an area where the translational speed is decelerated (decreased) when the customer service robot 1 enters the area. The deceleration of the translation speed is started, for example, after a certain period of time has elapsed from the customer service robot 1 entering the relaxation area A2.

When a certain time has passed since the customer service robot 1 entered the relaxation area A2, the arrival determination unit 106 decelerates the translation speed represented by the trajectory information supplied from the trajectory planning unit 105, and adjusts the translation speed. The orbit information including the information of is output to the orbit following unit 107.

In this way, the arrival determination unit 106 determines whether or not the goal has been reached, and receives the orbit information supplied from the orbit planning unit 105, which is an input to the orbit tracking model, according to the situation of the customer service robot 1. It has a function as an input adjustment unit for adjustment.

The AI orbit following unit 111 of the orbit following unit 107 has the above-mentioned orbit following model. The AI trajectory tracking unit 111 is supplied from the self-position information included in the self-machine information supplied from the self-position identification unit 102, the obstacle map information supplied from the obstacle map generation unit 103, and the arrival determination unit 106. The orbit information is input to the orbit tracking model. The AI trajectory tracking unit 111 outputs the speed information output from the trajectory tracking model to the monitoring unit 112 in response to inputting each information.

The AI trajectory tracking unit 111 inputs self-position information, obstacle map information, and trajectory information to the trajectory tracking model as input information, and uses the trajectory tracking model to generate velocity information as output information. Functions as. Instead of using all of the self-position information, the obstacle map information, and the orbit information as input information, the obstacle map information and the orbit information may be used as the input information, or the self-position information and the obstacle. Information different from the map information and the orbit information may be used as the input information.

The monitoring unit 112 adjusts the speed information supplied from the AI trajectory tracking unit 111 as the output of the trajectory tracking model. The speed information is adjusted by the monitoring unit 112 according to the distance from the self-position represented by the self-position information included in the self-position information supplied from the self-position identification unit 102 to the goal position. The velocity information is adjusted, for example, by decelerating (decreasing) the angular velocity of the translational velocity and the angular velocity represented by the velocity information supplied from the AI trajectory following unit 111.

FIG. 13 is a diagram showing an example of a region set in the vicinity of the goal position.

As shown in FIG. 13, two areas, a deceleration end area A11 and a deceleration start area A12, are set around the position P2 which is a position in front of the target user.

The deceleration end area A11 is an area where the deceleration of the angular velocity ends when the customer service robot 1 enters the area.

When the customer service robot 1 enters the deceleration end area A11, the monitoring unit 112 ends the deceleration of the angular velocity that started when the customer service robot 1 enters the deceleration start area A12. The deceleration of the angular velocity is performed when the customer service robot 1 is outside the deceleration end region A11 and inside the deceleration start region A12.

When the deceleration of the angular velocity is completed, the angular velocity of the customer service robot 1 becomes 0. The customer service robot 1 goes straight in the deceleration end area A11 while facing the target user.

On the other hand, the deceleration start area A12 is an area where the deceleration of the angular velocity is started when the customer service robot 1 enters the area. The deceleration start region A12 is set as a region wider than the deceleration end region A11 so as to include the deceleration end region A11. The deceleration start region A12 may be set as the same region as the relaxation region A2 of FIG. 12, or the deceleration start region A12 may be set as a different region.

When the customer service robot 1 enters the deceleration start area A12, the monitoring unit 112 starts decelerating the angular velocity. The deceleration of the angular velocity is performed so as to cancel the angular velocity represented by the velocity information output by the trajectory tracking model so that the customer service robot 1 approaches the target user while facing the target user.

The monitoring unit 112 outputs speed information including the adjusted angular velocity information to the movement control unit 108. The speed information output by the monitoring unit 112 includes information representing the adjusted angular velocity adjusted by the monitoring unit 112 and information representing the translational speed output by the trajectory tracking model of the AI trajectory tracking unit 111.

In this way, the monitoring unit 112 has a function as an output adjusting unit that monitors the output of the trajectory tracking model and adjusts the speed information output from the trajectory tracking model according to the situation of the customer service robot 1.

The movement control unit 108 controls the movement unit 52 according to the speed information supplied from the monitoring unit 112, and moves the customer service robot 1. The movement of the customer service robot 1 is controlled by the movement control unit 108 so as to move according to the adjusted angular velocity adjusted by the monitoring unit 112 and the translational speed output by the trajectory tracking model of the AI trajectory tracking unit 111. ..

The configuration of the control unit 51 as described above is such that the condition of determining that the robot has arrived at the goal is relaxed by the arrival determination unit 106 according to the situation of the customer service robot 1 such as the distance to the goal position.

Further, the configuration of the control unit 51 is such that the speed information input to the trajectory tracking model is adjusted by the arrival determination unit 106 according to the situation of the customer service robot 1.

Further, the configuration of the control unit 51 is such that the speed information output from the trajectory tracking model is adjusted by the monitoring unit 112 according to the situation of the customer service robot 1.

<Operation of customer service robot 1>
Here, the operation of the customer service robot 1 having the above configuration will be described.

-Movement control processing First, the movement control processing of the customer service robot 1 will be described with reference to the flowchart of FIG.

The processing of each step shown in FIG. 14 is not only performed in that order, but is appropriately repeated in parallel with other processing or before and after other processing. The same applies to the processing of each step shown in each flowchart after FIG.

In step S1, the environment sensing unit 101 senses the environment around the customer service robot 1.

In step S2, the self-position identification unit 102 identifies the self-position of the customer service robot 1 based on the environmental information output from the environment sensing unit 101.

In step S3, the obstacle map generation unit 103 generates an obstacle map showing the arrangement status of obstacles existing in the surroundings based on the environmental information output from the environment sensing unit 101.

In step S4, the application unit 104 searches for users in the vicinity.

In step S5, the application unit 104 determines one target user from the users found by the search.

In step S6, the trajectory planning unit 105 sets the goal position based on the user information output from the application unit 104, and plans the trajectory from its own position to the goal position. The orbit information representing the planned orbit is output to the arrival determination unit 106.

In step S7, arrival determination processes # 1 and # 2 are performed by the arrival determination unit 106. Arrival determination processes # 1 and # 2 determine whether or not the goal has been reached. When it is determined that the goal has been reached, the movement control process of FIG. 14 ends. Details of the arrival determination processes # 1 and # 2 will be described later with reference to the flowcharts of FIGS. 15 and 18, respectively.

In step S8, the trajectory tracking process is performed by the trajectory tracking unit 107. The trajectory tracking process adjusts the velocity information output from the trajectory tracking model. The details of the trajectory tracking process will be described later with reference to the flowchart of FIG.

In step S9, the movement control unit 108 controls the movement unit 52 according to the adjusted speed information to move the customer service robot 1. After that, the process returns to step S1 and the above processing is repeated.

-Arrival determination process Next, the arrival determination process # 1 performed in step S7 of FIG. 14 will be described with reference to the flowchart of FIG.

The arrival determination process # 1 shown in FIG. 15 and the arrival determination process # 2, which will be described later with reference to FIG. 18, are performed in parallel by the arrival determination unit 106.

In step S21, the arrival determination unit 106 acquires the own machine information output from the self-position identification unit 102.

In step S22, the arrival determination unit 106 determines whether or not the customer service robot 1 is in the relaxation area A2 (FIG. 12) based on the self-position information included in the own machine information.

When it is determined in step S22 that the customer service robot 1 is not in the relaxation area A2, the arrival determination unit 106 outputs the trajectory information including the translation speed information before adjustment to the trajectory tracking unit 107 in step S23. The trajectory information including the translation speed information that is not decelerated is output to the trajectory tracking unit 107 when, for example, it is determined for the first time that the customer service robot 1 is not in the relaxation region A2. After that, the process returns to step S21, and the above processing is repeated.

On the other hand, when it is determined in step S22 that the customer service robot 1 is in the relaxation area A2, in step S24, the arrival determination unit 106 determines whether or not a certain time has elapsed since the customer service robot 1 entered the relaxation area A2. To judge.

When it is determined in step S24 that a certain time has not passed since the customer service robot 1 entered the relaxation area A2, after the trajectory information including the translation speed information before adjustment is output in step S23, after step S21. Processing is repeated.

When it is determined in step S24 that a certain time has elapsed since the customer service robot 1 entered the relaxation area A2, the arrival determination unit 106 relaxes the goal condition in step S25.

For example, as shown in FIG. 16, when the customer service robot 1 is in the relaxation area A2 and the translational speed is sufficiently decelerated in the relaxation area A2, it is determined that the goal has been reached. The conditions are relaxed.

In step S26, the arrival determination unit 106 starts decelerating the translation speed. For example, the translational speed is decelerated by multiplying the translational speed by a predetermined coefficient (deceleration coefficient) less than 1.

FIG. 17 is a diagram showing an example of deceleration of translational speed.

The translational speed is decelerated, for example, for the translational speed at all positions on the orbit. By decelerating the translational speed by multiplying the same deceleration coefficient, the entire translational speed from the position in the relaxation region A2 to the position of the target user U1 is decelerated.

Returning to the description of FIG. 15, in step S27, the arrival determination unit 106 outputs the orbit information including the adjusted translation speed information to the orbit tracking unit 107.

In step S28, the arrival determination unit 106 detects the translation speed of the customer service robot 1 based on the information included in the own machine information.

In step S29, the arrival determination unit 106 determines whether or not the translational speed is sufficiently decelerated.

If it is determined in step S29 that the translational speed is not sufficiently decelerated, the process returns to step S21 and the above process is repeated.

On the other hand, if it is determined in step S29 that the translational speed is sufficiently decelerated, the arrival determination unit 106 determines in step S30 that the goal has been reached. After that, the customer service robot 1 stops in the vicinity of the target user and performs an operation for interaction.

By the above processing, it is assumed that the goal has been reached when a predetermined condition is satisfied, such as entering the relaxation area A2 and sufficiently decelerating the translational speed in the relaxation area A2 even if the goal area A1 is not entered. It is judged.

Also, when approaching the target user, the speed information obtained by decelerating the translation speed will be input to the trajectory tracking model. By decelerating the translation speed that is the input, the translation speed output from the trajectory tracking model is also decelerated, and as a result, movement that approaches the target user while decelerating the translation speed is realized.

By setting the distance at which the translational speed deceleration is started as a specification, the designer can control the movement of the customer service robot 1 by using the trajectory tracking model, in the vicinity of the target user. It is possible to prevent a movement that causes a sudden stop.

Next, the arrival determination process # 2 performed in step S7 of FIG. 14 will be described with reference to the flowchart of FIG.

In step S41, the arrival determination unit 106 acquires the own machine information output from the self-position identification unit 102.

In step S42, the arrival determination unit 106 determines whether or not the customer service robot 1 is in the goal area A1 based on the self-position information included in the own machine information.

When it is determined in step S42 that the customer service robot 1 is in the goal area A1, in step S43, the arrival determination unit 106 determines whether or not the translational speed of the customer service robot 1 is sufficiently decelerated.

If it is determined in step S43 that the translational speed is not sufficiently decelerated, the process returns to step S41 and the above process is repeated. Similarly, when it is determined in step S42 that the customer service robot 1 is not in the goal area A1, the process returns to step S41 and the above process is repeated.

On the other hand, if it is determined in step S43 that the translational speed is sufficiently decelerated, the arrival determination unit 106 determines in step S44 that the goal has been reached. After that, the customer service robot 1 stops in the vicinity of the target user and performs an operation for interaction.

-Orbit follow-up process Next, the trajectory follow-up process performed in step S8 of FIG. 14 will be described with reference to the flowchart of FIG.

In step S51, the trajectory following unit 107 acquires the own machine information output from the self-position identification unit 102.

In step S52, the AI trajectory tracking unit 111 inputs the self-position information, the obstacle map information, and the trajectory information included in the own machine information into the trajectory tracking model, and infers the speed. The speed information representing the speed of the inference result output from the trajectory tracking model is output to the monitoring unit 112.

In step S53, the monitoring unit 112 determines whether or not the customer service robot 1 is in the deceleration start area A12 based on the self-position information.

When it is determined in step S53 that the customer service robot 1 is not in the deceleration start area A12, the monitoring unit 112 outputs the speed information output from the trajectory tracking model to the movement control unit 108 as it is in step S54.

On the other hand, if it is determined in step S53 that the customer service robot 1 is in the deceleration start area A12, in step S55, the monitoring unit 112 starts decelerating the angular velocity output from the trajectory tracking model.

As shown in FIG. 20, when the customer service robot 1 is inside the deceleration start area A12 and outside the deceleration end area A11, the deceleration of the angular velocity is started. For example, the angular velocity is adjusted by multiplying the angular velocity by a predetermined coefficient (deceleration coefficient) less than 1.

In step S56, the monitoring unit 112 outputs the speed information including the adjusted angular velocity information to the movement control unit 108. The velocity information output from the monitoring unit 112 includes information on the adjusted angular velocity and information on the translational velocity output from the trajectory tracking model.

In step S57, the monitoring unit 112 determines whether or not the customer service robot 1 is in the deceleration end area A11 based on the self-position information.

If it is determined in step S57 that the customer service robot 1 is not in the deceleration end area A11, the process returns to step S53 and the above-mentioned process is repeated.

On the other hand, when it is determined in step S57 that the customer service robot 1 is in the deceleration end area A11, in step S58, the monitoring unit 112 outputs speed information with the angular velocity set to 0 to the movement control unit 108. After that, the process returns to step S8 of FIG. 14, and the subsequent processing is performed.

When the target user is approached by the above processing, the velocity information that cancels the angular velocity output by the trajectory tracking model is output to the movement control unit 108. By adjusting the angular velocity so as to cancel the angular velocity output by the orbit-following model, it is possible to prevent the movement from avoiding the target user, thereby making the target face-to-face. The movement that approaches the user is realized.

Finally, the angular velocity is set to 0, and the movement is realized so as to go straight toward the goal position.

In addition to such adjustment of the angular velocity, the translational velocity is adjusted by the arrival determination unit 106. As a result, when the target user is approaching, the translational velocity and each velocity are decelerated and finally. Specifically, the movement is realized so as to stop while facing each other.

By setting the distance to start deceleration as a specification, the designer can move so as to avoid the target user even when the movement of the customer service robot 1 is controlled by using the trajectory tracking model. Can be prevented from being done. In addition, the designer can stop the user while facing the user.

Through the above series of processes, when the customer service robot 1 is moved by using AI, it is possible to make the movement within a predetermined specification.

Normally, when controlling the movement of a robot using AI, it is necessary to relearn the trajectory tracking model when a movement that the designer does not expect occurs. By the above processing, the movement of the customer service robot 1 can be performed within the range of the specifications without performing re-learning.

<Modification example>
At least a part of the configuration of the control unit 51 shown in FIG. 11 may be realized in the control device 71 of FIG. In this case, the control device 71 transmits a command for controlling the movement to each customer service robot 1. Each customer service robot 1 moves according to the control by the control device 71.

The goal position was set targeting the user requesting the questionnaire, but other positions may be set as the goal position.

Velocity information representing translational velocity and angular velocity is output from the orbit tracking model, but various information (output information) related to movement that realizes movement following the orbit while avoiding obstacles is output. It may be done. For example, information indicating the acceleration and the moving direction may be output as output information from the trajectory tracking model, or information representing only the moving direction may be output as output information.

The technology of adjusting the input to the trajectory tracking model and the output of the trajectory tracking model and controlling the robot behavior using the adjusted output can also be applied when controlling the behavior of various robots other than movement. is there.

The AI of the robot is assumed to be realized by the inference model of the neural network generated by machine learning, but it may be realized by various inference models that input predetermined information and obtain a predetermined output. ..

The case where the movement of the customer service robot 1 is controlled by using AI has been described, but the above-mentioned technology can be applied to the movement of various moving objects such as automobiles, airplanes, and robots of other shapes.

-Computer configuration example The above-mentioned series of processes can be executed by hardware or software. When a series of processes are executed by software, the programs constituting the software are installed from the program recording medium on a computer embedded in dedicated hardware or a general-purpose personal computer.

FIG. 21 is a block diagram showing a configuration example of computer hardware that executes the above-mentioned series of processes programmatically.

When the movement of the customer service robot 1 is controlled by the control device 71, the control device 71 is configured by a computer as shown in FIG.

The CPU (Central Processing Unit) 1001, the ROM (Read Only Memory) 1002, and the RAM (Random Access Memory) 1003 are connected to each other by the bus 1004.

An input / output interface 1005 is further connected to the bus 1004. An input unit 1006 including a keyboard and a mouse, and an output unit 1007 including a display and a speaker are connected to the input / output interface 1005. Further, the input / output interface 1005 is connected to a storage unit 1008 including a hard disk and a non-volatile memory, a communication unit 1009 including a network interface, and a drive 1010 for driving the removable media 1011.

In the computer configured as described above, the CPU 1001 loads and executes the program stored in the storage unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004, thereby executing the above-mentioned series of processes. Is done.

The program executed by the CPU 1001 is recorded on the removable media 1011 or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting, and installed in the storage unit 1008.

The program executed by the computer may be a program in which processing is performed in chronological order in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program in which processing is performed.

In the present specification, the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a device in which a plurality of modules are housed in one housing are both systems. ..

The effects described in this specification are merely examples and are not limited, and other effects may be obtained.

The embodiment of the present technology is not limited to the above-described embodiment, and various changes can be made without departing from the gist of the present technology.

For example, this technology can have a cloud computing configuration in which one function is shared by a plurality of devices via a network and processed jointly.

In addition, each step described in the above flowchart can be executed by one device or shared by a plurality of devices.

Further, when one step includes a plurality of processes, the plurality of processes included in the one step can be executed by one device or shared by a plurality of devices.

-Example of combination of configurations This technology can also have the following configurations.

(1)
The input information including at least the information about the trajectory and the information about the obstacle is input, and the input information input to the inference model that outputs the output information about the movement following the trajectory is adjusted according to the situation of the moving body. Input adjustment unit and
A generation unit that generates the output information by inputting the adjusted input information to the inference model, and
A control device including a movement control unit that controls the movement of the moving body based on the output information generated by using the inference model.
(2)
The control device according to (1), further comprising a track planning unit for planning the trajectory from the position of the moving body to the goal position.
(3)
The control device according to (2), wherein the trajectory planning unit plans the trajectory connecting the position of the moving body and the goal position with a straight line.
(4)
The control device according to (2) or (3) above, wherein the goal position is the position of a target person.
(5)
The control device according to any one of (2) to (4), wherein the input adjusting unit adjusts the output information according to the distance to the goal position.
(6)
The information about the trajectory includes the translational velocity of the moving body and the angular velocity of the moving body.
The control device according to (5) above, wherein the input adjusting unit adjusts the translational speed of the moving body.
(7)
The control device according to (6) above, wherein the input adjusting unit reduces the translational speed of the moving body.
(8)
When the moving body stays in a range of a predetermined distance centered on the goal position for a certain period of time, the input adjusting unit determines that the moving body has arrived at the goal and inputs the input information to the inference model. The control device according to any one of (2) to (7) above.
(9)
The control device according to any one of (2) to (8), further comprising an output adjusting unit that adjusts the output information generated by using the inference model according to the distance to the goal position.
(10)
The output information includes the translational velocity of the moving body and the angular velocity of the moving body.
The control device according to (9) above, wherein the output adjusting unit adjusts the angular velocity of the moving body.
(11)
When the moving body enters the range of the first distance centered on the goal position, the output adjusting unit starts deceleration of the angular velocity of the moving body and sets it inside the range of the first distance. The control device according to (10) above, wherein when the moving body enters the range of the second distance, the angular velocity of the moving body is adjusted to 0.
(12)
The inference model is a model generated by machine learning using at least information about the orbit and information about the obstacle, and outputs the output information for avoiding the obstacle. 11) The control device according to any one of.
(13)
The control device
The input information including at least the information about the trajectory and the information about the obstacle is input, and the input information input to the inference model that outputs the output information about the movement following the trajectory is adjusted according to the situation of the moving body. And
The output information is generated by inputting the adjusted input information into the inference model.
A control method for controlling the movement of the moving body based on the output information generated by using the inference model.
(14)
On the computer
The input information including at least the information about the trajectory and the information about the obstacle is input, and the input information input to the inference model that outputs the output information about the movement following the trajectory is adjusted according to the situation of the moving body. And
The output information is generated by inputting the adjusted input information into the inference model.
A program for executing a process of controlling the movement of the moving body based on the output information generated by using the inference model.

1-1 to 1-3 Customer service robot, 11 housing, 12 top plate, 13 data processing terminal, 21 main body, 22-1 to 22-4 panel, 23 depth camera, 24 LiDAR, 25 support arm, 51 control unit , 52 moving unit, 53 elevating control unit, 54 camera, 55 sensor, 56 communication unit, 57 power supply unit, 71 control device, 101 environment sensing unit, 102 self-position identification unit, 103 obstacle map generation unit, 104 application unit, 105 orbit planning unit, 106 arrival judgment unit, 107 orbit tracking unit, 108 movement control unit, 111 AI orbit tracking unit, 112 monitoring unit

Claims

The input information including at least the information about the trajectory and the information about the obstacle is input, and the input information input to the inference model that outputs the output information about the movement following the trajectory is adjusted according to the situation of the moving body. Input adjustment unit and
A generation unit that generates the output information by inputting the adjusted input information to the inference model, and
A control device including a movement control unit that controls the movement of the moving body based on the output information generated by using the inference model.
The control device according to claim 1, further comprising a trajectory planning unit for planning the trajectory from the position of the moving body to the goal position.
The control device according to claim 2, wherein the trajectory planning unit plans the trajectory that connects the position of the moving body and the goal position with a straight line.
The control device according to claim 2, wherein the goal position is the position of a target person.
The control device according to claim 2, wherein the input adjusting unit adjusts the input information according to the distance to the goal position.
The information about the trajectory includes the translational velocity of the moving body and the angular velocity of the moving body.
The control device according to claim 5, wherein the input adjusting unit adjusts the translational speed of the moving body.
The control device according to claim 6, wherein the input adjusting unit reduces the translational speed of the moving body.
When the moving body stays in a range of a predetermined distance centered on the goal position for a certain period of time, the input adjusting unit determines that the moving body has arrived at the goal and inputs the input information to the inference model. The control device according to claim 2, which is stopped.
The control device according to claim 2, further comprising an output adjusting unit that adjusts the output information generated by using the inference model according to the distance to the goal position.
The output information includes the translational velocity of the moving body and the angular velocity of the moving body.
The control device according to claim 9, wherein the output adjusting unit adjusts the angular velocity of the moving body.
When the moving body enters the range of the first distance centered on the goal position, the output adjusting unit starts deceleration of the angular velocity of the moving body and sets it inside the range of the first distance. The control device according to claim 10, wherein when the moving body enters the range of the second distance, the angular velocity of the moving body is adjusted to 0.
The inference model is a model generated by machine learning using at least information about the orbit and information about the obstacle, and the output information for avoiding the obstacle is output according to claim 1. Control device.
The control device
The input information including at least the information about the trajectory and the information about the obstacle is input, and the input information input to the inference model that outputs the output information about the movement following the trajectory is adjusted according to the situation of the moving body. And
The output information is generated by inputting the adjusted input information into the inference model.
A control method for controlling the movement of the moving body based on the output information generated by using the inference model.
On the computer
The input information including at least the information about the trajectory and the information about the obstacle is input, and the input information input to the inference model that outputs the output information about the movement following the trajectory is adjusted according to the situation of the moving body. And
The output information is generated by inputting the adjusted input information into the inference model.
A program for executing a process of controlling the movement of the moving body based on the output information generated by using the inference model.