JP2012208782A - Move prediction apparatus, robot control device, move prediction program and move prediction method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a move prediction apparatus 10 which is capable of predicting a position where a person reaches, on the basis of a probability P of reach at each via-point (v), and to determine behavior of a robot 14 on the basis of the predicted position of reach or the set via-points (v).SOLUTION: A move prediction apparatus 10 comprises a plurality of LRFs 12 and the robot 14. Furthermore, the move prediction apparatus 10 records and analyzes move tracks of persons by means of the plurality of LRFs 12 to set a via-point (v) that becomes a target of short-term move into the space. The move of a person can be represented by a via-point sequence including a plurality of via-points, and the via-point sequence is recorded on a via-point list L which associating a move time (t) at the time and a move transition probability (p) of the via-point sequence therewith. The move prediction apparatus 10 then utilizes the via-point list L to calculate the probability P of reach for the person to reach each via-point (v) after the lapse of a predictive move time T.

Description

  The present invention relates to a movement prediction apparatus, a robot control apparatus, a movement prediction program, and a movement prediction method, and more particularly to a movement prediction apparatus, a robot control apparatus, a movement prediction program, and a movement prediction method that predict human movement, for example.

In the measuring apparatus disclosed in Patent Document 1, a plurality of laser range finders (LRF) are provided in a space, so that the position and moving speed of a person can be estimated and the person can be tracked.
JP 2009-168578 A [G01S 17/66, G01S 17/42, G01C 3/06, B25J 13/08]

  However, in the measuring device of Patent Document 1, it is impossible to accurately predict the position reached by the movement of a person at a place where a large number of persons come and go.

  Therefore, a main object of the present invention is to provide a novel movement prediction apparatus, robot control apparatus, movement prediction program, and movement prediction method.

  Another object of the present invention is to provide a movement prediction device, a robot control device, a movement prediction program, and a movement prediction method that can predict a position reached by movement.

  The present invention employs the following configuration in order to solve the above problems. The reference numerals in parentheses, supplementary explanations, and the like indicate the corresponding relationship with the embodiments described in order to help understanding of the present invention, and do not limit the present invention.

  In a first aspect of the present invention, in a space assuming a predetermined number of waypoints that are targets for short-term movement, a route point sequence including a plurality of waypoints representing a person's movement is converted into a travel time at that time and A movement prediction device for predicting movement of a person using a via point list associated with a movement transition probability of a via point series, and a time setting unit for setting a predicted movement time during which a person moves by the time of prediction When the travel time included in the via point list is less than the predicted travel time, based on the via point series included in the via point list, the moving transition probability calculating means for repeatedly calculating the moving transition probability to the next via point Each time the moving transition probability calculated by the moving transition probability calculating means is greater than or equal to the threshold, the updating means for updating the via point series, the moving time and the moving transition probability in the via point list, the transition included in the via point list When the time is equal to or longer than the predicted travel time, the arrival probability calculation means for calculating the arrival probability to each via point based on the via point list, and the arrival probability to each via point calculated by the arrival probability calculation means It is a movement prediction apparatus provided with the prediction means which estimates the arrival position of the person by movement.

  In the first invention, the movement prediction device (10: reference numeral exemplifying a corresponding part in the embodiment, hereinafter the same) passes through a predetermined number (for example, 30) as a short-term movement target. In the space where the point (v) is assumed, the movement of a person is predicted using the via point list (L). Also, in the waypoint list, the travel time (t) at that time and the movement transition probability (p) of the waypoint series are associated with a waypoint series including a plurality of waypoints representing the movement of a person. The time setting means (16, S61) sets a predicted travel time (T) in which a person moves by the time predicted by the movement prediction device, for example, in response to an input by the administrator of the movement prediction device. The movement transition probability calculating means (16, S123), when the movement time associated with a certain waypoint series is less than the predicted movement time, moves to the next waypoint based on the certain waypoint series. Probability is calculated repeatedly. In addition, every time the movement transition probability calculated repeatedly is greater than or equal to the threshold value, the next waypoint is added to the route point series, the movement time to the next waypoint is calculated, and the calculated movement transition probability and The product with the original movement transition probability is calculated. Then, the updating means (16, S107) updates the waypoint list based on the waypoint series, the moving time and the moving transition probability obtained in this way. The arrival probability calculation means (16, S159) calculates the arrival probability at each waypoint based on the waypoint list when the travel time is equal to or longer than the predicted travel time in the updated waypoint list. The predicting means (16, S183) predicts, for example, the via point having the highest arrival probability as the position where the person reaches after the predicted travel time.

  According to 1st invention, the position where a person reaches | attains can be estimated by calculating the arrival probability to each waypoint.

  The second invention is dependent on the first invention, and whenever the movement transition probability calculated by the movement transition probability calculation means is equal to or greater than a threshold value, the movement time to the next via point and the movement to the next via point Further comprising an adding means for adding the waypoint series associated with the transition probability to the waypoint list, the arrival probability calculating means is based on the moving transition probability in which the reaching waypoints are associated with the same waypoint series. The arrival probability is calculated.

  In the second invention, the adding means (16, S127) associates the movement time to the next waypoint and the movement transition probability to the next waypoint every time the calculated movement transition probability is equal to or higher than the threshold value. Add the waypoint series to the waypoint list. Therefore, the waypoint list includes a plurality of waypoint series. Then, the arrival probability calculating means calculates the arrival probability based on the movement transition probability associated with the same waypoint series that reaches the same waypoint among the plurality of waypoint series included in the waypoint list.

  According to the second aspect of the invention, the reliability of the calculated arrival probability can be improved by integrating the movement transition probabilities of a plurality of routes connecting the respective via points to calculate the arrival probability.

  The third invention is dependent on the first invention or the second invention, and is recorded by waypoint setting means for setting a predetermined number of waypoints in the space, recording means for recording a person's movement trajectory, and recording means. Based on the travel trajectory, a discriminating means for discriminating the moving direction of the person for each of the plurality of partial areas included in the space, and a predetermined number of routes set by the waypoint setting means based on the moving direction discriminated by the discriminating means Corresponding means for associating with points, and position updating means for updating the positions of a predetermined number of waypoints set by waypoint setting based on the movement directions associated with the points.

  In the third invention, the waypoint setting means (16, S31) randomly sets a predetermined number of waypoints in the space. The recording means (16, S5) records the movement trajectory of the person based on the position of the person measured by the LRF (12), for example. The discriminating means (16, S19) discriminates the main moving direction for each of the plurality of partial areas included in the space based on the movement trajectory of the person. The association means (16, S33) associates the main moving direction determined for each partial area with a predetermined number of waypoints set at random. The position updating means (16, S35) updates the positions of the waypoints until the position of the predetermined number of waypoints no longer changes based on the movement direction associated with each waypoint.

  According to the third aspect, the accuracy of prediction can be increased by setting the position of each waypoint based on the movement of a person.

  A fourth invention is dependent on the third invention, and based on the order calculated by the order calculating means and the order calculating means for calculating the order of moving the waypoints based on the movement trajectory recorded by the recording means. The moving transition probability calculation means further comprises a construction means for constructing a moving transition probability model for moving the via point, and the moving transition probability calculation means is based on the moving transition probability model constructed by the constructing means, based on the moving transition probability model for the next via point. Is calculated.

  In the fourth invention, the order calculating means (16, S41) calculates the order in which the person moves along the waypoint based on the recorded movement trajectory. For example, the movement transition probability when moving continuously through three waypoints is calculated, and the construction means (16, S43, S45) stores the calculated movement transition probability as a movement transition probability model in the database (34). To do. And a movement transition probability calculation means calculates the movement transition probability to the next waypoint based on the movement transition probability model preserve | saved at the database, for example.

  According to the fourth invention, by using the constructed movement transition probability model, it is possible to predict the position where a person reaches more accurately.

  5th invention is a robot control apparatus containing the movement prediction apparatus of 1st invention, Comprising: The action confirmation means which confirms the action of a robot based on the position where the person arrives estimated by the prediction means A robot control device.

  In the fifth invention, the robot control device (10) includes a movement prediction device that predicts a position where a person reaches by calculating an arrival probability to each waypoint. Further, the action confirming means (16, S185) determines the action of the robot so as to approach the predicted position to be reached, for example.

  According to the fifth aspect, the administrator of the robot control device can operate the robot in accordance with the predicted arrival position of the person.

  6th invention is a robot control apparatus including the movement prediction apparatus of 3rd invention, Comprising: The action decision means which decides the action of a robot based on the predetermined number of waypoints updated by the position update means Is a robot control device.

  In the sixth invention, the robot control device (10) includes a movement prediction device that updates the positions of a predetermined number of waypoints based on the movement trajectory of the person. The action determining means (16, S203) determines the action of the robot based on the waypoint whose position has been updated as described above.

  According to the sixth aspect of the invention, it is possible to determine the operation of the robot using the waypoint set based on the movement of the person. Thereby, it is possible to cause the robot to perform an action considering the movement of people.

  In a seventh aspect of the present invention, in a space that assumes a predetermined number of waypoints (v) that are targets for short-term movement, movement at that time is made into a series of waypoints that include a plurality of waypoints that represent person movement. A processor (16) of a movement prediction device (10) that predicts movement of a person using a via point list (L) associated with time (t) and the movement transition probability (p) of the via point sequence; Time setting means (S61) for setting a predicted travel time (T) in which a person moves by the predicted time point. When the travel time included in the via point list is less than the predicted travel time, the via points included in the via point list Based on the sequence, the moving transition probability calculating means (S123) for repeatedly calculating the moving transition probability to the next via point, and whenever the moving transition probability calculated by the moving transition probability calculating means is greater than or equal to the threshold value, Via point system in the list Updating means for updating the travel time and the travel transition probability (S107); when the travel time included in the transit point list is equal to or longer than the predicted travel time, the arrival probability for calculating the reach probability to each transit point based on the transit point list It is a movement prediction program that functions as a prediction means (S183) for predicting the arrival position of a person by movement based on the arrival probability at each waypoint calculated by the calculation means (S159) and the arrival probability calculation means. .

  In the seventh invention as well, as in the first invention, it is possible to predict the position where a person reaches by calculating the arrival probability to each waypoint.

  According to an eighth aspect of the present invention, in a space assuming a predetermined number of waypoints (v) that are targets for short-term movement, movement at that time is made into a waypoint series including a plurality of waypoints representing movement of people. A movement prediction method for a movement prediction apparatus (10) that predicts movement of a person using a via point list (L) associated with time (t) and a movement transition probability (p) of the via point series. A predicted travel time (T) in which a person moves by the predicted time point is set (S61), and when the travel time included in the via point list is less than the predicted travel time, the transit point sequence included in the via point list is set. Based on this, the transition transition probability to the next via point is calculated repeatedly (S123). Each time the calculated transition transition probability is equal to or greater than the threshold value, the via point series, the movement time, and the movement transition probability in the via point list are calculated. Is updated (S107) and the waypoint list When the included travel time is equal to or longer than the predicted travel time, the arrival probability to each via point is calculated based on the via point list (S159), and the person by movement is calculated based on the calculated arrival probability to each via point. This is a movement prediction method for predicting the arrival position (S183).

  In the eighth invention as well, as in the first invention, it is possible to predict the position where a person reaches by calculating the arrival probability to each waypoint.

  According to the present invention, it is possible to predict a position where a person reaches based on the arrival probability to each waypoint.

  The above object, other objects, features, and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

FIG. 1 is an illustrative view showing an outline of a movement prediction apparatus according to an embodiment of the present invention. FIG. 2 is a block diagram showing an electrical configuration of the movement prediction apparatus shown in FIG. FIG. 3 is an illustrative view showing the appearance of the robot shown in FIG. 1 from the front. FIG. 4 is a block diagram showing an electrical configuration of the robot shown in FIG. FIG. 5 is an illustrative view showing a measurement region of the LRF shown in FIGS. 1 and 2. FIG. 6 is an illustrative view showing one example of a movement locus detected using the LRF shown in FIG. FIG. 7 is an illustrative view showing a map of a certain space where the LRF shown in FIG. 1 is installed. FIG. 8 is an illustrative view showing one example of a divided area map created based on the map shown in FIG. FIG. 9 is an illustrative view showing one example of a movement locus in a certain divided region shown in FIG. FIG. 10 is an illustrative view showing one example of a movement direction calculated from each movement locus in a certain divided area shown in FIG. FIG. 11 is an illustrative view showing one example of a moving direction distribution and a mixed Gaussian distribution created based on the moving direction shown in FIG. FIG. 12 is an illustrative view showing one example of a main moving direction of a certain divided area, which is obtained from the mixed Gaussian distribution shown in FIG. FIG. 13 is an illustrative view showing one example of a moving direction map showing main moving directions in each divided area in the divided area map shown in FIG. 8. FIG. 14 is an illustrative view showing one example of a procedure for determining a position of a via point set in the moving direction map shown in FIG. FIG. 15 is an illustrative view showing one example of a waypoint map in which a waypoint is set in the movement direction map shown in FIG. FIG. 16 is an illustrative view showing one example of details of a route point and a route connecting the route points. FIG. 17 is an illustrative view showing one example of a waypoint list set based on the waypoints shown in FIG. FIG. 18 is an illustrative view showing one example of a procedure for updating the waypoint list shown in FIG. FIG. 19 is an illustrative view showing one example of a configuration of update result data obtained by updating the waypoint list by the procedure shown in FIG. FIG. 20 is an illustrative view showing one example of an arrival probability obtained based on the update result data shown in FIG. FIG. 21 is an illustrative view showing one example of a memory map of the memory shown in FIG. FIG. 22 is an illustrative view showing one example of a data storage area in the memory map shown in FIG. FIG. 23 is a flowchart showing the movement trajectory accumulation process of the processor shown in FIG. FIG. 24 is a flowchart showing movement direction calculation processing of the processor shown in FIG. FIG. 25 is a flowchart showing a waypoint setting process of the processor shown in FIG. FIG. 26 is a flowchart showing the movement transition probability model construction process of the processor shown in FIG. FIG. 27 is a flowchart showing main processing of the processor shown in FIG. FIG. 28 is a flowchart showing update processing of the processor shown in FIG. FIG. 29 is a flowchart showing additional processing of the processor shown in FIG. FIG. 30 is a flowchart showing an integration process of the processor shown in FIG. FIG. 31 is a flowchart showing robot control processing of the processor shown in FIG. FIG. 32 is a flowchart showing a robot control process of another embodiment of the processor shown in FIG.

  Referring to FIG. 1, the movement prediction apparatus 10 of this embodiment has several LRFs including LRFs 12a and 12b. LRF12a, 12b is installed in the space (location or environment) where a person can move arbitrarily. Places where people act arbitrarily are company floors, museums, shopping malls, or attraction venues, and the LRFs 12a and 12b are installed in various places.

  The movement prediction device 10 detects the position of a person who moves arbitrarily by using the LRFs 12a and 12b every predetermined time (for example, 1 second), and stores data of the detected position. Further, the movement prediction device 10 predicts a position that is reached when a person moves based on the stored position data. Then, the movement prediction device 10 performs wireless communication with the robot 14 via the network 100 to control the behavior of the robot 14. Therefore, the movement prediction device 10 may function as a robot control device.

  A person ID is given to a person whose position is detected in the order of detection. For example, ID “001” is assigned to the person who is detected first, and ID “002” is assigned to the person who is detected next.

  The robot 14 operates based on an action command given by the movement prediction device 10. The robot 14 is also an interaction-oriented robot (communication robot), and communication behavior including at least one of body movements such as gestures and voices with a communication target (communication target) such as a human being. It has a function to execute. The robot 14 also provides services such as route guidance and information provision to people as part of the communication.

  Although only one person is shown in FIG. 1 for simplicity, the movement prediction apparatus 10 can detect the positions of more people and store position data. Furthermore, although only one robot 14 is shown, the movement prediction apparatus 10 can simultaneously manage two or more robots 14.

  FIG. 2 is a block diagram showing an electrical configuration of the movement prediction apparatus 10. Referring to FIG. 2, movement prediction apparatus 10 includes LRFs 12a-12f, a processor 16, and the like. The processor 16 is sometimes called a microcomputer or CPU. In addition to the above-described LRF 12a and LRF 12b, the processor 16 is also connected with LRF 12c, LRF 12d, LRF 12e, and LRF 12f. In addition, when it is not necessary to distinguish LRF12a-12f, it is called "LRF12."

  The memory 16, LCD 20, input device 22, and communication LAN board 24 are connected to the processor 16, and a movement locus database (DB) 28, a movement direction distribution DB 30, a route point DB 32, and a movement transition probability DB 34 are also connected. Is done.

  The LRF 12 measures the distance to the object from the time it takes to irradiate the laser, reflect off the object (including a person), and return. For example, a laser beam emitted from a transmitter (not shown) is reflected by a rotating mirror (not shown), and the front is scanned in a fan shape by a certain angle (for example, 0.5 degrees). Here, as LRF12, a laser range finder (model LMS200) manufactured by SICK can be used. When this laser range finder is used, a distance of 8 m can be measured with an error of about ± 15 mm.

  Although not shown, the memory 18 includes a ROM, an HDD, and a RAM. In the ROM and HDD, a control program for controlling the operation of the movement prediction apparatus 10 is stored in advance. The RAM is used as a work memory or a buffer memory for the processor 16.

  The LCD 20 displays a management GUI of the movement prediction apparatus 10 and the like. The input device 22 includes a mouse and a keyboard. Then, the administrator inputs a command or the like using the keyboard or mouse of the input device 22 while confirming the GUI displayed on the LCD 20.

  The communication LAN board 24 is configured by a DSP, for example, and provides transmission data given from the processor 16 to the wireless communication device 26, and the wireless communication device 26 transmits the transmission data to the robot 14 via the network 100. For example, the transmission data is a signal (command) of an action command instructing the robot 14. The communication LAN board 24 receives data via the wireless communication device 26 and provides the received data to the processor 16.

  The movement locus DB 28 stores the movement locus of the person detected by the LRF 12. The movement direction distribution DB 30 stores a movement direction based on a person's movement locus. In the waypoint DB 32, information on waypoints based on the moving direction described above is stored. The movement transition probability model DB 34 stores a movement transition probability model constructed based on the above-described movement locus and via points. In addition, since each data preserve | saved in movement direction distribution DB30, via-point DB32, and movement transition probability DB34 are mentioned later, detailed description here is abbreviate | omitted.

  FIG. 3 is a front view showing the appearance of the robot 14 of this embodiment. Referring to FIG. 3, the robot 14 includes a carriage 40, and two wheels 42 and one slave wheel 44 that autonomously move the robot 14 are provided on the lower surface of the carriage 40. The two wheels 42 are independently driven by a wheel motor 46 (see FIG. 4), and the carriage 40, that is, the robot 14 can be moved in any direction, front, back, left, and right. The slave wheel 44 is an auxiliary wheel that assists the wheel 42. Therefore, the robot 14 can move in the arranged space by autonomous control.

  A cylindrical sensor mounting panel 48 is provided on the carriage 40, and a large number of infrared distance sensors 50 are mounted on the sensor mounting panel 48. These infrared distance sensors 50 measure the distance to the sensor mounting panel 48, that is, the object (human being, obstacle, etc.) around the robot 14.

  In this embodiment, an infrared distance sensor is used as the distance sensor, but a small LRF, an ultrasonic distance sensor, a millimeter wave radar, or the like can be used instead of the infrared distance sensor.

  On the sensor mounting panel 48, the body 52 is provided so as to stand upright. Further, the above-described infrared distance sensor 50 is further provided at the front center upper portion of the body 52 (a position corresponding to a person's chest), and measures the distance mainly to a person in front of the robot 14. Further, the body 52 is provided with a support column 54 extending from substantially the center of the upper end of the side surface, and an omnidirectional camera 56 is provided on the support column 54. The omnidirectional camera 56 photographs the surroundings of the robot 14 and is distinguished from an eye camera 80 described later. As the omnidirectional camera 56, for example, a camera using a solid-state imaging device such as a CCD or a CMOS can be employed. In addition, the installation positions of the infrared distance sensor 50 and the omnidirectional camera 56 are not limited to the portions, and can be changed as appropriate.

  An upper arm 60R and an upper arm 60L are provided at upper end portions on both sides of the body 52 (a position corresponding to a human shoulder) by a shoulder joint 58R and a shoulder joint 58L, respectively. Although illustration is omitted, each of the shoulder joint 58R and the shoulder joint 58L has three orthogonal degrees of freedom. That is, the shoulder joint 58R can control the angle of the upper arm 60R around each of three orthogonal axes. A certain axis (yaw axis) of the shoulder joint 58R is an axis parallel to the longitudinal direction (or axis) of the upper arm 60R, and the other two axes (pitch axis and roll axis) are orthogonal to the axes from different directions. It is an axis to do. Similarly, the shoulder joint 58L can control the angle of the upper arm 60L around each of the three orthogonal axes. A certain axis (yaw axis) of the shoulder joint 58L is an axis parallel to the longitudinal direction (or axis) of the upper arm 60L, and the other two axes (pitch axis and roll axis) are orthogonal to the axes from different directions. It is an axis to do.

  Further, an elbow joint 62R and an elbow joint 62L are provided at the tips of the upper arm 60R and the upper arm 60L, respectively. Although illustration is omitted, each of the elbow joint 62R and the elbow joint 62L has one degree of freedom, and the angles of the forearm 64R and the forearm 64L can be controlled around the axis (pitch axis).

  A sphere 66R and a sphere 66L corresponding to a human hand are fixedly provided at the tips of the forearm 64R and the forearm 64L, respectively. However, when a finger or palm function is required, a “hand” in the shape of a human hand can be used. Although not shown, the front surface of the carriage 40, the portion corresponding to the shoulder including the shoulder joint 58R and the shoulder joint 58L, the upper arm 60R, the upper arm 60L, the forearm 64R, the forearm 64L, the sphere 66R, and the sphere 66L, , A contact sensor 68 (shown generically in FIG. 4) is provided. The contact sensor 68 on the front surface of the carriage 40 detects contact of a person or other obstacle with the carriage 40. Therefore, when the robot 14 is in contact with an obstacle during its movement, the robot 14 can detect it and immediately stop driving the wheels 42 to suddenly stop the movement of the robot 14. In addition, the other contact sensors 68 detect whether or not the respective parts are touched. In addition, the installation position of the contact sensor 68 is not limited to the said site | part, and may be provided in an appropriate position (position corresponding to a person's chest, abdomen, side, back, and waist).

  A neck joint 70 is provided at the upper center of the body 52 (a position corresponding to a person's neck), and a head 72 is further provided thereon. Although illustration is omitted, the neck joint 70 has a degree of freedom of three axes, and the angle can be controlled around each of the three axes. A certain axis (yaw axis) is an axis directed directly above (vertically upward) of the robot 14, and the other two axes (pitch axis and roll axis) are axes orthogonal to each other in different directions.

  The head 72 is provided with a speaker 74 at a position corresponding to a human mouth. The speaker 74 is used for the robot 14 to communicate with a person around it by voice or sound. A microphone 76R and a microphone 76L are provided at a position corresponding to a human ear. Hereinafter, the right microphone 76R and the left microphone 76L may be collectively referred to as a microphone 76. The microphone 76 captures ambient sounds, in particular, the voices of humans who are subjects of communication. Furthermore, an eyeball part 78R and an eyeball part 78L are provided at positions corresponding to human eyes. The eyeball part 78R and the eyeball part 78L include an eye camera 80R and an eye camera 80L, respectively. Hereinafter, the right eyeball part 78R and the left eyeball part 78L may be collectively referred to as the eyeball part 78. Further, the right eye camera 80R and the left eye camera 80L may be collectively referred to as an eye camera 80.

  The eye camera 80 captures a human face approaching the robot 14, other parts or objects, and captures a corresponding video signal. The eye camera 80 can be the same camera as the omnidirectional camera 56 described above. For example, the eye camera 80 is fixed in the eyeball part 78, and the eyeball part 78 is attached to a predetermined position in the head 72 via an eyeball support part (not shown). Although illustration is omitted, the eyeball support portion has two degrees of freedom, and the angle can be controlled around each of these axes. For example, one of the two axes is an axis (yaw axis) in a direction toward the top of the head 72, and the other is orthogonal to the one axis and goes straight in a direction in which the front side (face) of the head 72 faces. It is an axis (pitch axis) in the direction to be performed. By rotating the eyeball support portion around each of these two axes, the tip (front) side of the eyeball portion 78 or the eye camera 80 is displaced, and the camera axis, that is, the line-of-sight direction is moved. In addition, the installation positions of the above-described speaker 74, microphone 76, and eye camera 80 are not limited to the portions, and may be provided at appropriate positions.

  As described above, the robot 14 of this embodiment has independent two-axis driving of the wheels 42, three degrees of freedom of the shoulder joint 58 (6 degrees of freedom on the left and right), one degree of freedom of the elbow joint 62 (2 degrees of freedom on the left and right), It has a total of 17 degrees of freedom, 3 degrees of freedom for the neck joint 70 and 2 degrees of freedom for the eyeball support (4 degrees of freedom on the left and right).

  FIG. 4 is a block diagram showing an electrical configuration of the robot 14. Referring to FIG. 4, robot 14 includes a processor 90. The processor 90 is also called a microcomputer or a processor, and is connected to the memory 94, the motor control board 96, the sensor input / output board 98, and the audio input / output board 110 via the bus 92.

  The memory 94 includes a ROM and a RAM (not shown). The ROM stores in advance a control program for controlling the operation of the robot 14. For example, a detection program for detecting the output (sensor information) of each sensor, a communication program for transmitting / receiving necessary data and commands to / from an external computer (movement prediction device 10), and the like are recorded. The RAM is used as a work memory or a buffer memory for the processor 90.

  The motor control board 96 is composed of, for example, a DSP, and controls the driving of motors for axes such as arms, neck joints, and eyeballs. That is, the motor control board 96 receives the control data from the processor 90, and controls two motors for controlling the respective angles of the two axes of the right eyeball portion 78R (in FIG. 4, they are collectively referred to as “right eyeball motor 112”). Control the rotation angle. Similarly, the motor control board 96 receives the control data from the processor 90, and shows two motors for controlling the respective angles of the two axes of the left eyeball portion 78L (in FIG. 4, collectively referred to as “left eyeball motor 114”). ) To control the rotation angle.

  The motor control board 96 receives control data from the processor 90, and includes a total of three motors that control the angles of the three orthogonal axes of the shoulder joint 58R and one motor that controls the angle of the elbow joint 62R. The rotation angles of four motors (collectively indicated as “right arm motor 116” in FIG. 4) are controlled. Similarly, the motor control board 96 receives control data from the processor 90, and includes three motors for controlling the angles of the three orthogonal axes of the shoulder joint 58L and one motor for controlling the angle of the elbow joint 62L. The rotation angles of a total of four motors (collectively indicated as “left arm motor 118” in FIG. 4) are controlled.

  Further, the motor control board 96 receives control data from the processor 90 and controls three motors that control the angles of the three orthogonal axes of the neck joint 70 (in FIG. 4, collectively indicated as “head motor 120”). ) To control the rotation angle. The motor control board 96 receives control data from the processor 90 and controls the rotation angles of the two motors that drive the wheels 42 (collectively indicated as “wheel motors 46” in FIG. 4). In this embodiment, a motor other than the wheel motor 46 uses a stepping motor (that is, a pulse motor) in order to simplify the control. However, a DC motor may be used similarly to the wheel motor 46. In addition, the actuator that drives the body part of the robot 14 is not limited to a motor that uses current as a power source, and may be changed as appropriate. For example, in another embodiment, an air actuator or the like may be applied.

  Similar to the motor control board 96, the sensor input / output board 98 is configured by a DSP, and takes in signals from each sensor and supplies them to the processor 90. That is, data relating to the reflection time from each of the infrared distance sensors 50 is input to the processor 90 through the sensor input / output board 98. The video signal from the omnidirectional camera 56 is input to the processor 90 after being subjected to predetermined processing by the sensor input / output board 98 as necessary. Similarly, a video signal from the eye camera 80 is also input to the processor 90. In addition, signals from the plurality of contact sensors 68 described above (collectively referred to as “contact sensors 68” in FIG. 4) are provided to the processor 90 via the sensor input / output board 98. Similarly, the voice input / output board 110 is also configured by a DSP, and voice or voice in accordance with voice synthesis data provided from the processor 90 is output from the speaker 74. Also, audio input from the microphone 76 is given to the processor 90 via the audio input / output board 110.

  The processor 90 is connected to the communication LAN board 122 via the bus 92. The communication LAN board 122 is configured by a DSP, for example, and provides transmission data given from the processor 90 to the wireless communication device 124, and the wireless communication device 124 sends the transmission data to the external computer (movement prediction device 10) via the network 200. Send to. In addition, the communication LAN board 122 receives data via the wireless communication device 124 and gives the received data to the processor 90. For example, the transmission data includes surrounding video data captured by the omnidirectional camera 56 and the eye camera 80.

  Next, the LRF 12 will be described in detail. Referring to FIG. 5, the measurement range of LRF 12 is indicated by a semicircular shape (fan shape) having a radius R (R≈8 m). That is, the LRF 12 can measure the direction of 90 degrees left and right within a predetermined distance when the front direction is the center.

  The laser used is a class 1 laser in the Japanese Industrial Standard JIS C 6802 “Safety Standards for Laser Products”, which is a safe level that does not affect human eyes. In this embodiment, the sampling rate of the LRF 12 is 37 Hz. This is to continuously detect the position of a person who moves by walking or the like.

  Further, each of the LRFs 12 is arranged so that the measurement ranges overlap, and although not shown, it is fixed at a height of about 90 cm from the floor surface. This height is for enabling detection of the body and arms (both arms) of the subject, and is calculated from, for example, the average height of Japanese adults. Therefore, the height at which the LRF 12 is fixed may be appropriately changed according to the location (region or country) where the movement prediction device 10 is provided and the age or age of the subject (for example, a child or an adult). In this embodiment, six LRFs 12 are set, but any number of LRFs 12 may be installed as long as the number is two or more.

  In the movement prediction apparatus 10 having such a configuration, the processor 16 estimates a change in the current position of the person using a particle filter based on the output (distance data) from the LRF 12. Then, the estimated change in position is recorded as a movement trajectory.

  For example, referring to FIG. 6, LRFs 12a and 12b are installed facing each other, and the range where the measurement ranges of LRF 12a and 12b overlap is shaded. The hatched area is a detection area, and the position of the person is continuously detected in this detection area. And the change of the position detected continuously becomes a movement locus. That is, the movement trajectory data is composed of a plurality of position data. Note that xy coordinates are set in the detection area, and the position data of a person can be indicated by coordinates (x, y).

  FIG. 7 is an illustrative view showing a map showing a place where the LRF 12 is installed. Referring to FIG. 7, the place corresponding to the map is a shopping mall. In addition, a passage is provided in the center of the map, and there are stores, elevators, and the like around it. The LRF 12 is provided in the passage, and a part of the shopping mall passage serves as a detection area. Therefore, the movement trajectory DB 28 stores the movement trajectories of people who go and go through the shopping mall passages.

  Here, in this embodiment, by analyzing the saved movement trajectory of people, a via point that is a target of short-term movement of the person is set, and the movement of the person is predicted based on the via point. The waypoints can be set by analyzing the movement of people for each of a plurality of partial areas provided in the detection area.

  Referring to FIG. 8, the partial area is a square (grid) with a side of 50 cm, and is provided in a part of a shopping mall passage. The movement trajectories of a plurality of people passing through the partial area can be shown as shown in FIG. 9, for example, and the movement direction of each movement trajectory can be obtained by the equation shown in Equation 1. In Equation 1, “θ” indicates an angle when passing through the partial area (see FIG. 10), and “x (t)” is a position vector of the person at time t when the person is closest to the center of the partial area. “Arg (x)” is a function for obtaining the angle of the position vector. In addition, “Δ” is a short time (for example, 4 seconds), which suppresses fluctuations in the movement trajectory caused by walking and the effects of short-term direction changes to avoid the person who came from before. Used to calculate the direction with high accuracy.

[Equation 1]
θ = arg (x (t + Δ) −x (t))
For example, when the movement direction in a certain partial region is calculated using the equation (1), the result is as shown in FIG. Referring to FIG. 10, in the region, the right side is 0 degree, the upper side is 90 degrees, the left side is 180 degrees, the lower side is 270 degrees, and the same position as 0 degree is 360 degrees. That is, in the partial area, the angle is set so as to increase counterclockwise with respect to the right side. In the movement direction distribution shown in FIG. 10, each black circle in the partial area indicates the direction in which the person has left the partial area, that is, the movement direction of the person. Further, as can be seen from FIG. 10, in a certain partial area, the movement direction of people is not concentrated in one direction but is distributed in several directions.

  Note that the function “arg (x)” in Equation 1 is not a special function defined for this embodiment. That is, the function “arg (x)” is a general function for calculating the angle of a vector. Therefore, in another embodiment, a general function that can determine the angle of a vector can be used instead of the function “arg (x)”.

  Next, FIG. 11 is a graph showing a plurality of moving directions shown in FIG. In this graph, the horizontal axis represents the angle, and the vertical axis represents the number of times the moving direction of the same angle has appeared (calculated). Therefore, a solid line in the graph indicates a change in the number of times the movement direction is calculated. In this embodiment, the main moving direction of a person passing through a certain partial area is obtained using this graph.

  Specifically, the center of a plurality of peaks (main movement directions) appearing in the movement direction distribution is obtained by modeling the movement direction distribution with a mixed Gaussian distribution. In FIG. 11, the modeled mixed Gaussian distribution is indicated by a dotted line, and a horizontal axis corresponding to each peak is given a cross mark. Accordingly, it can be seen that people passing through a certain partial region pass mainly in directions of about 130 degrees, about 225 degrees, and about 315 degrees.

  Modeling with a mixed Gaussian distribution is described in “IS Dhillon and S. Sra,“ Modeling Data using Directional Distributions, ”Technical report of University of Texas, TR-03-062, 2003”. The detailed explanation is omitted.

  FIG. 12 shows the top three movement directions among the movement directions (peaks) obtained by the above-described processing in a certain partial region. Referring to FIG. 12, the most frequent moving direction (about 315 degrees) is indicated by a dashed arrow, the second most frequent moving direction (about 225 degrees) is indicated by a dotted arrow, and the third most frequent moving direction (about 130). Degrees) is indicated by solid arrows. Similarly, the main movement directions are obtained for the other partial areas, and the results are stored in the movement direction distribution DB 30.

  FIG. 13A is a movement direction map stored in the movement direction distribution DB 30 and showing only the largest movement direction among the main movement directions in each partial area. FIG. 13B is an enlarged view of the area A in the movement direction map. Referring to FIG. 13A, it can be seen that a person is moving in approximately the same direction where the passage is narrow. Further, as can be seen from FIG. 13B, in the region A where the size of the passage is changed, the moving direction indicates various directions.

  Next, how to obtain a via point from the movement direction stored in the movement direction distribution DB 30 will be described. First, as an initial setting, a predetermined number of waypoints are set randomly using random numbers. For example, in a shopping mall of about 500 square meters, 30 waypoints are set at random.

  Next, one waypoint is associated with the moving direction obtained for each partial region. Specifically, a perpendicular line is dropped from the via point with respect to a straight line extending in the movement direction from the center o of the partial area, and the via point having the shortest perpendicular line is associated with the movement direction. Here, the via point where the perpendicular is lowered is within a predetermined distance (for example, 10 m) from the center o of the partial region, and the angle formed by the direction of the via point and the moving direction is a predetermined angle when viewed from the center o of the partial region. It is limited to those within (for example, 10 degrees).

For example, referring to FIG. 14 (A), when via point v ₁ and via point v ₂ meet the above-mentioned conditions at the center o of a certain partial area, a straight line obtained by extending movement direction vector Y is used. , perpendicular d1 and perpendicular d2 is unloaded from the waypoint _{v 1} and through point _{v 2.} In this case, towards the vertical line d1 is shorter than the vertical line d2, the waypoint v ₁ associated to the movement direction vector Y.

  When the via points are associated with all the moving directions, the position of each via point is updated. Specifically, at a certain waypoint, a perpendicular vector is lowered from a certain waypoint with respect to a straight line extending in the moving direction associated with a certain waypoint. Next, synthesized vectors are obtained by synthesizing the perpendicular vectors drawn from a certain waypoint, and the waypoint is moved in the direction of the synthesized vector. Referring to FIG. 14B, for example, when movement direction vectors Y1, Y2, and Y3 are associated at a certain waypoint v, a straight line extending movement direction Y vectors 1, Y2, and Y3 is used. Three perpendicular vectors are taken from a certain via point v. Based on the combined vector r obtained by combining the three perpendicular vectors, the position of the via point v is moved (updated) to the position of the via point v ′.

  Then, the above-described processing is repeated for all via points until the positions of all via points set at random do not change. As a result, when the position of the waypoint does not change, the position of each waypoint is stored in the waypoint DB 32.

  FIG. 15A is a waypoint map in which waypoints stored in the waypoint DB 32 are arranged on a moving direction map. Referring to FIG. 15A, most waypoints are set in the detection area, but some waypoints are set outside the detection area (in the store). In addition, referring to FIG. 15B, there is a tendency that many via points are set in places such as region A where there are many changes in the moving direction.

  Thus, the accuracy of prediction can be increased by setting the position of each waypoint based on the movement of a person.

  Subsequently, when predicting the movement of a person using the set waypoints, a movement transition probability model is first constructed based on the order of transition (movement) of the waypoints. And the position (arrival position) where a person reaches | attains after the estimated movement time T is estimated using the constructed model.

  First, in order to construct the movement transition probability model, the order in which people pass through the waypoints and the probability of passing are determined based on the movement trajectory stored in the movement trajectory DB 28. Here, in the present embodiment, it is determined that the person has passed through the waypoint when entering within a certain distance from the waypoint. However, in another embodiment, the determination of the passage of the via point uses the angle formed by the moving direction of the movement trajectory and the direction of the via point, the distance to the via point is within a predetermined distance, and the angle formed is If it is within a predetermined angle, it may be regarded as passing.

Further, the movement transition probability p between two waypoints is calculated based on the formula shown in Equation 2, and the movement transition probability p between the three waypoints is calculated based on the equation shown in Equation 3. In Equation 2, “C (i, j)” is the number of people in the stored movement trajectory that have passed through the transit point v _i and subsequently passed through the transit point v _j. i) ”is the number of people who have passed through the route point v _i . In Equation 3, “C (i, j, k)” is the number of people who have passed in the order of waypoints v _i , v _j , and v _k . In other words, the expression shown in Equation 2 represents the ratio of the people who have passed through the transit point v _j among the people who have passed through the transit point v _i . In addition, the equation shown in Equation 3 represents the ratio of people who have passed through the transit point v _j out of those who have passed through the transit point v _i and the transit point v _j .

  Here, when the movement transition probability p is obtained by Equations 2 and 3, there is a problem that C (i, j) and C (i, j, k) cannot be sufficiently obtained due to lack of data. In order to solve this problem, in the present embodiment, when the movement transition probability p (k | i, j) is calculated by backoff smoothing, interpolation is performed using the movement transition probability p (i | j), and the movement transition probability is calculated. Interpolation is performed using the movement transition probability p (i) when calculating p (i | j). Since this back-off smoothing is described in “Ken Kenji“ Probabilistic Language Model ”, University of Tokyo Press 1999”, detailed explanation is omitted.

  Also, the movement transition probability p (k | i, j) corresponding to the combination of the three waypoints is calculated for the other waypoints. The calculation result is stored in the movement transition probability model DB 34 as a movement transition probability model. Further, in the present embodiment, by using the constructed movement transition probability model, it is possible to predict the position where a person reaches more accurately.

  Next, the arrival position in the predicted movement time T is predicted using the movement transition probability model stored in the movement transition probability model DB 34. The predicted movement time T is the time required for the subject to move before the predicted time. When the arrival position is predicted, in addition to the movement transition probability model, the current position and movement direction of the person (target person) to be predicted are also used. Thereby, the arrival position in the predicted movement time T is predicted.

  First, in the movement prediction process, the waypoint list L is initialized based on the current waypoint and the waypoint that passed immediately before. In this waypoint list L, the travel time t at that time and the movement transition probability p of the waypoint sequence are associated with a waypoint series including a plurality of waypoints representing the movement of a person. Next, a new waypoint series including the next waypoint (the next waypoint) that the user is aiming for is added to the waypoint series in the waypoint list L, and the travel time t and movement of the waypoint series are added. The transition probability p is updated. At this time, if the movement transition probability p of the waypoint series is equal to or less than the threshold value, the waypoint series is not added to the waypoint list L.

  Furthermore, when the travel time t of the route point series exceeds the predicted travel time T as a result of the update, the route point sequence associated with the travel time t is moved to the update result data. Then, if all the waypoint sequences included in the waypoint list L are moved, the movement transition probabilities p of route points that reach the same waypoint are integrated based on the update result data, and the predicted movement time T An arrival probability P is calculated. When the arrival point of the person is predicted based on the arrival probability P, the behavior of the robot 14 is confirmed. In the following description, the movement prediction process is described assuming that the predicted movement time T is “4.0 seconds” and the threshold is “0.1%”.

FIG. 16 (A) shows a route connecting certain four waypoints v, and FIG. 16 (B) shows the time taken for moving those routes. Referring to FIG. 16A, the waypoint v _b is the waypoint v that the subject has passed immediately before, and the waypoint v _p is the waypoint v that is currently passing. The via points v ₁ and v ₂ are via points v through which the subject may pass in the future. In addition, the route connecting each waypoint v is the route v _b -v _p , the route v _b -v ₁ , the route v _b -v ₂ , the route v _p -v ₁ , the route v _p -v _2, and the route v _1-. v is _2. Since the path v _b -v _p has already moved, it is indicated by a solid line arrow. Referring to FIG. 16B, the time of each route is calculated based on the distance of each route connecting via point v and the average movement speed sp of the person. The distance of the route is calculated using the three-square theorem for the coordinates of two waypoints. In addition, the average moving speed sp is calculated from the moving speed of the subject in recent several seconds based on the moving track recorded using the LRF 12. However, when the movement trajectory of the person is not recorded, the average speed of the person moving in the same place may be calculated from the movement trajectory stored in the movement trajectory DB 28.

When the movement prediction process is executed when the subject is at the route point v _p , the route point list L is initialized as shown in FIG. Referring to FIG. 17, in the waypoint list L, the waypoint series is recorded as “v _b , v _p ”, and the corresponding movement time and movement transition probability are “0.0 seconds” and “100%”. Is recorded. In the following description, one route point series in the route point list L, and the corresponding movement time and movement transition probability are collectively referred to as “element e”.

Next, the waypoint list L performed before the arrival probability P is calculated will be described. First, the transit points v ₁ and v ₂ are included in the transit points v that can be transitioned from the current transit point v _p, and there is a possibility that the subject may stop or return. v _p and v _b are also included. Therefore, the candidates for the element e (route point series) newly added (updated) to the route point list L shown in FIG. 17 are “v _b , v _p , v _b ”, “v _b , v _p , “v _p ”, “v _b , v _p , v ₂ ” and “v _b , v _p , v ₁ ”.

Further, the movement transition probability model associated with these candidate waypoint sequences and the movement transition probability p that matches the mathematical formula shown in Equation 4 is read from the movement transition probability model DB 34. In Equation 4, “Nv” is the next waypoint, “Fv” is the last waypoint in the waypoint series, and “Fv ₋₁ ” is the waypoint immediately before the last waypoint. is there.

[Equation 4]
p = (Nv | Fv ₋₁ , Fv)
For example, when “Fv” is a transit point v _p , “Fv ₋₁ ” is a transit point v _b, and “Nv” is a transit point v ₂ , the equation shown in Equation 5 is “movement transition probability p = ( Via point v ₂ | via point v _b , via point v _p ) ”. Then, the moving transition probability model DB 34 reads out the moving transition probability model (20.0%) that passes through the transit point v _p and the transit point v _b and further passes through the transit point v ₂ .

Similarly, the result of obtaining the movement transition probability p for other candidates is as shown in FIG. Referring to FIG. 18 (A), the route point series (v _b , v _p , v _b ) indicating return to the previous route point v _b is associated with “0.5%” as the movement transition probability p. It is done. Via point sequence indicating that remains now via point _{v p (v b, v p} , v p) in the "0.8%" is associated as the moving transition probability p. “70.0%” is associated as the movement transition probability p with the via point series (v _b , v _p , v ₁ ) indicating that the vehicle proceeds to the next via point v ₁ . Similarly, “20.0%” is associated as the movement transition probability p with the via point series (v _b , v _p , v ₂ ) indicating that the vehicle proceeds to the next via point v ₂ .

  Then, by omitting candidates whose movement transition probability p is less than the threshold, the waypoint list L is updated, and the update result is as shown in FIG. Here, in the element e added by the update, the movement transition probability p is the movement transition probability p of the original element e and the movement transition probability p obtained by the movement transition probability model. Further, the travel time t is calculated based on the mathematical formula shown in Equation 5. Note that “Bt” in Equation 5 is the movement time t associated with the previous waypoint series, and “| Nv−Fv |” is the distance between the last waypoint and the next waypoint.

For example, if “Fv” is a transit point v _p , “Fv ₋₁ ” is a transit point v _b, and “Nv” is a transit point v ₂ , the previous travel time t is “0” as shown in FIG. .0 seconds ”, the equation shown in Equation 5 is“ 4.0 seconds = 0.0 + (| route point v ₂ −route point v _p |) / average moving speed sp ”. Further, since the movement transition probability p of the original element is “1.00 (initial value)”, the movement transition probability p calculated as a candidate here is recorded in the waypoint list L as it is. .

  After each movement time t is calculated, the element e whose movement time t exceeds the predicted movement time T is moved to the update result data. However, since the route point list L shown in FIG. 18B does not include the element e that satisfies the condition, the content of the route point list L does not change.

Subsequently, the candidate for the element e newly added to the waypoint list L shown in FIG. 18B is as shown in FIG. That is, a route point sequence including a route point v that may be passed next to the route points v ₁ and v ₂ is listed as a candidate. FIG. 18D shows the result of updating the waypoint list L based on the movement transition probability p associated with each candidate. At this time, the travel time t included in the elements e ₁ , e _3, and e ₄ of the updated waypoint list L is larger than the predicted travel time T (4.0 seconds). It is moved to the update result data shown. Therefore, the waypoint list L shown in FIG. 18 (D), only the elements _{e 2} remains.

For example, with reference to FIG. 19 (A), component e via point list L in FIG. 18 (D) is in the update result data transferred, the elements e ₁ is an element E ₁ of the update result data, the element e ₃ is an element _{E 2,} element _{e 4} is an element _{E 3.}

Next, the candidate of the element e newly added with respect to the waypoint list L shown in FIG.18 (D) becomes like FIG.18 (E). In this case, the candidates corresponding to the movement transition probability p exceeding the threshold are only “v _b , v _p , v ₁ , v _p , v ₂ ”, and the result of updating the waypoint list L is shown in FIG. become that way. And the element e ₁ of the updated waypoint list L has a movement time t longer than the predicted movement time T. Therefore, this element e ₁ is also transferred to the update result data. As a result, nothing is recorded in the route point list L, and the update result data is as shown in FIG. In other words, as a new element E4, elements e ₁ in FIG. 18 (F) is transferred to update result data, also ends the process of updating the waypoint list L.

Subsequently, in the movement prediction process, when the waypoint list L is not updated, the movement transition probabilities included in the update result data are integrated for each waypoint v. Specifically, the sum total of the movement transition probabilities p associated with the waypoint series in which the last waypoint Fv includes the same waypoint v is calculated as the arrival probability P (v). For example, in the update data in FIG. 19B, element E ₂ and element E ₄ include the same waypoint v2 as the last waypoint Fv. Therefore, in the arrival probability data shown in FIG. 20, the sum (42.2.%) Of the movement transition probability p between the element E ₂ and the element E ₄ is calculated as the arrival probability P (v ₂ ). Further, the last waypoint Fv of each of the elements E ₁ and E ₃ is not included in the other elements E. Therefore, the movement transition probability p of each of the element E ₁ and the element E ₃ is set as the arrival probability P (v _b ) = “2.5%” and the arrival probability P (v _p ) = “1.2%”. . Here, since there is no element E having the via point v ₁ as the last via point Fv, the arrival probability P (v ₁ ) is “0.0%”.

  As described above, in this embodiment, the reliability of the calculated arrival probability P is improved by calculating the arrival probability P by integrating a plurality of movement transition probabilities p connecting the respective via points.

Further, in this embodiment, the position where a person reaches is predicted based on the arrival probability P to each waypoint v. (1) For example, when the arrival position of a person (subject) is predicted based on the arrival probability data shown in FIG. 20, since the arrival probability P (v ₂ ) is the largest, the route point v after about 4.0 seconds It is predicted that there will be a subject near ₂ .

Then, by using this prediction result, as the robot 14 moves closer to the waypoint v _2, action instruction is issued to the robot 14. That is, the administrator can operate the robot 14 in accordance with the predicted arrival position of the person.

Note that the predicted travel time T and the threshold used in the above description are examples, and can be arbitrarily changed. For example, when a waypoint is set as shown in FIG. 15A, the predicted travel time T may be “10 seconds” and the threshold may be “10 ⁻¹⁰ ”.

  In addition, when there are many pedestrians in the space, the target person often changes the direction in order to avoid collision with other pedestrians that are facing him. Therefore, in another embodiment, a candidate for the next waypoint may be estimated using “Social Force Model (SFM)” that expresses the influence of the presence of another pedestrian on the subject (self). For example, if the position and direction of movement of a pedestrian existing in the current space is given to the SFM, the direction that each pedestrian is heading for (desired) after removing the influence of other pedestrians. Direction of movement) is estimated. In another embodiment, the next waypoint candidate is estimated by estimating the moving direction desired by the subject by SFM. This eliminates the need to search for the next candidate brute force, thereby shortening the processing time. Here, the details of SFM and its usage are described in “F. Zanlungo, T. Ikeda and T. Kanda,“ Social force model with explicit collision prediction, ”EPL, 93 (2011) 68005.”. Therefore, detailed description is omitted.

  In another embodiment, a simpler process may be selected that is close to an extension line in the current movement direction.

  FIG. 21 is an illustrative view showing one example of a memory map of the memory 18 in the movement prediction apparatus 10 shown in FIG. As shown in FIG. 21, the memory 18 includes a program storage area 302 and a data storage area 304. In the program storage area 302, as a program for operating the movement prediction apparatus 10, a movement locus accumulation program 310, a movement direction calculation program 312, a waypoint setting program program 314, a movement transition probability model construction program 316, and a movement prediction program 318 are stored. In addition, a robot control program 320 and the like are stored. The movement prediction program 318 includes a main program 318a, an update program 318b, an additional program 318c, and an integrated program 318d.

  The movement trajectory accumulation program 310 is a program for saving a movement trajectory obtained by continuously detecting a person's position in the movement trajectory DB 28. The movement direction calculation program 312 is a program for calculating a main movement direction for each partial area based on the movement locus stored in the movement locus DB 28 and saving the calculation result in the movement direction distribution DB 30. The waypoint setting program 314 is a program for setting a waypoint in the detection area and its periphery based on the movement direction stored in the movement direction distribution DB 30. The movement transition probability model construction program 316 is a program for calculating the movement transition probability p based on the movement trajectory and the waypoint and saving the calculation result as a movement transition probability model in the movement transition probability model DB 34.

  The movement prediction program 318 is a program for calculating the arrival probability P for each waypoint v. The main program 318a of the movement prediction program 381 is a program for creating and updating the waypoint list L for calculating the arrival probability P. The update program 318b is a program for updating the waypoint list L. The addition program 318c is a program for determining whether to add a new element to the waypoint list L. The integrated program 318d is a program for calculating the arrival probability P at each waypoint based on the update result data obtained by updating the waypoint list L.

  The robot control program 320 is a program for determining the action of the robot 14 and issuing an action command.

  Although illustration is omitted, the program for operating the movement prediction device 10 includes a program for converting the distance obtained by the LRF 12 into position (coordinate) data.

  Referring to FIG. 22, the data storage area 304 is provided with an update buffer 330, a predicted travel time buffer 332, and the like. Further, the data storage area 304 stores route point list data 334, update result data 336, arrival probability data 338, and the like.

  In the update buffer 330, the element e added to the waypoint list L is temporarily stored. The predicted travel time buffer 332 temporarily stores a predicted travel time used as a threshold value. The waypoint list data 336 is data configured as shown in FIGS. 17 and 18B, for example. The update result data 336 is data configured as shown in FIG. The arrival probability data 338 is data configured as shown in FIG. 20, and includes the arrival probability P for each waypoint.

  Although not shown, the data storage area 304 stores data indicating the total number (for example, 30) of via points v and array (set) data for storing the coordinates of each via point v. A buffer for temporarily storing the results of various calculations and other counters and flags necessary for the operation of the movement prediction apparatus 10 are also provided.

  Hereinafter, the program executed by the processor 16 of the movement prediction apparatus 10 will be described. The flowchart of FIG. 23 shows the processing by the movement locus accumulation program 310, the flowchart of FIG. 24 shows the processing by the movement direction calculation program 312, FIG. 25 shows the processing by the waypoint setting program 314, and FIG. FIG. 27 to FIG. 30 show processing by each subroutine included in the movement prediction program 318, and FIG. 31 shows processing by the robot control program 320.

  FIG. 23 is a flowchart of the movement locus accumulation process. When the power of the movement prediction apparatus 10 is turned on, the processor 16 of the movement prediction apparatus 10 records the current time in step S1, and detects position information of a person in step S3. In other words, the position coordinates of people existing in the detection area are detected corresponding to the time recorded in step S1. Subsequently, in step S5, the current time and position information are stored in the movement trajectory DB 28, and the process returns to step S1. That is, when the processing of steps S1 to S5 is repeated, the positions of people in the detection area are repeatedly stored in the movement locus DB 28. In addition, the change in the position of the people repeatedly stored in this way is read as a movement trajectory. Further, the processor 16 that executes the process of step S5 functions as a recording unit.

  FIG. 24 is a flowchart of the movement direction calculation process. For example, when the administrator of the movement prediction apparatus 10 performs an operation for calculating the movement direction, the processor 16 sets a partial area in step S11. For example, in a shopping mall passage where the LRF 12 is provided, a plurality of partial areas are set as shown in FIG. Subsequently, in step S13, the movement direction of the movement locus passing through each partial region is calculated. For example, the movement direction of people passing through a certain partial area is calculated based on the mathematical formula shown in Equation 1. Subsequently, in step S15, a moving direction distribution is created for each partial region. For example, the movement direction distribution (solid line) shown in FIG. 11 is created from the calculated movement direction. Subsequently, in step S17, the distribution in the moving direction is modeled by a mixed Gaussian distribution. For example, the mixed Gaussian distribution (dotted line) shown in FIG. 11 is obtained by modeling the movement direction distribution created in step S15. In step S19, the main moving direction of each partial area is determined. For example, the peak indicating the main moving direction is obtained from the mixed Gaussian distribution shown in FIG. The processor 16 that executes the process of step S19 functions as a determination unit.

  Subsequently, in step S21, the main movement direction of each partial area is stored in the movement direction distribution DB 30, and the movement direction calculation process is terminated. For example, when the Gaussian distribution shown in FIG. 11 is created, 315 degrees with the largest number of appearances is stored in the movement direction distribution DB 30 as the main movement direction.

  FIG. 25 is a flowchart of the waypoint setting process. For example, when the moving direction calculation process ends, a waypoint setting process is executed. When the waypoint setting process is executed, the processor 16 randomly sets a predetermined number of waypoints v in step S31. For example, 30 waypoints v are randomly set based on a function for generating random numbers. The processor 16 that executes the process of step S31 functions as a waypoint setting unit.

  Subsequently, in step S33, each moving direction is associated with the waypoint v. That is, each movement direction stored in the movement direction distribution DB 30 is within a predetermined distance (for example, 10 m) from the center o of the corresponding partial area, and the direction and movement of the via point v viewed from the center o of the partial area. The angle formed with the direction is associated with a via point v within a predetermined angle (for example, 10 degrees). When a plurality of waypoints v satisfy this condition, a waypoint v with the shortest perpendicular d is associated as shown in FIG. The processor 16 that executes the process of step S33 functions as an association unit.

  Subsequently, in step S35, an update process of the waypoint v is executed. That is, as shown in FIG. 14B, a combined vector is obtained based on the moving direction associated with each via point v, and the position of the via point v is updated (moved) based on the combined vector. The processor 16 that executes the process of step S35 functions as a position updating unit.

  Subsequently, in step S37, it is determined whether or not the position of the waypoint v has changed. That is, even if the position of each via point v is updated in the process of step S35, it is determined whether or not the via point position has changed. Further, as a specific process, all the coordinates of each waypoint are recorded before executing the process of step S35, and it is determined whether the coordinates of each waypoint have changed after the process of step S35 is executed. To do.

  If “NO” in the step S37, that is, if the position of the via point v is changed by the update, the process returns to the step S35. On the other hand, if “YES” in the step S37, that is, if the position of the via point v has not changed even after the update, the coordinates of each via point v are stored in the via point DB 32 and the via point setting process is ended. To do.

  In step S37 of another embodiment, “YES” may be determined if the change amount of the waypoint v is small. For example, if the change in the position of the waypoint v is 10 cm or less as a result of the update, “YES” may be determined in step S37.

  FIG. 26 is a flowchart of the movement transition probability model construction process. For example, when the waypoint setting process ends, a movement transition probability model construction process is executed. When the process is executed, the processor 16 calculates the order in which the respective movement trajectories pass through the route point v in step S41. The processor 16 that executes the process of step S41 functions as an order calculation unit.

  Subsequently, in step S43, the movement transition probability p is calculated based on each calculated order. That is, the movement transition probability p (k | i, j) is calculated based on the mathematical expressions of the above formulas 2 and 3. Subsequently, in step S45, the calculation result is stored in the movement transition probability DB 34 as a movement transition probability model, and the movement transition probability model construction process is terminated. The processor 16 that executes the processes of steps S43 and S45 functions as a construction unit.

  FIG. 27 is a flowchart of main processing which is a subroutine of the movement prediction program 318. For example, when the administrator of the movement prediction apparatus 10 inputs a command for predicting the movement of a person, the processor 16 sets the predicted movement time T in step S61. For example, the processor 16 displays a GUI prompting the input of the predicted travel time T on the LCD 20 of the motion prediction apparatus 10. The result input by the administrator is stored in the predicted travel time buffer 332 as the predicted travel time T. In another embodiment, data indicating a predetermined predicted travel time T may be stored in the memory 18 and stored in the predicted latitude time buffer 332. The processor 16 that executes the process of step S61 functions as a time setting unit.

Subsequently, in step S63, the average movement speed sp of the person is calculated. For example, as described above, the average movement speed sp of the person is calculated based on the movement locus stored in the movement locus DB 28. Subsequently, in step S65, the route point v _p through which the person is currently passing is specified, and in step S67, the route point v _b in which the person has passed immediately before is specified. That is, the via point v _p and the via point v _b are specified based on the movement locus of the person recorded in the movement locus DB 28.

Subsequently, in step S69, the waypoint list L is initialized based on the waypoints v _p and v _b . That is, when the waypoint list L is initialized, as shown in FIG. 17, the travel time t of “0.0 seconds” and “100%” are added to the waypoint series including the waypoints v _p and v _b . A movement transition probability p is associated. Subsequently, the update result data 336 is initialized. For example, no update result data 336 is recorded.

  Subsequently, in step S73, it is determined whether or not nothing is recorded in the waypoint list L. That is, as a result of updating the waypoint list L, it is determined whether or not all the elements e have been moved to the update result data 336. If “NO” in the step S73, that is, if the element e is included in the waypoint list L, the updating process is executed in a step S75. That is, in order to update the waypoint list L, an update process is executed. Then, when the process of step S75 ends, the process returns to step S73. On the other hand, if “YES” in the step S73, that is, if all the elements e of the waypoint list L are moved to the update result data 336, the integration process is executed in a step S77. When the integration process ends, the main process also ends. Note that details of the update process and the integration process will be described later, and a detailed description thereof will be omitted here.

  FIG. 28 is a flowchart of the update process that is a subroutine of the movement prediction program 318. When the update process of step S75 is executed in the main process, the processor 16 initializes the update buffer 330 in step S91. That is, nothing is stored in the update buffer 330. Subsequently, in step S93, the total number of elements e included in the waypoint list L is set in the variable M. For example, in the route point list L shown in FIG. 17, the variable M is set to “1”, and in the route point list L shown in FIG. 18B, the variable M is set to “2”. In the route point list L shown in FIG. 18D, the variable M is set to “4”, and in the route point list L shown in FIG. 18F, the variable M is set to “1”. Subsequently, in step S95, a variable m for identifying the element e is initialized. For example, “1” is set in the variable m.

Then, in step S97, the travel time t of the element e _m determines whether greater than predicted travel time T. For example, if the variable m is “1” and the waypoint list L is in the state shown in FIG. 17, it is determined whether or not the movement time t of the element e _{1 in} the first row is longer than the predicted movement time T. If "NO" in the step S97, that is the travel time t of the element e _m is not more than the predicted travel time T, the additional processing in step S99 is executed. The additional processing will be described with reference to the flowchart of FIG. 29, and thus detailed description thereof is omitted here.

For example, if the waypoint list L is in the state shown in FIG. 18D and the variable m is “1”, “YES” is determined in step S97, and the update result data 336 is updated in the element e _{1 in} step S101. Is moved. That is, the element e ₁ in FIG. 18D is moved as the element E ₁ of the update result data shown in FIG.

  Subsequently, in step S103, the variable m is incremented. That is, the variable m is incremented in order to execute the processing of steps S97 to S101 for another element e. Subsequently, in step S105, it is determined whether or not the variable m is larger than the variable M. That is, it is determined whether or not the process of step S99 or step S101 has been executed for all elements e in the waypoint list L. If “NO” in the step S105, that is, if the process of the step S99 or the step S101 is not executed for all the elements e, the process returns to the step S97.

  On the other hand, if “YES” in the step S105, that is, if the above-described processing is executed for all the elements e, the waypoint list L is updated based on the contents of the update buffer 330 in a step S107. . For example, when the process of step S107 is executed, the waypoint list L shown in FIG. 17 becomes the waypoint list L shown in FIG. Then, when the process of step S107 ends, the processor 16 returns to the main process. The processor 16 that executes the process of step S107 functions as an updating unit.

  When SFM is used in another embodiment, a process for estimating the next waypoint is added before step S99.

FIG. 29 is a flowchart of additional processing that is a subroutine of the movement prediction program 318. When step S99 of the update process is executed, a variable n for identifying the waypoint v is initialized in step S121. For example, “1” is set in the variable n. Then, in step S123, it reads out the moving transition probability p to the next waypoint Nv _n. For example, in the updating process, for example, the variable m is "1", via point list L is in the state of FIG. 17, if the next waypoint Nv _n is "v _1", "p = (v ₁ | V _b , v _p ) ”is read from the movement transition probability model DB 34. The processor 16 that executes the process of step S123 functions as a movement transition probability calculation unit.

  Subsequently, in step S125, it is determined whether or not the movement transition probability p is smaller than a threshold value. That is, it is determined whether or not the movement transition probability p obtained in step S123 is smaller than the threshold value. If “YES” in the step S125, as shown in the second row of FIG. 18A, for example, if the movement transition probability p is smaller than the threshold value, the process proceeds to a step S129.

On the other hand, if “NO” in the step S125, as shown in the fourth line of FIG. 18A, for example, if the moving transition probability p is equal to or greater than the threshold p, the element e including the via point Nv _n in the update buffer in the step S127. Add For example, in the waypoint series corresponding to the fourth row in FIG. 18A, the movement time t and the movement transition probability p are calculated, and the element e including them is added to the update buffer 330. The processor 16 that executes the process of step S127 functions as an adding unit.

  Subsequently, in step S129, the variable n is incremented. That is, the variable n is incremented to determine whether another waypoint v can be added to the update buffer 330 as the next waypoint Nv. Subsequently, in step S131, it is determined whether or not the variable n is larger than the total number N of via points v. That is, it is determined whether or not the processing of steps S123 to S127 has been executed for all via points v. If “NO” in the step S131, that is, if the above-described processing is not executed for all the via points v, the process returns to the step S123. On the other hand, if “YES” in the step S131, that is, if the above-described processing is executed for all the waypoints v, the additional processing is ended. Then, the processor 16 returns to the update process.

  FIG. 30 is a flowchart of the integration process that is a subroutine of the movement prediction program 318. When step S77 of the main process is executed, the arrival probability data 338 is initialized in step S151. That is, as shown in FIG. 20, “0” is set for each of the arrival probabilities P (v) to each waypoint v. Subsequently, in step S153, the total number of elements E included in the update result data is set in the variable K. For example, in the case of the action result data shown in FIG. 19B, “4” is set in the variable K. Subsequently, in step S155, a variable k for identifying the element E is initialized. For example, “1” is set in the variable k.

Then, in step S157, it extracts the last waypoint Fv included in the element _{E k.} For example, the update result data shown in FIG. 19 (B), if the variable k is "4", as the last route point Fv, via point v ₂ is extracted. Then, in step S159, Komu adding movement transition probabilities p element _{E k} the arrival probability P (Fv). For example, as described above, when the transit point v _b is extracted, in the arrival probability data shown in FIG. 20, the movement transition probability p (1.6%) of the transit point v ₂ is equal to the arrival probability P (v ₂ ). It is added. At this time, if the arrival probability P (v ₂ ) is “40.6%”, the added result is “42.2%”. The processor 16 that executes the process of step S159 functions as an arrival probability calculation unit.

  Subsequently, in step S161, the variable k is incremented. That is, the variable k is incremented in order to execute the processes of steps S157 and S159 for other elements E as well. Subsequently, in step S163, it is determined whether or not the variable k is larger than the variable K. That is, it is determined whether or not the processes of steps S157 and S159 have been executed for all elements E. If “NO” in the step S163, that is, if the above-described processing is not executed for all the elements E, the process returns to the step S157. On the other hand, if “YES” in the step S163, that is, if the above-described processing is executed for all the elements E, the integration processing is ended. Then, the processor 16 returns to the main process after completing the integration process.

  When SFM is used in another embodiment, steps S121, S129, and S131 are omitted in the additional process. In step S123, the movement transition probability p to the next via point is calculated based on the predicted next via point.

  FIG. 31 is a flowchart of the robot control process. For example, when the movement prediction process ends, a robot control process is executed. The processor 16 reads the arrival probability data 338 in step S181. That is, the arrival probability data 338 in which the arrival probability P for each waypoint v is recorded is read. Subsequently, in step S183, the arrival position of the person is predicted based on the arrival probability data. For example, the via point v having the highest arrival probability P is specified, and after the predicted travel time T, it is predicted that a person has reached the via point v. The processor 16 that executes the process of step S183 functions as a prediction unit.

  Subsequently, in step S185, the operation of the robot 14 is determined based on the predicted arrival position. For example, the behavior of the robot 14 is determined so as to move to the transit point v at the predicted travel time T and perform services such as providing information. Note that the processor 16 that executes the process of step S185 functions as an action determination unit.

  In step S187, an operation command is issued to the robot 14. That is, an action command for performing the above-described action is transmitted to the robot 14. And if the process of step S187 is complete | finished, a robot control process will be complete | finished.

  In another embodiment, the operation of the robot 14 may be determined based on the position of each via point v. For example, the robot 14 may be prevented from approaching a place with many waypoints, and an information providing service may be provided at a slightly separated place. Alternatively, the route guidance service may be provided by bringing the robot 14 close to a waypoint where the passages merge.

  FIG. 32 is a flowchart of a robot control process in another embodiment. When the robot control process is executed, the processor 16 reads each waypoint v from the waypoint DB 32 in step S201. For example, the positions of all via points shown in FIG. 15 are read out. Subsequently, in step S203, the action of the robot 14 is determined based on the provided waypoints. That is, as described above, the action of the robot 14 is determined based on the position of the read waypoint. Subsequently, in step S205, an operation command is issued to the robot 14, and the robot control process of another embodiment is terminated. Note that the processor 16 that executes the process of step S203 functions as an action determination unit.

  In this way, by confirming the operation of the robot 14 using the waypoints set based on the movement of people, it is possible to cause the robot 14 to perform an action considering the movement of people.

  In another embodiment, an action command may be issued to the robot 14 so that the robot 14 does not go to the waypoint v where the subject is most likely to go.

  In addition, the plurality of programs described in the present embodiment may be stored in the HDD of a data distribution server and distributed to a movement prediction device or the like in a system having the same configuration as that of the present embodiment via a network. Further, the storage medium may be sold or distributed in a state where these programs are stored in a storage medium such as an optical disk such as a CD, a DVD, or a BD (Blu-ray Disc), a USB memory, and a memory card. When the plurality of programs downloaded through the server and storage medium described above are installed in the movement prediction apparatus in the system having the same configuration as that of this embodiment, the same effect as that of this embodiment can be obtained.

  And, given in this specification, fixed time, distance, error, degree of freedom, radius, angle, frequency, size of partial area, Δ, space size, number of via points, predetermined distance, predetermined angle, Specific numerical values such as the predicted movement time T, the threshold value, the movement time t, the movement transition probability p, and the arrival probability P are merely examples, and can be appropriately changed according to needs such as product specifications.

10 ... Movement prediction device 12a-12f ... LRF
14 ... Robot 16 ... Processor 18 ... Memory 28 ... Movement locus DB
30 ... Movement direction distribution DB
32 ... via-point DB
34 ... Movement transition probability model DB

Claims (8)

Within a space that assumes a predetermined number of waypoints that are the targets of short-term movement, a route point sequence that includes multiple waypoints representing human movement, the travel time at that time, and the transition transition of the route point sequence A movement prediction device that predicts movement of a person using a waypoint list associated with a probability,
A time setting means for setting an estimated travel time for a person to move by the predicted time point,
When the travel time included in the via point list is less than the predicted travel time, based on the via point series included in the via point list, the moving transition probability calculating means for repeatedly calculating the moving transition probability to the next via point ,
Update means for updating the route point series, the movement time and the movement transition probability in the route point list each time the movement transition probability calculated by the movement transition probability calculation unit is equal to or greater than a threshold value;
When the travel time included in the via point list is equal to or longer than the predicted travel time, the arrival probability calculating means for calculating the arrival probability to each via point based on the via point list, and each of the arrival probability calculating means calculated by the arrival probability calculating means, A movement prediction apparatus comprising prediction means for predicting the arrival position of a person due to movement based on the arrival probability to a via point. Each time the movement transition probability calculated by the movement transition probability calculation means is equal to or greater than a threshold, a route point sequence associated with the movement time to the next route point and the movement transition probability to the next route point is It further includes an additional means for adding to the point list,
The movement prediction apparatus according to claim 1, wherein the arrival probability calculation means calculates an arrival probability based on a movement transition probability in which a reaching via point is associated with the same via point series. A waypoint setting means for setting a predetermined number of waypoints in the space;
Recording means for recording the movement trajectory of a person,
A discriminating means for discriminating a moving direction of a person for each of a plurality of partial areas included in the space based on the movement trajectory recorded by the recording means;
An association means for associating the movement direction discriminated by the discrimination means with a predetermined number of via points set by the via point setting means; and the transit point based on the movement direction associated by the association means The movement prediction apparatus according to claim 1, further comprising position update means for updating positions of a predetermined number of via points set by setting. An order calculating means for calculating the order of moving via points based on the movement trajectory recorded by the recording means, and a movement transition probability model for moving via points based on the order calculated by the order calculating means. Further comprising a construction means for construction,
The movement prediction apparatus according to claim 3, wherein the movement transition probability calculation unit calculates a movement transition probability to the next waypoint based on the movement transition probability model constructed by the construction unit. A robot control device including the movement prediction device according to claim 1,
A robot control apparatus comprising behavior determining means for determining a behavior of the robot based on a position reached by a person predicted by the prediction means. A robot control device including the movement prediction device according to claim 3,
A robot control apparatus, comprising: an action confirming unit that determines an action of the robot based on a predetermined number of waypoints updated by the position updating unit. Within a space that assumes a predetermined number of waypoints that are the targets of short-term movement, a route point sequence that includes multiple waypoints representing human movement, the travel time at that time, and the transition transition of the route point sequence A processor of a movement prediction device that predicts movement of a person using a list of waypoints associated with probabilities;
A time setting means for setting an estimated travel time for a person to move by the predicted time point,
When the travel time included in the via point list is less than the predicted travel time, based on the via point series included in the via point list, the moving transition probability calculating means for repeatedly calculating the moving transition probability to the next via point ,
Update means for updating the route point series, the movement time and the movement transition probability in the route point list each time the movement transition probability calculated by the movement transition probability calculation unit is equal to or greater than a threshold value;
When the travel time included in the via point list is equal to or longer than the predicted travel time, the arrival probability calculating means for calculating the arrival probability to each via point based on the via point list, and each of the arrival probability calculating means calculated by the arrival probability calculating means, A movement prediction program that functions as a prediction means for predicting the arrival position of a person due to movement based on the probability of reaching a via point. Within a space that assumes a predetermined number of waypoints that are the targets of short-term movement, a route point sequence that includes multiple waypoints representing human movement, the travel time at that time, and the transition transition of the route point sequence A movement prediction method of a movement prediction apparatus that predicts movement of a person using a waypoint list associated with a probability,
Set the estimated travel time for people to move by the time of the forecast,
When the travel time included in the waypoint list is less than the predicted travel time, based on the waypoint series included in the waypoint list, repeatedly calculate the movement transition probability to the next waypoint,
Every time the calculated moving transition probability is equal to or higher than the threshold, the waypoint series, the moving time and the moving transition probability in the waypoint list are updated.
When the travel time included in the via point list is equal to or longer than the predicted travel time, the arrival probability at each via point is calculated based on the via point list, and the movement is performed based on the calculated arrival probability at each via point. A movement prediction method that predicts the arrival position of a person.

Images