JP2012203646A - Flow-state discrimination device, flow-state discrimination method, flow-state discrimination program, and robot control system using the device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a robot to behave naturally in accordance with a flow of people by discriminating the state of a flow of a representative movement locus pre-set in an environment to allow prediction of a flow of people within the environment.SOLUTION: A robot control system 100 includes a central control device 10, a robot 12, and a position detection system 14. The central control device 10 has a flow database 118 for storing information relating to a representative movement locus, uses a large amount of accumulated movement behavior information on people and position history of people (96) within an environment appropriately detected by the position detection system 14 to discriminate a state of flow of people for each representative movement locus, and predicts occurrence, end, continuation and the like of the flow of people in the environment. Then the behavior of the robot 12 is determined on the basis of the information on the flow of people predicted by the central control device 10.

Description

  The present invention relates to a flow state discriminating apparatus, a flow state discriminating method, a flow state discriminating program, and a robot control system using them, and in particular, for example, using the movement behavior information of a plurality of people accumulated in advance, The present invention relates to a novel flow state determination device, a flow state determination method, a flow state determination program, and a robot control system using them, which predict a flow.

As an example of the prior art, Non-Patent Document 1 discloses a technology related to a network robot that combines an environment-installed ubiquitous sensor capable of stable sensing over a wide area and a humanoid robot capable of providing intuitive information. It is disclosed. In the technique of Non-Patent Document 1, the use state of space and the pattern of global behavior of people are analyzed from the movement trajectory information (movement behavior information) of people accumulated in large quantities. Then, by using the analysis result, the location where the person is likely to go and the local behavior that is likely to occur is estimated, so that the robot can automatically select an appropriate service provider.
Takayuki Kanda, Dylan F. Glas, Masahiro Shiomi, Norihiro Hamada, “Structure of Environmental Information for Robots that Provide Services to Moving People,” (Journal of Robotics Society of Japan 27 (4), 449-459, 2009-05- 15)

  In the technology of Non-Patent Document 1, a place where a person is likely to go or a local action that is likely to occur is predicted, that is, a single action of a target person is predicted, and the flow of the person is not predicted. . In other words, a technology for predicting the flow of people in the environment by using the movement behavior information of a large amount of people has not been realized so far.

  Therefore, a main object of the present invention is to provide a novel flow state determination device, a flow state determination method, a flow state determination program, and a robot control system using them.

  Another object of the present invention is to provide a flow state discriminating apparatus, a flow state discriminating method, a flow state discriminating program and a robot control system using them, which can predict the flow of people in the environment.

  The present invention employs the following configuration in order to solve the above problems. Note that reference numerals in parentheses and supplementary explanations indicate correspondence with embodiments described later in order to help understanding of the present invention, and do not limit the present invention.

  A first aspect of the present invention is a flow state discriminating apparatus that predicts the flow of people in an environment, and relates to one or more representative movement trajectories created in advance based on movement behavior information of a plurality of people accumulated in the environment Storage means for storing information, detection means for detecting position histories of a plurality of persons existing in the environment, and any representative movement for each of a plurality of persons existing in the environment based on the position histories detected by the detection means A trajectory determination unit that determines whether the trajectory belongs, an extraction unit that extracts a feature amount from a person determined to belong to the representative travel trajectory by the determination unit, and a feature amount extracted by the extraction unit. On the basis of this, it is a flow state discriminating device provided with a state discriminating unit that discriminates the state of the human flow for each of the representative movement trajectories.

  In the first invention, the flow state discriminating device (10) includes storage means (118) for storing information relating to the representative movement trajectory (F), and predicts the occurrence, termination, and continuation of human flow in the environment. . The representative movement trajectory indicates a typical flow observed in the environment, and is created in advance based on the movement behavior information of people accumulated in large quantities. The detection means (14, 112, 116, S7) detects, for example, the most recent position history for several seconds to several tens of seconds of all persons (96) existing in the environment. The locus determination means (112, S9) calculates the similarity between each person's position history and each representative movement locus using a technique such as DP matching, for example, so that each person belongs to any representative movement locus. It is determined whether or not. The extraction means (112, S17) extracts, for each representative movement trajectory, feature quantities such as the total number of persons, the average movement speed, the human density, and the number of groups from the people belonging to the representative movement trajectory. The state discriminating means (112, S19), based on the feature amount extracted by the extracting means, for each of the representative movement trajectories, “soon flow will occur”, “no sooner flow will occur”, “ The state of the person's flow such as “continue” or “the situation without flow continues” is determined.

  According to the first aspect of the present invention, it is possible to predict the flow of a person in the environment by determining the state of the flow of the representative movement locus set in the environment in advance.

  A second invention is dependent on the first invention, and the state determining means includes a discriminator using a machine learning method, which is created using teacher data to which a label indicating a human flow state is attached. .

  In the second invention, the state discriminating means (112, S19) includes a discriminator using a machine learning technique such as SVM. The discriminator extracts, for example, a feature amount from a plurality of teacher data previously assigned with labels indicating the state of a human flow such as “coming soon” or “no sooner”, and learning. Built.

  According to the second invention, it is possible to appropriately determine the flow state of each representative movement trajectory, and to accurately predict the flow of people in the environment.

  A third invention is a flow state determination method for predicting the flow of people in the environment, (a) storing movement behavior information of a plurality of people in the environment, and (b) storing in step (a). Based on the movement behavior information, a representative movement route through which a person in the environment passes is set as one or more representative movement trajectories, and (c) the flow of the person for each of the representative movement trajectories set in step (b) (D) detecting position histories of a plurality of persons existing in the environment, and (e) based on the position histories of the plurality of persons detected in step (d). Then, it is determined to which representative movement trajectory each of the plurality of persons belongs, and (f) for each of the representative movement trajectories, the feature amount from the person determined to belong to the representative movement trajectory in step (e). And (g) representative movement trajectory For each, by determining the extracted feature quantity in step (f) by discriminator generated in step (c), to determine the state of the flow of people, the flow state determination method.

  In the third invention, the generation, termination, and continuation of a human flow in the environment are predicted using the movement behavior information of the people accumulated in a large amount in the environment. First, in step (a), using the position detection system (14) or the like, for example, for about one week, the movement behavior information of all persons existing in the environment is detected and stored in the database. In step (b), based on the accumulated movement behavior information, a representative movement route through which a person in the environment passes is set as one or a plurality of representative movement trajectories (F). In step (c), for example, using a machine learning technique such as SVM, for each representative movement trajectory, the state of human flow such as “coming soon” or “no sooner” is determined. Create a classifier. In step (d), the most recent position history for several seconds to several tens of seconds of all persons (96) existing in the environment is detected (S7). In step (e), for example, by using a technique such as DP matching, the similarity between each person's position history and each representative movement trajectory is calculated to determine which representative movement trajectory each person belongs to. (S9). In step (f), for each representative movement locus, feature quantities such as the total number of people, average movement speed, human density, and number of groups are extracted from the people belonging to the representative movement locus (S17). Then, in step (g), the feature quantity extracted in step (f) is discriminated for each of the representative movement trajectories by the discriminator created in step (c), thereby “coming soon” or “coming soon”. The flow state of the person such as “no flow” is determined (S19).

  According to the third aspect of the invention, it is possible to predict the flow of a person in the environment by determining the state of the flow of the representative movement trajectory set in the environment in advance.

  According to a fourth aspect of the present invention, there is provided a processor for a flow state determination device having a storage unit that stores information relating to one or more representative movement trajectories created in advance based on movement behavior information of a plurality of people accumulated in an environment. Based on the position history detected by the detection means for detecting the position history of a plurality of people existing in the environment, the representative movement trajectory for each of the plurality of persons existing in the environment is determined. For each of the trajectory determination means and the representative movement trajectory, the representative movement based on the extraction means for extracting the feature amount from the person determined to belong to the representative movement trajectory by the determination means, and the feature amount extracted by the extraction means It is a flow state determination program that functions as a state determination unit that determines the state of a person's flow for each of the trajectories.

  In the fourth invention as well, as in the first invention, it is possible to predict the flow of people in the environment by determining the state of the flow of the representative movement trajectory set in the environment in advance.

  A fifth invention is a robot control system including a robot that is arranged in an environment where people coexist and provides a service, and is created in advance based on movement behavior information of a plurality of people accumulated in the environment 1 or Storage means for storing information on a plurality of representative movement trajectories, detection means for detecting position histories of a plurality of persons existing in the environment, and a plurality of persons existing in the environment based on the position histories detected by the detection means Trajectory determination means for determining which representative movement trajectory each belongs to, extraction means for extracting a feature amount from each person determined to belong to the representative movement trajectory by the determination means, and extraction means for each of the representative movement trajectories State determination means for determining the state of human flow for each of the representative movement trajectories based on the feature amount extracted by Based on the result of the judgment by means comprises control means for controlling the operation of the robot, a robot control system.

  In the fifth invention, the robot control system (100) includes a robot (12) that provides services such as guidance and luggage transportation in an environment where people coexist. The storage means (10, 118) stores information related to the representative movement locus (F). The representative movement trajectory indicates a typical flow observed in the environment, and is created in advance based on the movement behavior information of people accumulated in large quantities. The detection means (14, 112, 116, S7) detects, for example, the most recent position history for several seconds to several tens of seconds of all persons (96) existing in the environment. The trajectory determination means (10, 112, S9) calculates the similarity between each person's position history and each representative movement trajectory using a technique such as DP matching, for example, so that each person can be assigned to which representative movement trajectory. Determine if it belongs. The extraction means (10, 112, S17) extracts, for each representative movement trajectory, feature quantities such as the total number of persons, the average movement speed, the human density, and the number of groups from the people belonging to the representative movement trajectory. The state discriminating means (10, 112, S19) is based on the feature amount extracted by the extracting means, and “a flow will soon occur”, “a flow will soon disappear”, “there is a flow” for each representative movement trajectory. The state of human flow such as “the situation continues” or “the situation without a flow continues” is determined. The control means (10, 62, 112, S23) controls the operation of the robot based on the result determined by the state determination means, that is, the predicted human flow in the environment. For example, if the robot's task is tourist information, move the robot so that it passes through a busy place, that is, a place where there is a continuous flow, a place where a flow will soon occur, or a place where there are multiple flows. Control.

  According to the fifth aspect of the present invention, it is possible to predict the flow of the person in the environment by determining the flow state of the representative movement trajectory set in the environment in advance. Can be executed.

  According to the present invention, the flow of a person in the environment can be predicted by determining the flow state of the representative movement trajectory set in the environment in advance.

  The above object, other objects, features, and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

It is an illustration figure which shows the structure of the robot control system of one Example of this invention. It is an illustration figure which shows a mode that the external appearance of the mobile robot of FIG. 1 was seen from the front. It is a block diagram which shows the electric constitution of the mobile robot of FIG. It is an illustration figure which shows the mode of the environment where the position detection system of FIG. 1 was applied roughly. It is an illustration figure which shows the structure of the position detection system of FIG. It is an illustration figure which shows an example of the map of the environment where the robot control system of FIG. 1 is applied. It is an illustration figure which shows an example of the typical flow (representative movement locus | trajectory) observed in the environment of FIG. It is an illustration figure which shows an example of a mode that the environment of FIG. 6 was divided into the some area | region. It is an illustration figure which shows an example of the representative movement locus | trajectory set to one of the area | regions of FIG. It is an illustration figure which shows an example of the table memorize | stored in the flow database of FIG. It is a flowchart which shows an example of the whole process which CPU of the central control apparatus of FIG. 1 performs.

  Referring to FIG. 1, a robot control system 100 according to an embodiment of the present invention includes a central control device 10, a robot 12, and a position detection system 14, and includes a plurality of shopping centers, event venues, museums, amusement parks, and the like. Applies to various environments where people exist. In the robot control system 100, the central control device 10 functions as a flow state discriminating device, and predicts the generation, termination, and continuation of a human flow in the environment by using the movement behavior information of people accumulated in large quantities in advance. To do. Then, the behavior of the robot 12 is determined based on the human flow information predicted by the central controller 10.

  First, the robot 12 will be described with reference to FIGS. 2 and 3. The robot 12 is an interaction-oriented robot (communication robot) that performs a communication action with a person using physical movements or speech. The robot 12 provides various services (executes tasks) such as guidance, luggage transportation, customer attraction, entertainment, cleaning, and patrol monitoring within the environment in which it is arranged.

  FIG. 2 is a front view showing the appearance of the robot 12, and the hardware configuration of the robot 12 will be described with reference to FIG. The robot 12 includes a carriage 20, and two wheels 22 and one slave wheel 24 that autonomously move the robot 12 are provided on the lower surface of the carriage 20. The two wheels 22 are independently driven by a wheel motor 26 (see FIG. 3), and can move the robot 12 in any direction, front, back, left, and right. The slave wheel 24 is an auxiliary wheel that assists the wheel 22. Thus, the robot 12 can move freely within the environment in which it is placed. However, the moving mechanism of the robot 12 is not limited to the wheel type, and a known moving mechanism can be appropriately employed. For example, a biped walking type moving mechanism can also be employed.

  A cylindrical sensor mounting panel 28 is provided on the carriage 20, and an infrared distance sensor 30 is mounted on the sensor mounting panel 28. The infrared distance sensor 30 measures the distance between the robot 12 and a surrounding object (such as a person or an obstacle).

  Further, the body 32 is provided on the sensor mounting panel 28 so as to stand upright. The above-described infrared distance sensor 30 is further provided on the front center upper portion of the body 32 (a position corresponding to the chest). This measures the distance to the person mainly in front of the robot 12. The body 32 is provided with one omnidirectional camera 34. The omnidirectional camera 34 is provided, for example, on a support column 36 extending from substantially the center of the upper end on the back side. The omnidirectional camera 34 captures the surroundings of the robot 12 and is distinguished from an eye camera 60 described later. As this omnidirectional camera 34, for example, a camera using a solid-state imaging device such as a CCD or a CMOS can be adopted. In addition, the installation positions of the infrared distance sensor 30 and the omnidirectional camera 34 are not limited to the portions, and can be changed as appropriate.

  Upper arms 40R and 40L are provided by shoulder joints 38R and 38L, respectively, at upper end portions (positions corresponding to shoulders) on both side surfaces of the body 32. Although illustration is omitted, each of the shoulder joints 38R and 38L has three orthogonal degrees of freedom. That is, the shoulder joint 38R can control the angle of the upper arm 40R around each of the three orthogonal axes. A certain axis (yaw axis) of the shoulder joint 38R is an axis parallel to the longitudinal direction of the upper arm 40R, and the other two axes (pitch axis and roll axis) are axes orthogonal to each other from different directions. Similarly, the shoulder joint 38L can control the angle of the upper arm 40L around each of the three orthogonal axes. A certain axis (yaw axis) of the shoulder joint 38L is an axis parallel to the longitudinal direction of the upper arm 40L, and the other two axes (pitch axis and roll axis) are axes orthogonal to each other from different directions.

  Further, forearms 44R and 44L are provided at the tips of the upper arms 40R and 40L via elbow joints 42R and 42L, respectively. Although illustration is omitted, each of the elbow joints 42R and 42L has one degree of freedom, and the angle of the forearms 44R and 44L can be controlled around the axis (pitch axis).

  Spheres 46R and 46L corresponding to hands are fixedly provided at the tips of the forearms 44R and 44L, respectively. However, if a finger or palm function is required, a “hand” in the shape of a human hand can be used.

  Although illustration is omitted, a contact sensor (FIG. 3) is provided on each of the front surface of the carriage 20, a portion corresponding to the shoulder including the shoulder joints 38R and 38L, the upper arms 40R and 40L, the forearms 44R and 44L, and the spheres 46R and 46L. Is comprehensively shown as a contact sensor 48.). The contact sensor 48 on the front surface of the carriage 20 detects contact of a person or another obstacle to the carriage 20. Therefore, when there is contact with an obstacle while the robot 12 is moving, it can be detected, and the driving of the wheel 22 can be stopped immediately and the movement of the robot 12 can be stopped suddenly. The other contact sensors 48 mainly detect whether or not a person has touched each part of the robot 12. The installation position of the contact sensor 48 is not limited to these, and may be provided at an appropriate position (chest, abdomen, side, back, waist, etc.).

  A neck joint 50 is provided at the upper center of the body 32 (a position corresponding to the neck), and a head 52 is further provided thereon. Although illustration is omitted, the neck joint 50 has a degree of freedom of three axes, and the angle can be controlled around each of the three axes. A certain axis (yaw axis) is an axis directed directly above (vertically upward) of the robot 12, and the other two axes (pitch axis and roll axis) are axes orthogonal to each other in different directions.

  The head 52 is provided with a speaker 54 at a position corresponding to the mouth. The speaker 54 is used for the robot 12 to communicate with people around it by voice or sound. Further, microphones 56R and 56L are provided at positions corresponding to the ears. Hereinafter, the microphone 56R corresponding to the right ear and the microphone 56L corresponding to the left ear may be collectively referred to as “microphone 56”. The microphone 56 captures ambient sounds, particularly a voice of a person who is a target of communication. Further, eyeball portions 58R and 58L are provided at positions corresponding to the eyes. Eyeball portions 58R and 58L include eye cameras 60R and 60L, respectively. Hereinafter, the right eyeball portion 58R and the left eyeball portion 58L may be collectively referred to as “eyeball portion 58”, and the right eye camera 60R and the left eye camera 60L are collectively referred to as “eye camera 60”. Sometimes.

  The eye camera 60 captures a human face approaching the robot 12 and other parts or objects, and captures a corresponding video signal. As the eye camera 60, a camera similar to the omnidirectional camera 34 described above can be used. For example, the eye camera 60 is fixed in the eyeball part 58, and the eyeball part 58 is attached to a predetermined position in the head 52 via an eyeball support part (not shown). Although illustration is omitted, the eyeball support portion has two degrees of freedom, and the angle can be controlled around each of these axes. For example, one of the two axes is an axis (yaw axis) in a direction toward the top of the head 52, and the other is orthogonal to one axis and the direction in which the front side (face) of the head 52 faces. It is an axis (pitch axis) in the direction to be. By rotating the eyeball support portion around each of these two axes, the tip (front) side of the eyeball portion 58 or the eye camera 60 is displaced, and the camera axis, that is, the line-of-sight direction is moved. The installation positions of the speaker 54, the microphone 56, and the eye camera 60 described above are not limited to these, and may be provided at appropriate positions.

  FIG. 3 is a block diagram showing an electrical configuration of the robot 12. As shown in FIG. 3, the robot 12 includes a CPU 62 that controls the whole. The CPU 62 is also called a microcomputer or a processor, and is connected to the memory 66, the motor control board 68, the sensor input / output board 70, the audio input / output board 72, and the like via the bus 64.

  Although not shown, the memory 66 includes a ROM, an HDD, and a RAM. A control program for the robot 12 is stored in advance in the ROM and HDD. For example, an action control program for executing communication actions with a person, a movement control program for moving in an environment, and a communication for transmitting / receiving necessary information (such as control data) to / from an external computer Programs. The ROM or HDD appropriately stores voice or voice data (speech synthesis data) to be generated from the speaker 54 when executing the communication action, map data of the environment to be arranged, and the like. The RAM is used as a work memory or a buffer memory.

  The motor control board 68 is configured by, for example, a DSP, and controls driving of motors of the axes such as the arms, the neck joint 50, and the eyeball unit 58. That is, the motor control board 68 receives two control data from the CPU 62 and controls two motors for controlling the respective angles of the two axes of the right eyeball portion 58R (collectively, “right eyeball motor” in FIG. 3) 74. Control the rotation angle. Similarly, the motor control board 68 receives control data from the CPU 62, and controls two angles of the two axes of the left eyeball part 58L (in FIG. 3, collectively referred to as “left eyeball motor”). The rotation angle of 76 is controlled.

  The motor control board 68 receives control data from the CPU 62, and includes three motors for controlling the angles of the three orthogonal axes of the right shoulder joint 38R and one motor for controlling the angle of the right elbow joint 42R. The rotation angles of a total of four motors 78 (collectively referred to as “right arm motor” in FIG. 3) 78 are adjusted. Similarly, the motor control board 68 receives control data from the CPU 62 and includes three motors for controlling the angles of the three orthogonal axes of the left shoulder joint 38L and one motor for controlling the angle of the left elbow joint 42L. The rotation angles of a total of four motors (collectively referred to as “left arm motor” in FIG. 3) 80 are adjusted.

  Furthermore, the motor control board 68 receives the control data from the CPU 62 and controls three motors for controlling the angles of the three orthogonal axes of the neck joint 50 (in FIG. 3, collectively referred to as “head motors”). The rotation angle of 82 is controlled. Furthermore, the motor control board 68 receives control data from the CPU 62 and controls the rotation angle of two motors 26 (hereinafter collectively referred to as “wheel motors”) 26 that drive the wheels 22.

  In this embodiment, stepping motors or pulse motors are used for the motors other than the wheel motor 26 in order to simplify the control. However, as with the wheel motor 26, a DC motor may be used.

  Similarly, the sensor input / output board 70 is also configured by a DSP, and takes in signals from each sensor and gives them to the CPU 62. That is, data relating to the reflection time from each of the infrared distance sensors 30 is input to the CPU 62 through the sensor input / output board 70. The video signal from the omnidirectional camera 34 is input to the CPU 62 after being subjected to predetermined processing by the sensor input / output board 70 as necessary. Similarly, the video signal from the eye camera 60 is also input to the CPU 62. Further, signals from the plurality of contact sensors 48 described above are given to the CPU 62 via the sensor input / output board 70.

  Similarly, the voice input / output board 72 is also configured by a DSP, and a voice or voice according to voice synthesis data provided from the CPU 62 is output from the speaker 54. Also, the voice input from the microphone 56 is taken into the CPU 62 via the voice input / output board 56.

  The CPU 62 is connected to the communication LAN board 84 via the bus 64. The communication LAN board 84 is constituted by a DSP, gives transmission data sent from the CPU 62 to the wireless communication device 86, and transmits the transmission data from the wireless communication device 86 to an external computer via a network such as a wireless LAN. . The communication LAN board 84 receives data via the wireless communication device 86 and gives the received data to the CPU 62. That is, the communication LAN board 84 and the wireless communication device 86 allow the robot 12 to perform wireless communication with an external computer (the central control device 10 and the position detection system 14).

  Next, the position detection system 14 will be described. FIG. 4 is an illustrative view schematically showing an environment where the position detection system 14 is applied, and FIG. 5 is an illustrative view showing an electrical configuration of the position detection system 14. As shown in FIGS. 4 and 5, the position detection system 14 includes a computer 90 that executes a position detection process, a plurality of laser range finders (LRF) 92 and a wireless ID tag reader 94 installed in the environment, and each person 96. A wireless ID tag 98 attached to the device is included.

  The computer 90 of the position detection system 14 is a general-purpose computer such as a personal computer or a workstation. A plurality of LRFs 92 and a wireless ID tag reader 94 are connected to the computer 90 via a general-purpose interface such as RS-232C. Data detected by these devices 92 and 94 is transmitted to the computer 90, and appropriately stored in a predetermined area of the memory of the computer 90 or an external database together with the detection time. Further, in the memory of the computer 90, position data of the devices 92 and 94 represented by an XY two-dimensional plane coordinate system and map data of the environment are stored in advance.

  The LRF 92 is a sensor that measures the distance from the time it takes to irradiate a laser and is reflected back to the object and returns to the object. For example, the laser irradiated from the transmitter is reflected by a rotating mirror and the front is fan-shaped. Scan at a fixed angle. The LRF 92 is installed on a pedestal having a height of about 90 cm, for example. As LRF92, LRF (model LMS 200) manufactured by SICK, LRF (model UTM-30LX) manufactured by HOKYUYO, or the like can be used.

  In the position detection system 14, the computer 90 acquires sensor information (distance information) from the LRF 92 at a predetermined frequency (for example, 37.5 times / second), and uses a particle filter or a human shape model to detect the current position of the person 96. Estimate changes. For example, when scanned by the LRF 92, a visible area where the person 96 is not present, a shadow area where the person 96 is present, and an edge of the person 96 are detected. Further, particles are evenly distributed in the virtual space corresponding to the real space, and the likelihood is obtained for each LRF 92. Furthermore, each particle is updated by integrating the likelihood for each LRF 92. Then, a change in the current position of the person 96 is estimated from each updated particle. The likelihood is a constant value in the visible area, and is the sum of the constant value and the likelihood of the edge in the shadow area. Based on the change of the current position estimated in this way, the position of the person 96 is obtained, and the numerical value (position data) indicating the plane coordinates of the position is associated with the time data, and the movement action information (movement trajectory data). It accumulates separately for each person 96. Further, for example, the movement speed of each person 96 is calculated by differentiating the amount of change in the movement trajectory of each person 96 at regular intervals with time. Note that details of such person position detection using LRF are described in Japanese Patent Application Laid-Open No. 2009-168578 filed earlier by the present applicant.

  The wireless ID tag 98 is attached to each person 96 present in the environment. As the wireless ID tag 98, for example, an RFID (Radio Frequency Identification) tag can be used. RFID is an automatic recognition technology using a non-contact IC tag using electromagnetic waves. Specifically, the RFID tag includes an IC chip, an antenna, and the like provided with a memory for identification information, a control circuit for communication, and the like. Information capable of identifying the person 96 is stored in advance in the memory of the RFID tag, and the identification information is output from the antenna by electromagnetic waves or radio waves having a predetermined frequency.

  The wireless ID tag reader 94 is installed on, for example, a rail laid on the ceiling in the environment, and detects output information from the wireless ID tag 98 described above. Specifically, the wireless ID tag reader 94 receives a radio wave superimposed with identification information transmitted from the wireless ID tag 98 via an antenna, amplifies the radio signal, and separates the identification information from the radio signal. The information is demodulated (decoded). The wireless ID tag reader 94 detects the distance from the wireless ID tag 98 based on the detected radio wave intensity.

  In this embodiment, the computer 90 acquires sensor information (identification information and distance information) from the wireless ID tag 98 detected by the wireless ID tag reader 94 at a frequency of, for example, 5 times / second. And based on the acquired identification information, the person 96 who possesses the wireless ID tag 98 is specified. Further, a person who possesses the wireless ID tag 98 based on the distance information acquired from the three wireless ID tag readers 94 for the same wireless ID tag 98 and the position data of the wireless ID tag reader 94 stored in advance. 96 position coordinates are specified.

  Then, the computer 90 of the position detection system 12 integrates the sensor information (positioning data) detected by the devices 92 and 94 or information estimated from the sensor information, and determines the position coordinates (X, Y) of each person 96. The data is stored in a memory or database in association with the detection time t. That is, for each person 96 (identification information or ID number), the movement trajectory is stored as time series data. Such movement behavior information is transmitted to the central control device 10 at any time in response to a request from the central control device 10 or every predetermined time.

  The position detection system 14 is not limited to the above-described configuration as long as it can detect the position coordinates or position history of each person 96. For example, an infrared sensor, a video camera, a floor sensor, or a GPS is used. It may be used as appropriate. If it is not necessary to individually identify the person 96, the wireless ID tag reader 94 and the wireless ID tag 98 are not necessarily provided.

  Returning to FIG. 1, the central control device (flow state determination device) 10 includes a computer 110, and the movement behavior information of each person 96 transmitted as needed from the position detection system 14 as described above, and a large amount of information is stored in advance. Predicting the state of the flow of people in the environment (such as the occurrence, termination, and continuation of the flow) based on the movement behavior information of the people.

  The computer 110 is, for example, a general-purpose personal computer or workstation including a CPU 112, a memory 114, a communication device 116, an input device, a display device, and the like. The memory 112 of the computer 110 includes an HDD, a ROM, and a RAM. A program for controlling the operation of the central control device 10 is stored in the HDD or ROM, and the CPU 112 executes processing according to this program. For example, the HDD or ROM includes a prediction program (trajectory determination program, feature amount extraction program, flow state determination program, etc.) for predicting the flow of people in the environment, and an external computer such as the robot 12 or the position detection system 14. A communication program for transmitting / receiving necessary information (such as control data of the robot 12 and movement action information of the person 96) is stored. The RAM is used as a work area or a buffer area for the CPU 112. The communication device 116 is for communicating with an external computer via a network such as a wireless LAN or the Internet.

  A flow database (DB) 118 is connected to the computer 110 of the central controller 10. As will be described in detail later, the flow DB 118 is a database that stores trajectory data of the representative movement trajectory F and the like (see FIG. 10). The flow DB 118 may be provided outside the computer 110 as shown in FIG. 1, or may be provided in an HDD or the like in the computer 110.

  As described above, the central control device 10 uses people's movement behavior information accumulated in large quantities in advance in the application environment. In this embodiment, a discriminator for discriminating the representative movement trajectory F and the state of the person's flow is created in advance based on the movement behavior information of the people accumulated in large quantities. FIG. 6 is a map showing an example of an environment (commercial facility) to which the central controller 10 (robot control system 100) is applied. As shown in FIG. 6, there is an escalator (Esc) in the center of this environment, and a staircase on the left side. In addition, a hatched portion in FIG. 6 is a portion where structures such as stores and pillars exist and do not serve as a passage for the person 96 or the robot 12.

  Hereinafter, the representative movement trajectory F and the discriminator created in advance will be described. First, the position detection system 14 and the like are used to detect the position histories (movement behavior information) of all persons existing in the environment for a certain period (for example, one week) and accumulate them in the database. When the accumulated movement behavior information is analyzed, it can be understood that the path through which people in the environment pass (that is, the path along which the person flows) is determined to some extent, so a representative movement path is set as the representative movement path F. . As shown in FIG. 7, as a result of analyzing the accumulated movement behavior information, as shown in FIG. 7, there is a main flow that moves from right to left in the environment or moves from left to right. Suppose that it was observed. It is also a secondary flow to avoid a moderate flow that moves from a staircase or escalator to the main flow, from the main flow to the staircase or escalator, or moves along a pillar, or a stationary person. Suppose the existence of a simple flow is observed. In such a case, each of these observed representative flows (movement paths) can be set as the representative movement locus F. Note that there may be one or more representative movement trajectories F.

  The entire applicable environment can be set as one region, and each of the representative flows as described above can be set as the representative movement trajectory F. However, the environment is divided into a plurality of regions, and the representative flows as described above. Each part of the simple flow may be used as the representative movement trajectory F. For example, as shown in FIG. 8, the environment can be divided into a plurality of areas and managed, and a representative movement trajectory F can be set for each area as shown in FIG. Note that the environment classification method, that is, the size, number, and shape of the region is appropriately set according to the application environment. For example, it can be regularly classified into a lattice shape, or irregularly divided into an indefinite shape. You can also Such an environment classification may be set manually in detail by a developer or an administrator, or may be automatically set using a k-means method or the like.

  Information regarding such a representative movement trajectory F is stored in the flow DB 118. FIG. 10 shows an example of a table stored in the flow DB 118. As illustrated in FIG. 10, the flow DB 118 stores trajectory data of each representative movement trajectory F in association with the ID set for each representative movement trajectory F. The trajectory data includes coordinates (X, Y) constituting the movement trajectory. Further, the trajectory data may include an action mode (A) performed at the coordinates. The behavior mode (A) includes “stop”, “slowly travel”, “swiftly travel” and the like estimated based on the moving speed. For example, the trajectory data of “(Xa1, Ya1, Aa1), (Xa2, Ya2, Aa2),...” Is stored in association with the representative movement trajectory F whose ID is “Fa”.

  When the representative movement trajectory F is set (created), a discriminator for discriminating the state of human flow is created for each representative movement trajectory F using a machine learning technique. As a machine learning method, a known method such as a support vector machine (SVM), a neural network, or boosting can be used. In the following, a case where SVM is used as an example will be described, but since SVM is a widely general method, detailed description thereof is omitted.

  The SVM is constructed by extracting feature values from a plurality of pre-labeled teacher data (data sets) and learning. Here, a plurality of teacher data is created in advance by a developer or the like, and each of the teacher data is manually provided with a label about a human flow state. The teacher data is, for example, time-series data such as a movement trajectory for n seconds, and a plurality of developers or the like observe the accumulated movement behavior information. Are treated as However, it is also possible to define teacher data mechanically.

  The labels given to the teacher data are “flow will soon occur”, “flow will soon stop (terminate)”, “flow will continue (continue)”, “no flow will continue” , “There is a flow”, “there is no flow”, “the stagnation of the flow (congestion)”, “the stagnation of the flow is resolved”, and the like. In addition, “person flow” expresses the movement aspect of multiple people from the overall viewpoint, and what kind of state is judged as “flow” or “no flow”, etc. Further, the developer or the like can appropriately set the state of the label to be used.

  Further, the feature quantity extracted from the teacher data includes the total number of people, the average moving speed, the human density (number of people per unit length), the number of groups, and the like existing in the trajectory (flow). However, the type and number of feature quantities are not uniquely determined, and various statistical quantities can be used as feature quantities according to the application environment and the representative movement trajectory F.

  The central control device 10 uses a discriminator based on such a machine learning technique to discriminate the state of the human flow on each representative movement trajectory F and predict the human flow in the environment. Hereinafter, a method (operation) in which the central controller 10 predicts the flow of people in the environment will be described.

  First, each representative 96 is assigned to one of the representative movement trajectories F by determining which representative movement trajectory F the person 96 existing in the environment belongs to. However, there may be a person 96 who does not belong to any representative movement trajectory F. The determination result is appropriately stored in the memory 112 or the like. Specifically, the latest movement trajectory data (position history data) for several seconds to several tens of seconds is acquired from the position detection system 14 for all persons existing in the environment and temporarily stored in the memory 112 or the like. Then, when each of the acquired movement trajectory data of each person 96 is compared with each of the representative movement trajectories F, there is a representative movement trajectory F having a degree of similarity equal to or greater than a predetermined value with respect to the movement trajectory data of each person 96. In addition, it is determined that the person 96 belongs to the representative movement locus F. If there are a plurality of representative movement trajectories F having a similarity equal to or greater than a predetermined value, it is determined that the movement belongs to the representative movement trajectory F having the highest similarity, and the representative movement having a similarity equal to or greater than a predetermined value. If there is no trajectory F, it is determined that it does not belong to any representative movement trajectory F. For determining the similarity between the movement trajectory data of each person 96 and the representative movement trajectory F, a known method such as DP matching may be used. Note that a method for determining the similarity of trajectories using DP matching is described in Japanese Patent Application Laid-Open No. 2010-231470 filed earlier by the present applicant.

  When the distribution of each person 96 to each representative movement trajectory F is completed, the state of the person's flow is determined for each representative movement trajectory F. Specifically, for each representative movement trajectory F, feature quantities such as the total number of people, average movement speed, human density, and number of groups are extracted from the people belonging to the representative movement trajectory F. And the state of the person's flow of the representative movement locus F is discriminated by making the extracted feature quantity discriminate by the above-mentioned discriminator prepared in advance. In other words, each of the representative movement trajectories F is “a flow is about to occur (for example, after several tens of seconds)”, “a flow is about to stop (terminates)”, “a state with a flow continues”, and “a state where there is no flow” It is determined whether the state is “Continued”. By integrating these pieces of flow information determined for each representative movement trajectory F, it can be seen that a flow will soon occur at a certain place in the environment, a flow at a certain place will soon disappear. That is, it is possible to predict the occurrence, termination, and continuation of human flow in the environment.

  In the robot control system 100, information regarding the human flow in the environment predicted by the central control device 10 is used for determining (selecting) the action of the robot 12. In this embodiment, the central controller 10 also functions as a robot controller, and control data corresponding to the flow of people in the environment is transmitted from the central controller 10 to the robot 12. The robot 12 controls the operation according to control data transmitted from the central controller 10.

  For example, when the task of the robot 12 is a tourist guide, the robot 12 passes through a lively place, that is, a place where a flow situation continues, a place where a flow will soon occur, or a place where a plurality of flows are present. Be controlled. Further, for example, when the task of the robot 12 is to carry a load, the robot 12 is controlled so as to pass through a vacant place, that is, a place where a situation without a flow continues or a place where there is no soon flow. Furthermore, for example, when the task of the robot 12 is flow control (guidance arrangement), the robot 12 moves to a crowded place and speaks to a person at that place until the place is no longer flowing. It is controlled to apply. However, these are merely examples, and how the behavior of the robot 12 is controlled according to the flow of people in the environment can be appropriately determined.

  Instead of transmitting control data from the central control device 10 to the robot 12, information on the flow of people in the environment itself is sent to the robot 12, and the robot 12 (CPU 62) itself sends the flow of people in the environment. You can also decide your actions according to Further, another computer that generates control data for the robot 12 according to the flow of the person in the environment can be interposed between the central control device 10 and the robot 12 to function as a robot control device. Furthermore, the computer 110 of the central controller 10 can execute the function of the computer 90 of the position detection system 14.

  Next, the operation of the robot control system 100 (central control apparatus 10) will be described using a flowchart. Specifically, the CPU 112 of the computer 110 shown in FIG. 1 executes the entire process according to the flowchart shown in FIG.

  Referring to FIG. 11, CPU 112 determines whether or not there is an end process in step S1. For example, it is determined whether or not an operation for ending this entire process has been performed by an operator or the like. If “YES” in the step S1, that is, if there is an end process, the entire process is ended. On the other hand, if “NO” in the step S1, the process proceeds to a step S3.

  In step S3, a variable X indicating the ID of the person 96 is initialized (X = 0). For example, when the ID of the person 96 existing in the environment is set to 0 to L, X = 0 is set. In a succeeding step S5, it is determined whether X = L. That is, it is determined whether or not the trajectory determination process (the processes of steps S7 and S9) has been executed for all the people 96 present in the environment. If “YES” in the step S5, that is, if the trajectory determination process is performed for all the persons 96 existing in the environment, the process proceeds to a step S13. On the other hand, if “NO” in the step S5, that is, if there is a person 96 who has not performed the locus determination process, the process proceeds to a step S7.

  In step S7, the position history of the person 96 whose ID is X is acquired. That is, the latest position history data (movement behavior information) of the person 96 for several seconds to several tens of seconds is acquired from the position detection system 14 via the communication device 116. In subsequent step 9, the movement behavior information of the person 96 is compared with each of the representative movement trajectories F to determine which representative movement trajectory F the person 96 belongs to. That is, the similarity between the position history data of the person 96 and each representative movement trajectory F is calculated using DP matching or the like. For example, when there is a representative movement locus F having a similarity equal to or greater than a predetermined value, it is determined that the person 96 belongs to the representative movement locus F. This determination result is appropriately stored in the memory 114 or the like. In the subsequent step S11, the variable X is incremented (X = X + 1), and the process returns to step S5.

  In step S13, a variable Y indicating the ID (Fa, Fb, Fc,...) Of the representative movement locus F is initialized (Y = 0). For example, when the ID of the representative movement trajectory set in the environment is 0 to M, Y = 0 is set. In a succeeding step S15, it is determined whether or not Y = M. That is, it is determined whether or not the flow state determination process (the processes in steps S17 and S19) has been executed for all the representative movement trajectories F set in the environment. If “YES” in the step S15, that is, if the flow state determination process is performed for all the representative movement trajectories F set in the environment, the process proceeds to a step S23. On the other hand, if “NO” in the step S15, that is, if there is a representative movement locus F that has not been subjected to the flow state determination process, the process proceeds to a step S17.

  In step S17, feature amounts are extracted from people belonging to the representative movement locus F whose ID is Y. That is, feature quantities such as the total number of people belonging to the representative movement locus F, the average movement speed, the human density, and the number of groups are extracted from the people 96 determined to belong to the representative movement locus F in step S9. In a succeeding step S19, the state of the person's flow is discriminated using the discriminator. That is, the feature amount extracted in step S17 is discriminated by a discriminator created in advance, thereby discriminating the state of the person's flow on the representative movement trajectory F. For example, it is determined whether the current state is “coming soon” or “no longer going”. This determination result is appropriately stored in the memory 114 or the like.

  In step S23, control data corresponding to the flow of people in the environment is transmitted to the robot 12. For example, when the task set for the robot 12 is sightseeing guidance, control data for controlling the robot 12 so as to pass through a place where a flow will soon occur is created, and the robot 12 is communicated via the communication device 116. Send.

  According to this embodiment, by determining the flow state of the representative movement trajectory F set in the environment in advance, a flow will soon occur in a certain place in the environment, or a flow in a certain place will soon disappear. Predictable. That is, it is possible to predict the occurrence, termination, and continuation of human flow in the environment.

  Moreover, since the flow of the person in the environment can be predicted, the robot 12 can execute a natural action according to the flow of the person.

  In the above-described embodiment, the state of the person's flow in the representative movement locus F is determined from the feature amount of the people belonging to the representative movement locus F. That is, the representative movement locus is determined from the feature amount of the representative movement locus Fa. Although the state of the flow of Fa is determined, the present invention is not limited to this. For example, by calculating the correlation between representative movement trajectories F in advance, it is also possible to determine the state of a person's flow in another representative movement trajectory F from the feature amount of people belonging to a certain representative movement trajectory F. it can. Further, for example, it is possible to determine the state of the flow of people in the area from the feature amount of people belonging to the representative movement locus group (collection of representative movement locus F) included in a certain area.

  Further, in the above-described embodiment, as an example of the robot 12, an interaction-oriented robot that executes a communication action with a person using body movement or speech has been described, but the type of the robot 12 is particularly limited. Not.

  Further, in the above-described embodiment, the behavior of the robot 12 is controlled based on the predicted information about the flow of the person. However, the present invention is not limited to this, and is used for controlling other devices such as digital signage and traffic lights. You can also give instructions to security guards and other people. For example, when controlling a digital signage, a method of using an advertisement to be presented only when the field of view of the digital signage is in a state where there is a flow or is about to flow can be considered.

  Note that the specific numerical values such as the time and the number of times mentioned above are merely examples, and can be changed as appropriate.

10 ... Central control device (flow state discrimination device)
12 ... Robot 14 ... Position detection system 62 ... Robot CPU
DESCRIPTION OF SYMBOLS 66 ... Robot memory 86 ... Robot communication apparatus 90 ... Position detection system computer 92 ... Laser range finder 94 ... Wireless ID tag reader 100 ... Robot control system 110 ... Central controller computer 112 ... Central controller CPU
114 ... central control unit memory 116 ... central control unit communication unit 118 ... flow database F ... representative movement trajectory

Claims (5)

A flow state determination device that predicts the flow of people in the environment,
Storage means for storing information on one or more representative movement trajectories created in advance based on movement behavior information of a plurality of persons accumulated in the environment;
Detecting means for detecting position histories of a plurality of persons existing in the environment;
Trajectory determination means for determining which of the representative movement trajectories each of a plurality of persons existing in the environment based on the position history detected by the detection means,
For each of the representative movement trajectories, the representative movement is extracted based on the extraction means for extracting a feature amount from a person determined to belong to the representative movement locus by the determination means, and the feature amount extracted by the extraction means. A flow state discriminating device comprising state discriminating means for discriminating a human flow state for each of the trajectories.   The flow discrimination device according to claim 1, wherein the state discrimination means includes a discriminator using a machine learning method created using teacher data to which a label indicating the flow state of the person is attached. A flow state determination method for predicting the flow of people in an environment,
(A) Accumulating movement behavior information of a plurality of people in the environment;
(B) Based on the movement behavior information accumulated in the step (a), a representative movement route through which a person in the environment passes is set as one or a plurality of representative movement trajectories,
(C) For each of the representative movement trajectories set in step (b), create a discriminator for discriminating the state of human flow,
(D) detecting position histories of a plurality of people existing in the environment;
(E) Based on the position history of the plurality of people detected in the step (d), it is determined which of the representative movement trajectories each of the plurality of people belongs to,
(F) For each of the representative movement trajectories, a feature amount is extracted from the person determined to belong to the representative movement trajectory in step (e), and (g) for each of the representative movement trajectories, the step ( A flow state determination method for determining a state of a human flow by determining the feature amount extracted in step (f) by the classifier created in c). A processor for a flow state discriminating apparatus having storage means for storing information on one or more representative movement trajectories created in advance based on movement behavior information of a plurality of persons accumulated in the environment,
Detecting means for detecting position histories of a plurality of persons existing in the environment;
Trajectory determination means for determining which of the representative movement trajectories each of a plurality of persons existing in the environment based on the position history detected by the detection means,
For each of the representative movement trajectories, the representative movement is extracted based on the extraction means for extracting a feature amount from a person determined to belong to the representative movement locus by the determination means, and the feature amount extracted by the extraction means. A flow state determination program that functions as a state determination unit that determines the state of a person's flow for each trajectory. A robot control system including a robot that provides services in an environment where people coexist,
Storage means for storing information on one or more representative movement trajectories created in advance based on movement behavior information of a plurality of persons accumulated in the environment;
Detecting means for detecting position histories of a plurality of persons existing in the environment;
Trajectory determination means for determining which of the representative movement trajectories each of a plurality of persons existing in the environment based on the position history detected by the detection means,
For each of the representative movement trajectories, an extraction means for extracting a feature amount from a person determined to belong to the representative movement trajectory by the determination means;
Based on the feature amount extracted by the extraction means, state determination means for determining the state of human flow for each of the representative movement trajectories, and based on the result determined by the state determination means, A robot control system comprising a control means for controlling operation.

Images