JP2008225852A - Population state estimation system, population caution control system, and active sensing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a new population state estimation system which can precisely estimate human being's population state with respect to an object. <P>SOLUTION: In a population state estimation system (100), a computer (14) uses an output (reaction spot data) from a floor sensor (18) and estimates a population state of human being near the object like a communication robot (10). The reaction spot is clustered and a population is divided into a plurality of clusters. Each cluster is classified into a "lump cluster" or a "stray cluster". According to a positional relation to the objects of the "lump cluster" or the "stray cluster", it is estimated whether the population state at that time corresponds to a population state 1 or a population state 12. Since the population state around the object can be estimated with sufficient accuracy, the population caution control and active sensing can be performed exactly and efficiently. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a collective state estimation system, a collective attention control system, and an active sensing system, and more particularly to collective attention by estimating a collective state of a human that exists in the same closed space as an object (object) such as a communication robot or an exhibit. The present invention relates to a collective state estimation system, a collective attention control system using the same, and an active sensing system, which are applied to control and active sensing.

  For example, Non-Patent Document 1 discloses a robot that constructs an appropriate model according to the distance between a robot and a human and simultaneously interacts with a plurality of humans. However, the system of Non-Patent Document 1 is not compatible with a dialogue between a group of many humans and a robot.

On the other hand, Non-Patent Document 2 discloses a technique by which the inventors can smoothly interact with a human group and a robot.
Tasaki, T., Matsumoto S., Yamamoto S., Toda M., Komatani, K., Ogata T., and Okuno, HG “Dynamic Communication of Humanoido Robot with Multiple Peaple Based on Interaction Distance,” Journal of the Japanese Society for Artificial Intelligence , Vol. 20, No.3, pp.209-219, 2005 Masahiro Shiomi, Satoshi Koizumi, Muneyuki Kanda, Hiroshi Ishiguro, Norihiro Hirota “Communication Robotic Control of Communication” Human Interface Symposium 2006, CD-ROM 2006

  Although it was verified that the technique disclosed in Non-Patent Document 2 is effective in the interaction between the robot and people, it was not an autonomous system.

  Therefore, the main object of the present invention is to provide a novel collective state estimation system.

  Another object of the present invention is to provide a collective state estimation system capable of finely estimating a human collective state for an object.

  The present invention employs the following configuration in order to solve the above problems. The reference numerals in parentheses, supplementary explanations, and the like indicate correspondence relationships with embodiments described later to help understanding of the present invention, and do not limit the present invention in any way.

  A first invention is a collective state estimation system for recognizing a collective state of a plurality of humans existing in the same closed space as an object, and detects position of each of the plurality of humans to acquire position data Clustering means for performing clustering according to position data and partitioning into a plurality of clusters, classification means for classifying each cluster into a first cluster and a second cluster according to the number of persons included in each of the plurality of clusters, and a first A collective state recognition system comprising collective state estimation means for estimating collective states based on the presence of clusters and second clusters and the positional relationship between objects and outputting collective state data.

  In the first invention, the collective state estimation system (100) uses, for example, a floor (pressure) sensor (18) and a computer (14) to acquire human position data. In a clustering means (S7, S35) such as a computer (14), all reaction points (human positions) are first merged into one cluster by, for example, a hierarchical clustering technique, and then the reaction point distribution state (for example, The appropriate number of clusters is determined according to the solid distance or the public distance), and one cluster is divided (partitioned) into a plurality of clusters. Similarly, each cluster is classified into a “cluster cluster” (first cluster) or a “staggered cluster” (second cluster) by classification means (S7, S37, S61-S83) that can be configured by the computer (14). The collective state estimation means (S7, S91-S105), which may be a computer (14), is the presence of “cluster clusters” and “stray clusters” and the positional relationship between them and objects, in the embodiment, the robot (10). Based on (for example, near or far, whether the distance is the same or not), it is estimated whether the collective state is, for example, 12 collective states, “collective state 1” − “collective state 12”.

  According to the first invention, after clustering, the collective state is estimated based on the cluster data, and therefore the collective state can be estimated accurately.

  For example, the first cluster and the second cluster can be classified by the number of elements constituting the cluster (the number of people. In the embodiment, the number of reaction points).

  A second invention is dependent on the first invention, and the collective state estimating means estimates the collective state based on the number of the first clusters and the second clusters and the distance between them and the object. System.

  In the second invention, for example, the collective state is estimated based on whether there is one “cluster cluster”, whether there is a “stray cluster”, whether the “stray cluster” is far from or near the object, and the like. The population status can be estimated in detail.

  A third aspect of the invention is a collective state recognition system for recognizing a collective state of a plurality of humans existing in the same closed space as an object, and detecting position of each of the plurality of humans to acquire position data , Clustering means for clustering according to the position data and partitioning into a plurality of clusters, and determination means for determining whether the collective state is in an orderly state or a cluttered state based on the feature quantities of the plurality of clusters It is a collective state estimation system.

  In the third invention, the collective state estimation system (100) uses, for example, a floor (pressure) sensor (18) and a computer (14) to acquire human position data. In a clustering means (S7, S35) such as a computer (14), all reaction points (human positions) are first merged into one cluster by, for example, a hierarchical clustering technique, and then the reaction point distribution state (for example, The appropriate number of clusters is determined according to the solid distance or the public distance), and one cluster is divided (partitioned) into a plurality of clusters. In the discrimination means (S191-S193) that can also be configured by the computer (14), the feature amount of each cluster, for example, the number of humans, the number of clusters, the average of the distance between the reaction point and the object constituting the cluster, the standard deviation thereof, Based on the angle between the center of gravity of the cluster and the front of the object, it is determined whether the group state at that time is an “orderly” state or a “cluttered” state.

  In the third invention, by using the feature amount of the cluster, it is possible to accurately discriminate even if it is an extremely subjective evaluation criterion of “orderly” or “cluttered”.

  A fourth invention is a group comprising group attention control means for estimating a group state using the group state estimation system according to any one of the first to third inventions and executing group attention control according to the group state. Attention control system.

  In the fourth invention, since the presumption of the collective state is appropriate, proper collective attention control suitable for the collective state at that time can be performed.

  A fifth invention is according to the fourth invention, wherein the group attention control means includes a communication robot, and the communication robot receives the estimation result of the group state and executes group attention control adapted to the group state. Attention control system.

  In the fifth invention, since the communication robot is used as the group attention control means, the communication robot performs the group attention control by voice and gesture output by itself.

  A sixth aspect of the present invention is an active sensing system comprising active sensing means for estimating a collective state using the collective state estimation system according to any one of the first to third aspects and executing active sensing according to the collective state. It is.

  In the sixth invention, since the presumption of the collective state is appropriate, appropriate active sensing suitable for the collective state at that time can be performed.

  An seventh aspect of the present invention is an active sensing system according to the sixth aspect of the present invention, wherein the active sensing means includes a communication robot, and the communication robot receives the estimation result of the collective state and executes active sensing adapted to the collective state. It is.

  In the seventh invention, since the communication robot is used as the active sensing means, the communication robot can efficiently perform active sensing using its own camera or the like.

  According to the present invention, the collective state around an object such as a robot can be estimated with high accuracy. Therefore, when applied to collective attention control, collective attention control can be performed more accurately. Moreover, if applied to active sensing, sensing can be performed efficiently according to the collective state.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  Referring to FIG. 1, a collective state estimation system (hereinafter simply referred to as “system”) 100 of this embodiment includes a communication robot (hereinafter simply referred to as “robot”) 10, a display device 12, a computer 14, and a database. 16 and a floor pressure sensor (floor sensor) 18. The system 100 is a system in which the robot 10 performs a group attention control action in accordance with a human group state transferred from the computer 14, for example.

However, after the computer 14 estimates the collective state, in addition to performing the collective attention control as described above, it is also effective for active sensing that efficiently performs sensing by changing the sensing area according to the collective state. Can be applied to. That is, it should be noted in advance that the collective state data obtained by the computer 14 executing the collective state estimation has various application examples.

  For example, the robot 10 is connected to a computer 14 so as to be communicable by, for example, short-range radio represented by a wireless LAN or Bluetooth (trade name). The computer 14 includes a display device 12, a database 16, and a floor sensor. 18 are connected by wire. However, the robot 10 and the computer 14 may be connected by wire.

  The robot 10 is interaction-oriented mainly for the purpose of performing communication with a communication target such as a human, and will be described in detail later with reference to FIGS. 3 and 4. It has a function to execute communication (group attention control, etc.) using the body movements.

  The display device 12 is, for example, a CRT display or a liquid crystal display. As described above, the computer 14 is connected and display control is performed by the computer 14. However, information necessary for group attention control can be visually displayed on the display device 12 according to an instruction from the robot 10.

  The computer 14 is a computer such as a general-purpose personal computer (PC) or a workstation, and basically estimates a collective state according to a sensor value (“1” or “0”) from the floor sensor 18. Status data is provided to the robot 10 or the like. In addition, the computer 14 may control the display of the display device 12 or the communication behavior of the robot 10 as described above.

  As is well known, the computer 14 is composed of, for example, a ROM, a hard disk, and the like, and presets a program for group state estimation, control data, image data, and the like as will be described with reference to a flowchart described later. Program memory 15a. The computer 14 further includes, for example, a data memory 15b that includes a RAM, a flash memory, and the like and is used as a working memory or a buffer memory. The data memory 15b sets a temporary storage area for temporarily storing sensor values from a floor sensor 18 (to be described later) and coordinate data of points (positions) at which the sensors are reacting, and various flags and various registers. It is allocated to the flag register area etc.

  The database 16 stores, for example, map data necessary for route guidance as an interpersonal service, explanation data necessary for explaining the exhibition, and the like, and a control command for controlling the operation of the robot 10. The command name is stored. This command name is the name of the behavior module of the robot 10. When the computer 14 controls the robot 10, the computer 14 transmits a control command (action module name) of the robot 10 to the robot 10. The robot 10 executes a communication action (speech, gesture, etc.) in accordance with a control command from the computer 14. Further, map data (two-dimensional data) such as a venue to which the system 100 is applied is set in the database 16. However, the database 16 may be omitted and the computer 14 and / or the robot 10 may store the above-described map data, control commands, map data, and the like.

  The floor sensor (floor pressure sensor) 18 includes a set of a large number of detection elements (pressure-sensitive sensors: for example, switches that are turned on / off by weighting), for example, a square of 500 × 500 mm, a thickness of about 15 mm, and a detection resolution. Is about 100 × 100 mm as an example. For example, RS232C (standard name) is used for the interface of the floor sensor 18, and the computer 14 can acquire sensor outputs at intervals of 200 milliseconds, for example. In the embodiment, as shown in FIG. 2, a large number of floor sensors 18 are embedded (laid) on the floor of a closed space (room) 101 to which the system 100 is applied, and a robot 10 or a human that exists in the same closed space. Is detected. In the embodiment, n floor sensors 18 (S1, S2,..., Sh, Sh + 1,..., Si, Si + 1,..., Sn-1, Sn) are installed on the floor of the room 101. Each of the floor sensors 18 (S1,..., Sn) has a sensor value of high level or “1” when the weight is felt, and a sensor value of low level or “0” when the weight is not felt. Output each.

  FIG. 3 is a front view showing the appearance of the robot 10. Referring to FIG. 3, the robot 10 includes a carriage 20, and two wheels 22 and one wheel for moving the robot 10 autonomously are provided on the lower surface of the carriage 20. A follower wheel 24 is provided. The two wheels 22 are independently driven by a wheel motor 26 (see FIG. 4), and the carriage 20, that is, the robot 10 can be moved in any direction, front, back, left, and right. The slave wheel 24 is an auxiliary wheel that assists the wheel 22. Therefore, the robot 10 can freely move in the arranged space.

  A cylindrical sensor mounting panel 28 is provided on the carriage 20, and an infrared distance sensor 30 is mounted on the sensor mounting panel 28. The infrared distance sensor 30 measures the distance from the sensor mounting panel 28, that is, the object (human or obstacle) around the robot 10.

  Further, the body 32 is provided on the sensor mounting panel 28 so as to stand upright. An infrared sensor 31 different from the above-described infrared distance sensor 30 is further provided at the front upper center of the body 32 (a position corresponding to the chest). This measures the distance to the human mainly in front of the robot 10. One omnidirectional camera 34 is provided. The omnidirectional camera 34 is provided, for example, on a column 36 that extends upward from substantially the center of the upper end on the back side of the body 32. The omnidirectional camera 34 captures the surroundings of the robot 10 and is distinguished from an eye camera 60 described later. As this omnidirectional camera 34, for example, a camera using a solid-state imaging device such as a CCD or a CMOS can be adopted. In addition, the installation positions of the infrared distance sensor 30 and the omnidirectional camera 34 are not limited to the portions, and can be changed as appropriate.

  Upper arms 40R and 40L are provided at upper end portions (positions corresponding to shoulders) on both side surfaces of the body 32 by shoulder joints 38R and 38L, respectively. Although illustration is omitted, the shoulder joints 38R and 38L each have three orthogonal degrees of freedom. That is, the shoulder joint 38R can control the angle of the upper arm 40R around each of the three orthogonal axes. A certain axis (yaw axis) of the shoulder joint 38R is an axis parallel to the longitudinal direction (or axis) of the upper arm 40R, and the other two axes (pitch axis and roll axis) are axes orthogonal to each other from different directions. It is. Similarly, the shoulder joint 38L can control the angle of the upper arm 40L around each of the three orthogonal axes. The yaw axis of the shoulder joint 38L is an axis parallel to the longitudinal direction (or axis) of the upper arm 40L, and the other pitch axis and roll axis are axes orthogonal to each other from different directions.

  Further, forearms 44R and 44L are provided at the tips of the upper arms 40R and 40L via elbow joints 42R and 42L, respectively. Although illustration is omitted, each of the elbow joints 42R and 42L has one degree of freedom, and the angle of the forearms 44R and 44L can be controlled around the axis (pitch axis).

  Spheres 46R and 46L corresponding to hands are fixedly provided at the tips of the forearms 44R and 44L, respectively. However, when a finger or palm function is required, a “hand” in the shape of a human hand can be used.

  Although not shown, the front surface of the carriage 20, the parts corresponding to the shoulders including the shoulder joints 38R and 38L, the upper arms 40R and 40L, the forearms 44R and 44L, and the spheres 46R and 46L are each provided with a contact sensor (FIG. 4). 48) is provided. The contact sensor 48 on the front surface of the carriage 20 detects contact of a person or another obstacle with the carriage 20. Therefore, if there is contact with an obstacle while the robot 10 is moving, it can be detected, and the driving of the wheel 22 can be immediately stopped to suddenly stop the movement of the robot 10. The other contact sensors 48 mainly detect whether or not a human has touched each part of the robot 10. The installation position of the contact sensor 48 is not limited to these, and may be provided at an appropriate position (chest, abdomen, side, back, waist, head, etc.).

  A neck joint 50 is provided at the upper center of the body 32 (a position corresponding to the neck), and a head 52 is further provided thereon. Although illustration is omitted, the neck joint 50 has a degree of freedom of three axes, and the angle can be controlled around each of the three axes. A certain axis (yaw axis) is an axis directed directly above (vertically upward) of the robot 10, and the other two axes (pitch axis and roll axis) are axes orthogonal to each other in different directions.

  The head 52 is provided with a speaker 54 at a position corresponding to the mouth. The speaker 54 is used for the robot 10 to communicate or perform group attention control with a voice or sound with respect to the people around it. Further, microphones 56R and 56L are provided at positions corresponding to the ears. Hereinafter, the microphone 56R corresponding to the right ear and the microphone 56L corresponding to the left ear may be collectively referred to as the microphone 56. The microphone 56 captures ambient sounds, particularly a human voice that is an object for performing communication. Further, eyeball portions 58R and 58L are provided at positions corresponding to the eyes. Eyeball portions 58R and 58L include eye cameras 60R and 60L, respectively. Hereinafter, the right eyeball portion 58R and the left eyeball portion 58L may be collectively referred to as the eyeball portion 58, and the right eye camera 60R and the left eye camera 60L may be collectively referred to as the eye camera 60.

  The eye camera 60 captures a human face approaching the robot 10, other parts or objects, and captures a corresponding video signal. As the eye camera 60, a camera similar to the omnidirectional camera 34 described above can be used. For example, the eye camera 60 is fixed in the eyeball part 58, and the eyeball part 58 is attached to a predetermined position in the head 52 via an eyeball support part (not shown). Although illustration is omitted, the eyeball support portion has two degrees of freedom, and the angle can be controlled around each of these axes. For example, one of the two axes is an axis (yaw axis) in a direction toward the top of the head 52, and the other is orthogonal to one axis and the direction in which the front side (face) of the head 52 faces. It is an axis (pitch axis) in the direction to be. By rotating the eyeball support portion around each of these two axes, the tip (front) side of the eyeball portion 58 or the eye camera 60 is displaced, and the camera axis, that is, the line-of-sight direction is moved.

  The installation positions of the speaker 54, the microphone 56, and the eye camera 60 described above are not limited to these, and may be provided at appropriate positions.

  FIG. 4 is a block diagram showing an electrical configuration of the robot 10. With reference to FIG. 4, the robot 10 includes a processor 62 for controlling the whole. The processor 62 is connected to the memory 66, the motor control board 68, the sensor input / output board 70, and the audio input / output board 72 via the bus 64.

  The memory 66 includes a program memory 66a composed of, for example, a ROM or a hard disk. The program memory 66a includes a control program for the robot 10 (an action control program, a group attention control program for executing communication with humans, Active sensing program, etc.), and angles for presenting voice or voice data (speech synthesis data) and predetermined gestures to be generated from the speaker 54 when performing communication or group attention control operations Data etc. are also stored. In the program memory 66a, a communication program for transmitting / receiving necessary information to / from an external computer (such as the computer 14) is recorded. Memory 66 further includes, for example, a RAM or a flash memory, and includes a data memory 66b used as a working memory or a buffer memory. The data memory 66b sets a temporary storage area for temporarily storing group state data and the like sent from the computer 14 as will be described later, and also sets a caution control flag A and various registers described later. It is allocated to the flag register area etc.

  The memory 66 may be a storage device (internal memory) built in or incorporated in the processor 62, or may be a storage device provided separately from the internal memory of the processor 62.

  The motor control board 68 is composed of, for example, a DSP, and controls the driving of each axis motor such as each arm, neck joint, and eyeball. That is, the motor control board 68 receives the control data from the processor 62, and controls two angles of the two axes of the right eyeball part 58R (in FIG. 4, they are collectively referred to as “right eyeball motor”). The rotation angle of 74 is controlled. Similarly, the motor control board 68 receives the control data from the processor 62, and shows two motors for controlling the respective angles of the two axes of the left eyeball portion 58L (in FIG. 4, collectively referred to as “left eyeball motor”). ) The rotation angle of 76 is controlled.

  The motor control board 68 receives control data from the processor 62, three motors for controlling the angles of the three orthogonal axes of the right shoulder joint 38R, and one motor for controlling the angle of the right elbow joint 42R. The rotation angles of the four motors (collectively referred to as “right arm motor” in FIG. 4) 78 are adjusted. Similarly, the motor control board 68 receives control data from the processor 62, three motors for controlling the angles of the three orthogonal axes of the left shoulder joint 38L, and one motor for controlling the angle of the left elbow joint 42L. The rotation angles of the four motors (collectively referred to as “left arm motor” in FIG. 4) 80 are adjusted.

  Further, the motor control board 68 receives the control data from the processor 62 and controls three motors for controlling the respective angles of the three orthogonal axes of the neck indirect 50 (referred to collectively as “head motors” in FIG. 4). ) 82 is controlled. Furthermore, the motor control board 68 receives the control data from the processor 62 and controls the rotation angle of two motors (collectively referred to as “wheel motors” in FIG. 4) 26 that drive the wheels 22.

  In this embodiment, stepping motors or pulse motors are used for the motors other than the wheel motor 26 in order to simplify the control. However, as with the wheel motor 26, a DC motor may be used.

  Similarly, the sensor input / output board 70 is also configured by a DSP, and takes in signals from each sensor and supplies them to the processor 62. That is, data relating to the reflection time from each of the infrared distance sensors 30 is input to the processor 62 through the sensor input / output board 70. The video signal from the omnidirectional camera 34 is input to the processor 62 after being subjected to predetermined processing by the sensor input / output board 70 as required. The video signal from the eye camera 60 is also input to the processor 62 in the same manner. In addition, signals from the plurality of contact sensors described above (collectively indicated as “contact sensors 48” in FIG. 4) are provided to the processor 62 via the sensor input / output board 70.

  Similarly, the voice input / output board 72 is also configured by a DSP, and voice or voice in accordance with voice synthesis data provided from the processor 62 is output from the speaker 54. For example, information is emitted from the speaker 54 as voice or voice when performing route guidance or group attention control. Also, the voice input from the microphone 56 is taken into the processor 62 via the voice input / output board 72.

  The processor 62 is connected to the communication LAN board 84 via the bus 64. The communication LAN board 84 is configured by a DSP, gives transmission data sent from the processor 62 to the wireless communication device 86, and sends the transmission data from the wireless communication device 86 to an external computer (for example, via a network such as a wireless LAN). In FIG. 1 and FIG. 4, it is transmitted to a computer 14) or the like. The communication LAN board 84 receives data via the wireless communication device 86 and gives the received data to the processor 62. That is, the communication LAN board 84 and the wireless communication device 86 allow the robot 10 to perform wireless communication with the computer 14 and the like.

  With reference to FIGS. 5 to 13, a method in which the computer 14 estimates the collective state and the operation at that time will be described.

  If the stop command by the operator is not detected in step S1 in FIG. 5, the computer 14 uses the RS232C or the like in the next step S3, for example, using a polling technique as shown in FIG. The sensor information, that is, the sensor number (n), the sensor value (Si = 1 or 0), and the time when the sensor value is acquired are read from all the floor sensors 18 arranged in the inside. This sensor information is temporarily stored in the data memory 15b shown in FIG.

  In subsequent step S5, the computer 14 converts the sensor information acquired in step S3 into xy coordinates. That is, the position coordinates of the reaction point of the floor sensor that detects the robot 10 or a human is calculated. However, since the robot 10 weights the floor sensor with two wheels 22, the reaction point of the robot 10 is calculated as the midpoint of the two wheels, and on the other hand, it is assumed that a human is also standing on two legs. Therefore, the midpoint of the two feet is calculated as the reaction point of the floor sensor by the human. In order to convert such reaction points into xy coordinates, specifically, the procedure shown in FIG. 6 is executed.

If a stop command by the operator is not detected in the first step S21 in FIG. 6, the computer 14 determines whether sensor information has been received from each floor sensor in the subsequent step S23, and if “NO”, waits until it is received. . If the sensor information has been received, in the next step S25, the computer 14 uses the equation (1) to calculate the coordinates of the point at which the floor sensor is responding.
[Equation 1]
xj = i% h
yj = i / h (Si = 1, j = 1, 2,…, m)… (1)
However, m is the total number of points to which the floor sensor is responding, and the floor sensor is arranged as shown in FIG.

  Then, in the next step S27, the computer 14 obtains the coordinates (x1, y1), (x2, y2),..., (Xm, ym) of the points to which the floor sensor reacts, obtained by calculation in step S25. Is transmitted to the next step S7 in FIG. That is, these coordinate data are temporarily stored in the data memory 15b shown in FIG. This step S7 is a collective state estimation program and is shown in detail in FIGS.

  If the stop command by the operator is not detected in the first step S31 in FIG. 7, the computer 14 coordinates (x1, y1), (x2, y2),... , (Xm, ym) is determined, and if “NO”, it waits until it is received. If “YES”, in the next step S35, the computer 14 performs clustering based on the received coordinate data, and divides or divides the reaction point (of the floor sensor) into several clusters.

  In step S35, clustering is performed using the Euclidean distance between the reaction points as the similarity (the similarity is higher as the distance is closer). In the embodiment, a so-called hierarchical clustering technique is employed. In hierarchical clustering, when m reaction point data are given, first, m clusters including only one data (element) are set as an initial state. From this state, the distance between one cluster and another cluster (intercluster distance) is calculated, and the two clusters with the closest distance are merged sequentially, and all reaction points are merged into one cluster. This is a technique of acquiring a hierarchical structure between clusters by repeatedly processing up to. Hereinafter, this will be described in detail with reference to FIG.

  If no stop command is detected by the operator in the first step S41 in FIG. 8, the computer 14 sets m reaction points to m clusters each having only one element in the subsequent step S43. Subsequently, in step S45, the computer 14 calculates an intercluster distance (FIG. 9) of m clusters.

  In the embodiment, the distance between clusters is calculated using the shortest distance method, although it is not particularly limited. The shortest distance method is a method in which the distance between the nearest elements among the elements included in two clusters is defined as a cluster distance.

  There are several other algorithms for calculating the cluster distance, such as the centroid method and the Ward method. However, the shortest distance method uses the distance between the nearest elements as the intercluster distance. Can make chain clusters. On the other hand, the center of gravity method and the Ward method mainly create ellipsoidal clusters. When the robot 10 interacts with a human as in the embodiment, when performing clustering on such a human group, a chain cluster is more suitable than an ellipsoidal cluster. Conceivable. This is because when one person is lined up horizontally with respect to the robot, that one line can be handled as one cluster.

  In the next step S47, the computer 14 merges the clusters having the shortest inter-cluster distance. This step S47 is repeatedly executed together with the previous step S45 until it is determined in step S49 that all reaction points (elements) have been merged into one cluster.

  When it is determined in step S49 that all reaction points have been merged into one cluster, the computer 14 refers to the coordinate data of the reaction points stored in the data memory 15b (FIG. 1) in the next step S51. The appropriate number of clusters C is determined according to the distribution state of each reaction point. The number C of clusters is based on the knowledge of the inventors regarding the interpersonal distance, and the reaction points whose reaction points are 70 cm (solid distance) or less always belong to the same cluster and are separated by 110 cm (official distance) or more. Reaction points are determined to belong to different clusters.

  Furthermore, in the embodiment, in addition to such a distance condition, the number of clusters is determined based on the pseudo t2 statistic represented by the equation (2). The pseudo t2 statistic is a statistic used as an index for determining the number of clusters, as is well known.

  The pseudo t2 statistic when cluster P and cluster Q are combined to form cluster R is given by equation (2). Here, W is the sum of the distances between the center of gravity of each cluster and all elements (reaction points) included in the cluster, and N is the number of elements in each cluster. The number of clusters for which the value of the pseudo t2 statistic greatly changes is a candidate for the appropriate number of clusters C.

  That is, in this embodiment, the number of clusters C is determined under the following conditions based on the interpersonal distance and the pseudo t2 statistic.

  1. The number of clusters whose intercluster distance is equal to or greater than the solid distance (70 cm) is determined as a candidate number of clusters to be divided. If there is no candidate, the number of clusters is 1.

  2.1. The number of clusters in which the inter-cluster distance is equal to or less than the public distance (110 cm) is selected as the candidate number of clusters to be divided. If there is no candidate, the number of clusters with the largest inter-cluster distance is taken as the division number.

  3. Among the selected candidates, the number of clusters in which the amount of change in the pseudo t2 statistic changes the largest in the positive direction is set as a candidate for the number of divided clusters.

  After determining the appropriate number of clusters C in this way at step S51, the computer 14 at step S53, the number of clusters C and the number Pi of reaction points (elements) included in each cluster (i = 1,...,. C. However, P1 ≦ P2 ≦... ≦ Pc) is transmitted to the step to be executed next to the state estimation program (FIG. 7: step S37). That is, the data memory 15b shown in FIG. 1 stores the number C of clusters, the number Pi of reaction points included in each cluster, and the cluster barycentric position as cluster data as necessary.

  Returning to FIG. 7, in step S37, based on the number C of clusters calculated or determined in this way and the number Pi of elements included in each cluster, whether each cluster is a “cluster cluster” is determined. Is determined. However, the “cluster” refers to the first cluster, and is a cluster composed of a larger number of reaction points (elements) than the “staggered cluster” that means the second cluster. In other words, a “staggered cluster” can be defined as a cluster composed of a smaller number of elements than a “cluster of clusters”. Specifically, such a cluster determination step is executed according to the procedure shown in FIG.

If a stop command by the operator is not detected in the first step S61 in FIG. 10, the computer 14 determines in the subsequent step S63 whether or not the above-described clustering result, that is, cluster data has been received. Wait until. If “YES”, it is determined in the next step S65 whether or not the cluster having the largest number of reaction points (elements) is equal to or more than a% of all reaction points based on the equation (3). However, all judgment formulas including this judgment formula are such that when the cluster having the largest number of elements is a% or more of the total number of elements, the cluster having the number of elements of b% or less of the total number of elements is defined as the “stray cluster”. It should be understood in advance that the result is a candidate and is considered so that the number of elements of all “staggered clusters” is equal to or less than c% of the number of elements of all “cluster clusters”.
[Equation 3]
Pc ≧ m (a / 100) (3)
When the determination result in step S65 is “NO”, since the cluster having the largest number of elements does not exceed a% of the total number of elements, the computer 14 determines in step S67 that all the clusters are “cluster clusters”. To do.

Next, when “YES” is determined in step S65, since the cluster having the largest number of elements is a% or more of the total number of elements, the computer 14 performs determination based on the expression (4) in the subsequent step S69. .
[Equation 4]
P1 ≦ m (b / 100) (4)
When “NO” is determined in step S69, the number of elements is not less than b% of the total number of elements. Therefore, in step S71, the computer determines that all clusters are “cluster clusters”.

If "YES" is determined in the step S69, the computer 14 sets the maximum i satisfying the expression (5) to H (separation) in the next step S73, and the minimum i satisfying the expression (6) in the subsequent step S75. The determination based on the determination formula (7) in step S77 is executed as K (a mass). However, C is the number of clusters.
[Equation 5]
Pi ≦ m (b / 100) (i = 1, 2,..., C) (5)
[Equation 6]
Pi ≧ m (b / 100) (i = 1, 2,..., C) (6)
[Equation 7]
P _H ≦ P _K (c / 100) (7)
When the computer 14 determines “NO” in step S77, the number of elements of “stray cluster” is not less than c% of the number of elements of all “cluster clusters”. It is determined that the cluster is a “cluster”. Further, when “YES” is determined in step S77, the number of elements of “stray cluster” is equal to or less than c% of the number of elements of all “cluster clusters”. ) A cluster that satisfies the expression is determined as a “stray cluster”, and the other clusters are determined as a “cluster cluster”.

  In this way, in step S67, S71, S79 and S81, the type of each cluster is determined as to whether it is a “cluster cluster” or “stray cluster”, and the computer 14 determines the result of the determination (for each cluster). Type) is transmitted as cluster data to the collective state estimation program in step S83, ie, step S39 in FIG. That is, it is stored in the data memory 15b of FIG.

  The collective state estimation program executes a collective state estimation operation (step S7: FIG. 5) based on this determination result (cluster type). The group state estimation program is specifically shown in FIG. 11 in detail, but in this embodiment, basically, the three cases as shown in FIG. 13 are first classified and finally [group state 1] -Estimate by classifying into [population state 12].

  In the first step S91 in FIG. 11, the computer 14 refers to each cluster type of the cluster data stored in the data memory 15b and determines whether there is only one “cluster cluster”. If “YES”, it is one of the collective states included in the section “one cluster cluster” shown in FIG. 13, and in the next step S93, the computer 14 determines whether there is a “stray cluster”. If “NO” is determined in step S93, since there is one “cluster cluster” and no “stray cluster”, for example, a so-called collective group state, the computer 14 determines such a collective state as “group”. State 1 ”is defined, and the collective state at that time is estimated as collective state 1.

  If “YES” is determined in the step S93, since “YES” is determined in the previous step S91, the robot 101 as an object and one “cluster cluster” are present in the room 101 (FIG. 2). It means that there is one or more “staggered clusters” and it is one of the collective states 2, 3 and 4 shown in FIG. Therefore, in step S95, the computer 14 refers to the cluster data, reaction point data, etc., and the positional relationship between the “staggered cluster”, “cluster” and the object (robot 10), in particular, “stray cluster” is an object, that is, a robot. For 10, it is specified which group state is based on whether it is closer to the “cluster cluster”.

  FIG. 12 shows details of step S95 for determining the status of “stray cluster”. Referring to FIG. 12, in step S111, the computer 14 determines that the “stray cluster” is more than the “cluster cluster” than the robot 10 ( Judge whether it is near (object). If “YES”, the “stray cluster” is labeled “+ near stray”. If “NO” is determined in the step S11, the computer 14 then determines whether or not the “stray cluster” is farther from the object than the “cluster” in a step S113. If “YES”, the “stray cluster” is labeled “+ far away”, and “NO” is labeled “+ stray” in the sense that it is neither near nor far.

  In this way, in FIG. 12, when the reaction point closest to the object belongs to the “stray cluster”, it is determined as “+ close stray”, and the reaction point farthest from the object belongs to the “stray cluster”. It is judged as “+ far away”, otherwise it is recognized as “+ far away”.

  As can be seen from FIG. 13, when there is only one “cluster cluster” and one “stray cluster” labeled “+ stray”, the collective state at that time is “collective state 2”. It is. When there is only one “clump cluster” and one “stray cluster” labeled “+ far away”, the collective state at that time is “collective state 3”. When there is only one “cluster cluster” and one “stray cluster” labeled “+ stray”, the collective state at that time is “collective state 4”. In this way, when “YES” is determined in step S91, that is, when only one “cluster cluster” exists, the computer 14 estimates the collective states 1, 2, 3, or 4.

If “NO” is determined in step S91, it means that there are two or more “cluster clusters”, and the computer 14 determines whether each “cluster cluster” and the robot 10 (object) are in the next step S97. Determine if distances are equal. The distance between the point closest to the object in the “cluster of clusters” and the object is assumed to be di. When the maximum value of the distance di is dimax and the minimum value is dimin, if the equation (8) is satisfied, the distance between each “cluster cluster” and the object is determined to be equal (equal distance). However, D is a constant which shows a threshold value.
[Equation 8]
dimax−dimin <D (8)
If "YES" is determined in the step S97, the computer 14 then determines whether or not there is a "stray cluster" in a step S99. The collective state determined as “NO” is a collective state in which there are two “cluster clusters” that are equidistant with respect to the robot 10 (object). Therefore, “multiple cluster clusters (distance equal)” in FIG. One of the divided group states is “collection state 5”.

  In step S <b> 101, the computer 14 classifies the collective state based on the positional relationship between the “stray cluster”, the “cluster cluster”, and the object (robot 10), as in step S <b> 95.

  As can be seen with reference to FIG. 13, there are two “clusters”, one of which is the same distance from the objects of the “clusters” and labeled “+ close”. ", The collective state at that time is" collective state 6 ". When there are two “cluster clusters” equidistant from the robot 10 and one “stray cluster” labeled “+ far away”, the collective state at that time is “collective state 7”. Then, when there are two “cluster clusters” with equal distance and one “stray cluster” labeled “+ stray”, the collective state at that time is “collective state 8”. In this way, when there are two “cluster clusters” having an equal distance, that is, when “YES” is determined in step S97, the computer 14 estimates the collective state 5, 6, 7 or 8.

  If “NO” is determined in step S97, it means that there are two “cluster clusters” having unequal distances from the object (robot 10). In the next step S103, the computer 14 determines whether or not there is a “stray cluster”. Since the collective state determined as “NO” is a collective state in which there are two “cluster clusters” that are unequal to the robot 10 (object), “multiple cluster clusters (non-uniform distance)” in FIG. "A collective state 9", one of the collective states.

  If “YES” is determined in step S103, the computer 14 collects in a step S105 based on the positional relationship between the “stray cluster”, the “cluster cluster”, and the object (robot 10) as in the previous steps S95 and S101. Classify the state.

  As can be seen with reference to FIG. 13, there are two “chunk clusters” and the two “chunk clusters” that are not equal in distance from the object and that are labeled “+ close”. When there is a “cluster”, the collective state at that time is “collective state 10”. When there are two “cluster clusters” with unequal distances from the robot 10 and one “stray cluster” labeled “+ far away”, the collective state at that time is “collective state 11”. . When there are two “cluster clusters” with uneven distance and one “stray cluster” labeled “+ stray”, the collective state at that time is “collective state 12”. In this way, when there are two “clusters” with uneven distance, that is, when “NO” is determined in step S97, the computer 14 estimates the collective state 9, 10, 11 or 12.

  Table 1 lists the twelve collective states that can be estimated by this system and the states of each cluster.

  Up to this point, it has been described that the collective state estimation system 100 of the embodiment shown in FIG. 1 can correctly estimate the collective state of humans existing in the same closed space 101 as the robot 10 (object). Data (collective state data) indicating any one of the collective state 1 to the collective state 12 estimated in this way is stored in the data memory 15b together with the time data in step S9 of FIG. At the same time, the collective state data and the time data are transmitted to a device that uses the collective state estimated by the computer 14, in the embodiment, the robot 10. The robot 10 temporarily stores the collective state data received in this manner in its own data memory 66b. In the embodiment, the collective state data is intended to include not only the collective state data but also the position (center of gravity) data of each cluster and the position data of the reaction points included therein. Therefore, by referring to the collective state data, not only the collective state at that time, but also the number and position data of the “cluster cluster” and “stray cluster”, and the number and position data of each reaction point (element) are all included. I understand.

  FIG. 14 is a flowchart illustrating an example of a group attention control operation performed by the robot 10 that has received the group state data (group state data) estimated by the computer 14 from the computer 14. Here, an example of group attention control will be described with reference to FIG.

  If the stop command by the operator is not detected in the first step S121 in FIG. 14, the processor 62 of the robot 10 determines whether or not the collective state data transmitted from the computer 14 is received in the subsequent step S123, and “NO”. Then wait until you receive it. If “YES”, in the next step S125, for example, the caution control flag A set in the data memory 66b is reset to “0” (A = 0). The attention control flag A indicates that the robot 10 needs to execute group attention control on the group when it is reset, and when it is reset (A = 1), for example, Indicates that services other than collective attention control, such as directions and exhibition explanations, should be provided to the intended audience.

  In step S127, the processor 62 of the robot 10 refers to the collective state data stored in the data memory 66b, and determines whether there is a “stray cluster”. That is, it is determined whether or not the collective state data received at that time is collective state data other than “collective state 1”, “collective state 5”, and “collective state 9” shown in FIG. When “YES” is determined in step S 127, the processor 62 of the robot 10 in step S 129 calls a voice message that calls for cancellation of the “stray cluster” to humans existing around the robot 10, for example, “everyone. ,Please gather around. Is output from the speaker 54 (FIGS. 3 and 4), and the process ends.

  When “NO” is determined in step S127, that is, the collective state data at that time indicates any one of “collective state 1”, “collective state 5”, and “collective state 9” shown in FIG. In the subsequent step S131, the processor 62 of the robot 10 determines whether or not it is in the “multiple cluster (distance unevenness)” state, that is, the “collective state 9” state. If “YES”, in the subsequent step S133, the processor 62 of the robot 10 is in a “multiple cluster (distance equal)” state, that is, a “collective state 5” state or a “single group” voice. Message, for example, “Please come here, everyone. Is output from the speaker 54 and the process ends.

  When “NO” is determined in step S131, that is, when the collective state data is collective state data other than “collective state 1” and “collective state 5” shown in FIG. 13, in subsequent step S135, The processor 62 of the robot 10 calculates the distance between the robot 10 and each person, and obtains the variance of the distance between the robot 10 and the person for each “cluster cluster”. Then, in the next step S137, the processor 62 of the robot 10 determines whether there is a “cluster cluster” in which the variance calculated above is D or more, that is, a human, that is, a reaction point is included in each “cluster cluster”. Determine if it is distributed. If “YES”, for example, “Everyone, please come together. Is output, a call is made so that the “cluster” is gathered, and the process ends.

  When “NO” is determined in step S137, the processor 62 of the robot 10 proceeds to step S141. In this step S141, for example, “Everyone, please leave a little further. Or "Please come a little closer, everyone." "Thank you, everyone. ”Or“ Look over there (or right or left). The group attention control is executed by outputting an appropriate voice message such as “”, and the distance between the robot 10 and the human being, the orientation of people's faces, and the like are controlled at the same time.

  However, in addition to the voice message as described above, for example, the right arm motor 78 and the left arm motor 80 (both shown in FIG. 4) are controlled by outputting control data to the motor control board 68, and the right arm or the left arm is pointed. ”Operation may be performed to perform group attention control. Alternatively, group attention control may be performed by controlling the head motor 82 (FIG. 4) by outputting control data to the motor control board 68 and performing a “neck-raising” operation. Further, the group attention control can be performed by outputting the control data to the motor control board 68 to control the right eyeball motor 74 and the left eyeball motor 76 (FIG. 4) to perform the “gaze” operation. That is, the group attention control can be executed with an arbitrary voice or gesture.

  After performing group attention control in step S141, the processor 62 of the robot 10 sets the attention control flag A again (A = 1) in step S143, and ends the process.

  According to this embodiment, since the computer 14 can correctly estimate the collective state, the robot 10 can appropriately perform collective attention control adapted to the collective state at that time.

  In the above-described embodiment, the computer 14 estimates the collective state, transfers collective state data obtained by the collective state estimation operation from the computer 14 to the robot 10, and the robot 10 performs collective attention control according to the collective state. It was explained as something to do. However, the computer 14 simply sends the reaction point data (coordinate data) of the floor sensor 18 to the robot 10, and the robot 10 itself estimates the collective state using the coordinate data according to the algorithm described above. Of course it is good.

  In the embodiment, the robot 10 performs collective attention control on the surrounding humans. Conversely, the collective attention control is performed by the computer 14 or other devices or means instead of the robot 10 according to the estimated collective state. Can also be executed.

  In the above-described embodiment, the embodiment in which the robot 10 (or another device or means) performs the group attention control using the group state data estimated by the computer 14 has been described. However, the group state estimation result can be effectively used for, for example, active sensing whose flowchart is shown in FIG. 15 in addition to such group attention control.

  If a stop command by the operator is not detected in the first step S151 in FIG. 15, the processor 62 of the robot 10 determines whether or not the collective state data transmitted from the computer 14 has been received in the subsequent step S153, and “NO”. Then wait until you receive it. If “YES”, in the next step S155, for example, with reference to the collective state data stored in the data memory 66b, it is determined whether or not there is a “stray cluster” at that time. That is, it is determined whether or not the collective state data received at that time is collective state data other than “collective state 1”, “collective state 5”, and “collective state 9” shown in FIG. If “YES”, the processor 62 of the robot 10 determines whether or not he / she wants to pay attention to the “stray cluster” in the next step S157. For example, when a command to pay attention to “stray cluster” is given to the robot 10 by the computer 14, “YES” is determined in the step S157, and the process proceeds to the next step S159.

  In step S159, with reference to the position data indicating the center of gravity position of the “stray cluster” included in the collective state data, the control data is sent to the motor control board 68 so that the eye camera 60 faces in the direction of the position. The motors 74 and 76 are driven. That is, sensing is performed on the position of the “staggered cluster”.

  When it is determined as “NO” in both steps S155 and S157, the processor 62 of the robot 10 refers to the collective state data at that time in the next step S161, and whether or not there are a plurality of “cluster clusters”, to decide. A group state in which there is no “stray cluster” and there are a plurality of “cluster clusters” is any one of the group state 1, the group state 5, or the group state 9. Therefore, in step S161, it is determined whether or not any of the collective state 1, collective state 5, and collective state 9 is obtained. If “NO”, the collective state at that time is the collective state 1 (only one “cluster cluster” exists), and the processor 62 of the robot 10 continues to check the eye in the same manner as described above in step S163. The camera 60 is controlled to perform sensing for the position of the “cluster cluster”.

  When “YES” is determined in step S161, the collective state at that time is either the collective state 5 or the collective state 9, and in subsequent step S165, the processor 62 of the robot 10 refers to each “cluster cluster”. It is determined whether the distance to the object (robot 10) is constant, that is, whether the collective state 5 or not. When the state is the collective state 5, the processor 62 of the robot 10 refers to the cluster data included in the collective state data, and in step S167, the “cluster cluster” with the larger number of the two “cluster clusters” is selected. Sensing.

  However, when “NO” is determined in step S165, that is, when the collective state is “9”, in step S169, the computer 14 is close to the robot 10 based on the cluster position data included in the collective state data. Sensing the “cluster of clusters”.

  According to this embodiment, since the collective state can be correctly estimated by the computer 14, the robot 10 can appropriately perform active sensing adapted to the collective state at that time.

  In the above-described embodiment, the computer 14 estimates the collective state, transfers collective state data obtained by the collective state estimation operation from the computer 14 to the robot 10, and the robot 10 performs active sensing according to the collective state. Described as what to do. However, the computer 14 simply sends the reaction point data (coordinate data) of the floor sensor 18 to the robot 10, and the robot 10 itself uses the coordinate data to estimate the collective state according to the algorithm described above. Of course, active sensing may be performed.

  Conversely, the active sensing according to the estimated collective state can be executed not by the robot 10 but by the computer 14 or other means or apparatus.

  As an active sensing method, not only the camera is pointed but also any method such as a method of changing the resolution of the camera or a voice microphone pointing in that direction to acquire sound with high sensitivity can be considered.

  In the above-described embodiment, the collective state of the humans surrounding the robot 10 is estimated as the collective state 1-12. However, according to another embodiment of the invention shown in FIGS. 16-18, there are only two types of group states to be classified, for example, “ordered” state or “cluttered” state, thereby allowing various controls, such as It can also be applied to group attention control. However, the “orderly” state is a state where people gather around the robot and the robot is suitable for providing information to many humans. The “cluttered” state is a state where humans are robots. It is a state where the robot is not gathered around the center and a state where the robot is not suitable for providing information to people.

  Also in the two classification examples, the system basically has the configuration shown in FIG. 1, uses the robot 10 shown in FIGS. 3 and 4, and uses a large number of floor sensors arranged as shown in FIG. Get robot and human position information. Then, clustering is performed by the same method as described above, thereby obtaining various feature amounts, and the feature amounts are calculated by, for example, SVM (Support Vector Machine) (Christianini, J., Shawe-Tylor, J. (Large (Takeshi Kita) “Introduction to Support Vector Machine”, Kyoritsu Shuppan 2005) classifies the above two states. However, in this embodiment, the SVM is assumed to be possessed by the computer 14, and the robot 10 receives the two-state data determined by the computer 14 and provides various services such as group attention control to humans. .

  Referring to FIG. 16, if a stop command by the operator is not detected in the first step S171, the computer 14 deletes the reaction of the robot 10 from the floor sensor using the position information of the robot 10 in the subsequent step S173. This is a device for avoiding confusion as to whether the reaction point of the floor sensor 18 is a robot or a human, but this step may be omitted as in the previous embodiment.

  In the next step S175, the computer 14 clusters the reaction points for the floor sensor using the same method as previously described in detail with reference to FIG. However, the description will not be repeated.

In subsequent step S177, the computer 14 calculates the ratio FN of the number of reaction points of the floor sensor according to the equation (9).
[Equation 9]
FN = m / n (9)
Here, n is the total number of floor sensors (FIG. 2), and m is the number of points reacting.

  In subsequent step S179, the computer 14 calculates a distance average μi (i = 1,..., C) between the reaction points (elements) constituting the clusters C1-Cc and the robot 10. Where c is the number of clusters.

  17 is a schematic diagram when a group is arranged in a fan shape with respect to the robot 10, and when the number of reaction points belonging to the cluster A is 1 (el), the reaction points belonging to the cluster A are A1,. And Each reaction point is expressed in xy coordinates as described above. When the coordinates of the reaction points are A1x, A1y,..., Alx, Aly, and the coordinates of the robot 10 are Rx, Ry, the distance Dist (A1) between the robot 10 and the reaction point A1 is given by the equation (10).

  Then, the distance average μA between the cluster A and the robot 10 can be calculated by the equation (11).

  In the next step S181, the computer 14 calculates the standard deviation σi (i = 1,..., L) of the distance average obtained in step S179 according to the equation (12). That is, the standard deviation σA of the average distance between the cluster A and the robot can be calculated by the equation (12).

  Next, in step S183, the computer 14 calculates the center of gravity of each cluster. However, this step S183 can be omitted when the centroid data is included in the cluster data as described above. The center of gravity of cluster A is shown in FIG. The center of gravity G (CA) of cluster A is calculated using equation (13).

  Further, in step S185, the computer 14 calculates an angle θi (i = 1,..., L) between the front direction of the robot 10 and the center of gravity of each cluster. The angle θA (FIG. 17) is a straight angle given by the equation (14).

  All data such as the total number of reaction points m and the number of clusters C of the floor sensor, and the FN, distance average μi, standard deviation σi thereof, center of gravity Gi and angle θi of the clusters calculated in the above-mentioned steps are calculated each time. 14 data memory 15b (FIG. 1) is stored and held as data indicating the feature amount.

  Therefore, in step S191 in FIG. 18, the data is acquired from the data memory 15b. That is, in order to create a 2-class discriminant model using SVM in this embodiment, the total number of reaction points m of the floor sensor, the number of clusters C, the distance average μi between the reaction points constituting the cluster and the robot, and the standard deviation thereof Each feature quantity of σi and an angle θi between the center of gravity G of the cluster and the robot front is used as teacher data. Since the distance average, the standard deviation, and the angle are obtained for the top three clusters having many reaction points, the feature amount is 11 in total (= 1 + 1 + (3 × 3)).

  In FIG. 17, when the distance between the first reaction point A1 and the robot is Dist (A1), the number of reaction points l (el) constituting the cluster A is Dist (A1). The distance average μA is expressed by the above equation (11). The standard deviation σA of the distance average between the robot and all the reaction points of the cluster A is expressed by the above equation (12). The angle θA between the front direction of the robot and the center of gravity of cluster A is as shown in FIG.

  The total number of reaction points and the number of clusters of the floor sensor indicate how many people and groups exist around the robot. The distance average μ, standard deviation, and angle θ It has a shape and represents the positional relationship. By using these feature quantities as teacher data, a two-class discrimination model is created, and collective state estimation is performed in other embodiments.

  In the experiments by the inventors, a floor sensor is laid on the premises of a station, and a communication robot such as Robovie (product name) as shown in FIG. 3 is placed on the floor sensor. The interaction was made possible.

  In order to present a video to a person who evaluates the collective state of people around the robot, the video of this experiment was recorded with six cameras, and the video, robot position, angle, and floor sensor output were recorded. Etc. were all stored in the database 16 (FIG. 1).

  In the experiment, many situations where multiple people gathered around the robot were observed. In those situations, there were mixed scenes where people lined up around the robot and people lined up in tandem.

  Teacher data was created from the data obtained in such experiments. First, the recorded video was observed, and the scene where the person interacting with the robot was stationary for 2 seconds or more was cut out. The video of the 177 scenes cut out was presented to two evaluators and classified into either “orderly” or “cluttered” state. The graders were informed in advance that the “ordered” state was desirable for the robot to provide information.

  The κ statistic was used as a measure for measuring the degree of agreement between the two raters. The κ statistic is generally used as a measure of the degree of agreement between two raters, and is said to be moderate if κ = 0.40 and fairly consistent if κ = 0.60. ing. The κ statistic calculated through the experiment was 0.49.

  There were 44 scenes with different opinions among the graders. If the floor sensor output that is necessary to calculate the feature value is omitted from the scenes that agree, the number of “ordered” states is 36, and the number of “cluttered” states is 72. Among them, 18 “ordered” states and 36 “cluttered” states acquired in the first week of the experiment were used to create teacher data.

  The above-described feature amount is used for creating the teacher data, but the feature amount is calculated by using the floor sensor output for one second in each scene for creating the teacher data in the “cluttered” state. Since the “ordered” state is half of the “cluttered” state (18 against 36), the feature quantity is calculated using the floor sensor output for 2 seconds in each scene in order to equalize the number of data. did.

  In addition, dummy data was created by inverting the output of the floor sensor in the x-axis direction, y-axis direction, and xy-axis direction, respectively. Through these procedures, the final number of teacher data used in the experiment was 720 in each state.

  As test data for verifying the performance of the discriminant model, a total of 54 data acquired in the second week of the experiment is used, and dummy data is created for these data in the same manner as when the teacher data is created, The final test data was 720 in each state.

  As a library for creating a two-state discrimination model using SVM, LIBSVM (Chang, C, C., and Lin, C, J., “LIBSVM: Introduction and Benchmarks,” http: //www.csie.ntu .Edu.tw / cjlin / libsvm). This library can change the performance of the discriminant model by changing various parameters. Therefore, in the experiment, the three parameters of γ, Cost, and Weight were changed within the range shown in Table 2, and the one in which the collective state estimation accuracy for the teacher data was about 90% on average was adopted as the discrimination model. Although a parameter with a discrimination rate of 100% can be set, the model is not used in order to prevent a decrease in generalization performance due to overlearning.

  As a result of discriminating teacher data using the two-state discrimination model designed as described above, it was possible to estimate the “ordered” state with an accuracy of 87.9% and the “cluttered” state with an accuracy of 93.2%. . The parameters of this discrimination model were γ = 2, Cost = 100, and Weight = 2.

  The estimation system for teacher data and test data based on the model created in such a procedure was able to estimate the “ordered” state with an accuracy of 84.6% and the “cluttered” state with an accuracy of 76.3%. .

  In this way, in step S193 of FIG. 18, it is possible to determine whether the state is “orderly” or “cluttered” from the feature amount at that time using the above-described two-classification determination model. Then, such a determination result (group state data) is transmitted from the computer 14 to the robot 10.

  Then, the robot 10 that has received such group status data can change its behavior, such as the method of group attention control, according to the group status at that time, thereby enabling smoother interaction with the group. realizable. The specific method of the group attention control is arbitrary, and the case where the computer 10 or other means or device uses other media is also conceivable in addition to the case where the robot 10 performs it by voice or gesture. Similarly, according to the two states estimated in this way, the active sensing as described above can be performed by the robot 10, the computer 14, or the like, for example.

  In any of the above-described embodiments, the floor sensor is used to acquire the position data of the object and the surrounding human. Since the floor sensor is not affected by disturbances such as occlusion and changes in ambient light, it is advantageous in that stable operation can be expected. In addition, a robot is obtained from image data captured by a ceiling camera provided on the ceiling of the room 101. The position of a person or a person may be specified. Each person has a passive tag, and the person's position is detected by arranging receivers distributed in the room 101 (or vice versa). Can do.

  Furthermore, in the embodiment, the case where the object is the communication robot 10 has been described. However, other objects, such as an exhibition in a museum or the like, may be any object or person that is in the same plane space as a person and in which a group of people may exist around the object.

FIG. 1 is an illustrative view showing one embodiment of a collective state recognition system of the present invention. FIG. 2 is an illustrative view showing a closed space (room) to which the embodiment of FIG. 1 is applied and floor sensors (S1,..., Sn) laid on the floor thereof. FIG. 3 is an illustrative view showing the appearance of the communication robot included in the embodiment of FIG. 1 from the front. FIG. 4 is a block diagram showing an electrical configuration of the communication robot shown in FIG. FIG. 5 is a flowchart showing the collective state estimation operation in the computer 14 of the embodiment of FIG. FIG. 6 is a flowchart showing in detail the operation of step S5 (xy conversion of reaction points) in FIG. FIG. 7 is a flowchart showing in detail a part of the operation of step S7 (collection state estimation) in FIG. FIG. 8 is a flowchart showing in detail the operation of step S35 (clustering) in FIG. FIG. 9 is an illustrative view showing a distance between clusters at the time of clustering. FIG. 10 is a flowchart showing in detail the operation of step S37 (cluster classification) in FIG. FIG. 11 is a flowchart showing in detail the final operation of step S7 (collection state estimation) in FIG. FIG. 12 is a flowchart showing an operation of determining whether “stray cluster” in FIG. 11 corresponds to “+ near stray”, “+ far stray”, or “+ stray”. FIG. 13 is an illustrative view showing a relationship between a state estimated by the collective state estimation program shown in FIG. 11 and a “cluster cluster” or “staggered cluster”. FIG. 14 is a flowchart showing an example of the group attention control operation executed by the robot 10 that has received the group state data from the computer 14. FIG. 15 is a flowchart showing an example of the active sensing operation performed by the robot 10 that has received the collective state data from the computer 14. FIG. 16 is a flowchart showing an example of the operation when the computer 14 calculates the feature amount in another embodiment of the present invention. FIG. 17 is an illustrative view illustrating the angle between the center of gravity of the cluster and the front direction of the robot. FIG. 18 is a flowchart showing an operation when the feature quantity acquired in FIG. 16 is discriminated by the two-state discrimination model.

Explanation of symbols

100 ... Group state estimation system 101 ... Room (closed space)
DESCRIPTION OF SYMBOLS 10 ... Communication robot 14, 62 ... Computer 15a, 66a ... Program memory 15b, 66b Data memory 18 ... Floor sensor

Claims (7)

A collective state estimation system for estimating a collective state of a plurality of humans existing in the same closed space as an object,
Position detecting means for detecting position of each of the plurality of humans and acquiring position data;
Clustering means for clustering according to the position data and dividing into a plurality of clusters,
Classification means for classifying each cluster into a first cluster and a second cluster according to the number of people included in each of the plurality of clusters, and the presence of the first cluster and the second cluster and the position of the object A collective state estimation system comprising collective state estimation means for estimating collective state based on the relationship and outputting collective state data.   The collective state estimation system according to claim 1, wherein the collective state estimation unit estimates a collective state based on the number of the first cluster and the second cluster and a distance between them and the object. A collective state recognition system for recognizing a collective state of a plurality of humans existing in the same closed space as an object,
Position detecting means for detecting position of each of the plurality of humans and acquiring position data;
Clustering means for performing clustering according to the position data and partitioning into a plurality of clusters, and determination means for determining whether the collective state is in an orderly state or a cluttered state based on the feature amounts of the plurality of clusters. A collective state estimation system. A population state is estimated using the population state estimation system according to any one of claims 1 to 3,
A group attention control system comprising group attention control means for executing group attention control according to the group state.   The group attention control system according to claim 4, wherein the group attention control means includes a communication robot, and the communication robot receives the estimation result of the group state and executes group attention control adapted to the group state. A population state is estimated using the population state estimation system according to any one of claims 1 to 3,
An active sensing system comprising active sensing means for performing active sensing according to the collective state.   The active sensing system according to claim 7, wherein the active sensing unit includes a communication robot, and the communication robot receives an estimation result of the collective state and performs active sensing adapted to the collective state.

Images