JP2017177228A - Service provision robot system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a suitable service by a robot.SOLUTION: A service provision robot system (100) uses a robot (10). The robot is remotely controlled by a computer (14). The computer (14) recognizes a group according to an attribute of each person acquired by a people tracking system (S5, S7), and selects contents conforming to an attribute of a group when talking to a membership of the group (S13). Then, the selected contents are spoken to the membership of the group (S15).SELECTED DRAWING: Figure 6

Description

  The present invention relates to a service providing robot system, and more particularly to a service providing robot system that provides services such as store advertisements and advertisements to a person using a robot.

  In patent document 1 and patent document 2, etc., in order for a robot to carry out advertisements and promotions, a technique for changing the contents of advertisements depending on the location is proposed.

JP2007-229855 [G05D 1/02] JP2005-172879 [G09F 21/00, B25J 5/00, ...]

  Neither Patent Document 1 nor 2 considers actively talking to a person from a robot and providing a service suitable for that person.

  Therefore, a main object of the present invention is to provide a novel service providing robot system.

  Another object of the present invention is to provide a service providing robot system capable of providing services suitable for people.

  The present invention employs the following configuration in order to solve the above problems. The reference numerals in parentheses, supplementary explanations, and the like indicate correspondence relationships with embodiments described later to help understanding of the present invention, and do not limit the present invention in any way.

  A first invention is a service providing robot system in which a robot provides a service to a person, the group recognition unit for recognizing the group according to the attribute of each person acquired by the person tracking system, and a member of the group When providing a service, the service providing robot system includes a content selection unit that selects content that matches the attribute of the group, and a content provision unit that provides the selected content to members.

  In the first invention, a service providing robot system (100: reference numerals exemplifying corresponding parts in the embodiment; the same applies hereinafter) includes, for example, a robot (10) capable of speaking, and the robot serves a person. (For example, provision of various information, promotion of shops and products, advertisement). The group recognition unit (14, S5, S7) recognizes a group based on the attributes of each person, for example, position information and history. When a service is to be provided to a member of any group, the content selection unit (14, S13) selects content that matches the attribute of the group from the utterance content database (20), for example, and the service provision unit (14, S15) provides the content to the member.

  According to the first aspect of the invention, the content suitable for the attribute of the group to which the member belongs can be provided, so that the success rate of service provision is increased and the service can be efficiently provided.

  The second invention is dependent on the first invention, and the group recognizing unit recognizes whether the pairs constitute the same group based on a parameter indicating the time-series similarity of the behavior of two members of the pair. This is a service providing robot system.

  In the second invention, the group recognition unit (14, S5, S7) determines the pair (p1) based on the time-series similarity of the behavior (for example, movement) of each person p1, p2 of the pair (p1, p2). , P2) recognizes whether or not to form a group. The fact that the behaviors of the two people are similar in time series indicates that there is a high possibility that the two people form the same group, and the group recognition can be performed accurately.

  According to the second invention, since it is recognized whether or not they are the same group based on the parameter indicating the time-series similarity of the behavior of the two persons in the pair, the group can be recognized accurately.

  The third invention is dependent on the second invention, and the parameter is the similarity between the movement trajectories of the two persons in the pair, and the group recognition unit is the similarity between the average density and the movement trajectory of each of the two persons in the pair. A service providing robot system that recognizes whether a pair forms a group based on

  In the third invention, the similarity between the movement trajectories of the two pairs is adopted as a parameter indicating the time-series similarity of the behavior of the two pairs. Then, the group recognizing unit determines whether or not the two members of the pair form the same group based on the average density around each person separately detected and the similarity of the movement trajectory. According to the experiments by the inventors, the person who composes the group has almost the same two people in the pair at the time of the follow-up phenomenon (a phenomenon in which another group member chases a little behind the group member). We confirmed passing through the place, and also confirmed that the similarity of this movement trajectory is influenced by the average density of the surroundings. Therefore, the group recognition unit (14, S5, S7) determines whether or not two members of a pair form a group by combining the similarity of the movement trajectory and the average density.

  According to the third aspect, since the similarity of the movement trajectory is adopted as a parameter for group recognition, group recognition can be performed more accurately.

  The fourth invention is dependent on the third invention, and the group recognizing unit considers at least one of the similarity in the movement direction, the similarity in the movement speed, and the average movement distance of the two pairs as another parameter. This is a service providing robot system that recognizes whether two members of a pair are included in the same group.

  In the fourth invention, based on the idea that people who are moving at a similar speed in a certain direction in a certain amount of time for a certain amount of time are a group, Determine if two members of a pair are groups.

  According to the fourth invention, even when the similarity of the movement trajectory is affected by the average density, it is possible to reliably recognize whether the two members of the pair are a group.

  A fifth invention is a service providing program executed by the content of a service providing robot system in which a robot provides a service to a person, the program according to an attribute of each person acquired by the person tracking system. Functions as a group recognition unit for recognizing a group, a content selection unit for selecting content according to an attribute of the group when providing a service to the group member, and a content providing unit for providing the selected content to the member This is a service providing program.

  In the fifth invention, the same effect as in the first invention can be expected.

  According to the service providing robot system of the present invention, the content according to the attribute of the group including the person who provides the service is provided, so that the success rate of the service provision becomes high and the service can be provided efficiently. .

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

FIG. 1 is an illustrative view showing an arrangement example of a shopping center as an example of a place where a service providing robot system according to an embodiment of the present invention can be applied. FIG. 2 is a block diagram showing a service providing robot system according to an embodiment of the present invention. FIG. 3 is an illustrative view showing one example of a robot that can be used in the service providing robot system of the embodiment. FIG. 4 is a block diagram showing an electrical configuration of the robot of FIG. 3 embodiment. FIG. 5 is an illustrative view showing one example of a memory map of a memory attached to the computer of FIG. FIG. 6 is a flowchart illustrating an example of service providing processing executed by the computer of the embodiment. FIG. 7 is a flowchart showing an example of the group recognition process. FIG. 8 is an illustrative view showing one example of a method for sensing a human position and height. FIG. 9 is an illustrative view showing one example of a result of the group recognition processing shown in FIG. FIG. 10 is an illustrative view showing utterance contents according to the group attributes in the embodiment shown in FIG. FIG. 11 is a flowchart showing an example of tuning processing for tuning the group recognition processing shown in FIG.

  Referring to FIG. 1, a service providing robot system 100 according to this embodiment is used in a space (environment) where various people come and go, such as a shopping mall. Within the space, the robot 10 and a person can move arbitrarily, and a plurality of distance image sensors 12 are provided at a relatively high place such as a ceiling.

  The robot 10 is an interaction-oriented robot (communication robot), and has a function of executing communication behavior including voice with a person who is an object of communication. In addition, as part of the communication, the robot 10 provides services such as notification of event information, advertisement of stores in shopping malls, and advertisements to people. The robot 10 basically operates based on an action command given from a remote operation computer 14 (FIG. 2), which will be described later, but moves autonomously in a shopping mall if necessary to provide a service. You can also

  The service providing robot system 100 according to this embodiment can be used not only in a shopping mall but also in an event venue, an attraction venue, or the like.

  In FIG. 1, only one person is shown for simplicity, but there are usually many people in the shopping mall. Furthermore, although only one robot 10 is shown, the service providing robot system 100 can simultaneously control two or more robots 10.

  With reference to FIG. 2, the remote operation computer 14 of the service providing robot system 100 is arbitrarily moved by a plurality of distance image sensors 12 every predetermined time (for example, 1 second), which will be described in detail later. While detecting the position of a person, the direction of the person is detected. For this purpose, the computer 14 receives the sensor output from the distance image sensor 12.

  A memory 16 and a communication LAN board 18 are connected to the computer 14 and an utterance content database (DB) 20 is also connected thereto. The utterance content database 20 registers in advance utterances necessary for talking to people when the robot 10 provides services such as advertisements and promotions to the people and utterances necessary for advertisements and promotions. When necessary, the utterance sentence necessary for the content 4 is read out and given to the robot 10.

  However, the utterance content database 20 may be provided in the robot 10. In that case, the computer 14 need only send to the robot 10 a command for designating an utterance sentence to be uttered to the robot 10.

  The distance image sensor 12 emits light such as infrared light or laser, and captures light reflected from the object (reflected light) by an optical sensor such as a CCD sensor. The distance image sensor 12 measures the actual distance to the object by measuring the time until the light returns for each pixel or measuring the distortion of the image pattern. The distance image sensor 12 of the embodiment employs a product called Xtion manufactured by ASUS (registered trademark). In another embodiment, the distance image sensor 12 is a Kinect (registered trademark) sensor manufactured by Microsoft (registered trademark), a three-dimensional distance image sensor D-IMAGEr (registered trademark) manufactured by Panasonic (registered trademark), or the like. Can also be used. This type of sensor is sometimes called a three-dimensional distance measuring sensor, a 3D scanner, or the like.

  The distance image sensor 12 can perform measurement while changing the inclination angle of the laser scanning surface (scanning surface), and therefore, by one measurement (during a period when the inclination angle changes from the minimum inclination angle to the maximum inclination angle) A three-dimensional shape can be calculated. In addition, even if the object is stopped, its 3D shape can be calculated (however, if the object is moving, measurement can be performed with various positional relationships, so that the calculation accuracy of the 3D shape is improved. preferable).

  In addition, by changing the tilt angle of the scan surface in this way, the measurement area is expanded, and even when a plurality of objects are densely packed, one is based on measurement data measured directly above or in the vicinity thereof. It can be easily separated into one object. And based on the measurement data measured with various inclination angles, the three-dimensional shape of each separated object can be calculated with high accuracy.

  Here, the processing procedure of the computer 14 that implements the person tracking system using the distance image sensor 12 will be described. The detailed processing procedure is disclosed in the co-pending Japanese Patent Application Laid-Open No. 2012-2012 related to the application of the present applicant. 215555, the description is cited here, and only an outline is described here.

  First, the computer 14 executes state estimation processing for estimating a target state (for example, a position, a moving direction, a three-dimensional shape, and an attitude) in real time using a particle filter based on measurement data from each distance image sensor 12. .

  In this embodiment, for example, as shown in FIG. 8, the detection data from the three-dimensional distance image sensor 12 installed above a person is subjected to clustering processing at two heights of the head height and the shoulder height. Calculate the shoulder line from the longest line of the cluster (principal component analysis). The cluster above the shoulder line is defined as the head, and the center of gravity of the head is defined as the person's position (x, y). However, we followed the model in which the head cluster was slightly ahead of the shoulder cluster. And let the front orthogonal to a shoulder line be the direction (movement direction (theta)) of a human body.

  However, a method for extracting the position and movement direction of a person from such clustering is, for example, the above-mentioned Japanese Patent Application Laid-Open No. 2012-215555 and the paper D. Brscic, T. Kanda, T. Ikeda, T. Miyashita, Person tracking. Since it is described in detail in large public spaces using 3D range sensors, IEEE Transactions on Human-Machine Systems, Vol. 43, No. 6, pp. 522-534, 2013, details are omitted here.

  As is well known, the particle filter is a type of time series filter that estimates the current target state by repeating prediction and observation. Specifically, the next state that can occur from the current state is determined. Estimating the likelihood (similarity) between the observed state for each particle as a large number of particles for each particle, and estimating the weighted average of all particles according to the likelihood as the current target state . Then, by generating new particles according to the weight and repeating the same processing, the target state can be estimated sequentially.

  In the state estimation process, the state of each target is estimated by a dedicated particle filter. Therefore, for example, in the state where ten targets are detected, ten particle filters operate in parallel, and when another target is detected, an eleventh particle filter is newly generated.

  In parallel with the state estimation process as described above, the computer 14 also estimates a group estimation process for estimating whether each object is “one person” or a group based on the state of each object, Individual action estimation processing for estimating an action to be performed individually (for example, an action of looking at a store or a guide board) is also executed.

  Then, after the various estimation processes are completed, the computer 14 further analyzes the group behavior based on the estimation result. In this group behavior analysis process, each group is classified into categories such as “friends”, “family”, “couple”, etc., individual behavior information is analyzed for each group, group behavior information is created, The behavior information is analyzed for each category, and a group behavior pattern (for example, behavior patterns such as friends paying attention to a specific store or information board, or family members passing a specific passage) is created.

  However, in this embodiment, as will be described in detail later with reference to FIG. 7, the position information (x, x, x) of the data measured as described above from the current time t to Δt seconds ago. y, z) and the movement speed v and movement direction θ based on the history are used to execute the group recognition process.

  The memory 16 shown in FIG. 2 includes ROM, HDD, RAM, and the like. In the ROM and HDD, a control program for controlling the operation of the computer 14 is stored in advance. The RAM is used as a work memory or buffer memory of the computer 14.

  The communication LAN board 18 is configured by a DSP, for example, and provides transmission data provided from the computer 14 to the wireless communication module 22, and the wireless communication module 22 transmits the transmission data to the robot 10 via the network 24. For example, the transmission data includes data necessary for autonomous movement of the robot 10, data necessary for providing a service, and a signal (command) of an action command instructing the robot 10. The communication LAN board 18 receives data via the wireless communication module 22 and gives the received data to the computer 14.

  The computer 14 may include an output device such as a display and input devices such as a mouse and a keyboard.

  Here, with reference to FIG. 2 and FIG. 3, the configuration of the robot 10 will be described within a range necessary for understanding the present invention. The robot 10 includes a carriage 30, and two wheels 32 and one slave wheel 34 that move the robot 10 autonomously are provided on the lower surface of the carriage 30. The two wheels 32 are independently driven by a wheel motor 36 (see FIG. 3), and the carriage 30, that is, the robot 10 can be moved in any direction, front, back, left, and right.

  A cylindrical sensor attachment panel 38 is provided on the carriage 30, and a number of distance sensors 40 are attached to the sensor attachment panel 38. These distance sensors 40 measure distances from objects (such as people and obstacles) around the robot 10 using, for example, infrared rays or ultrasonic waves.

  A body 42 is provided upright on the sensor mounting panel 38. Further, the above-described distance sensor 40 is further provided at the front upper center of the torso 42 (a position corresponding to a person's chest), and measures the distance from the person mainly in front of the robot 10. Further, the body 42 is provided with a support column 44 extending from substantially the center of the upper end of the side surface, and an omnidirectional camera 46 is provided on the support column 44. The omnidirectional camera 46 photographs the surroundings of the robot 10 and is distinguished from an eye camera 70 described later. As this omnidirectional camera 46, for example, a camera using a solid-state imaging device such as a CCD or a CMOS can be adopted.

  An upper arm 50R and an upper arm 50L are provided at upper end portions on both sides of the torso 42 (position corresponding to a human shoulder) by a shoulder joint 48R and a shoulder joint 48L, respectively. Although illustration is omitted, each of the shoulder joint 48R and the shoulder joint 48L has three orthogonal degrees of freedom. That is, the shoulder joint 48R can control the angle of the upper arm 50R around each of three orthogonal axes. A certain axis (yaw axis) of the shoulder joint 48R is an axis parallel to the longitudinal direction (or axis) of the upper arm 50R, and the other two axes (pitch axis and roll axis) are orthogonal to the axes from different directions. It is an axis to do. Similarly, the shoulder joint 48L can control the angle of the upper arm 50L around each of three orthogonal axes. A certain axis (yaw axis) of the shoulder joint 48L is an axis parallel to the longitudinal direction (or axis) of the upper arm 50L, and the other two axes (pitch axis and roll axis) are orthogonal to the axes from different directions. It is an axis to do.

  In addition, an elbow joint 52R and an elbow joint 52L are provided at the respective distal ends of the upper arm 50R and the upper arm 50L. Although illustration is omitted, each of the elbow joint 52R and the elbow joint 52L has one degree of freedom, and the angle of the forearm 54R and the forearm 54L can be controlled around the axis (pitch axis).

  At the tip of each of the forearm 54R and the forearm 54L, a hand 56R and a hand 56L corresponding to a human hand are provided. Although the detailed illustration is omitted, these hands 56R and 56L are configured to be openable and closable so that the robot 10 can grip or hold an object using the hands 56R and 56L. However, the shape of the hands 56R and 56L is not limited to the shape of the embodiment, and may have a shape and a function very similar to a human hand.

  Although not shown, the front surface of the carriage 30, a portion corresponding to the shoulder including the shoulder joint 48R and the shoulder joint 48L, the upper arm 50R, the upper arm 50L, the forearm 54R, the forearm 54L, the hand 56R, and the hand 56L, A contact sensor 58 (shown generically in FIG. 3) is provided. The contact sensor 58 on the front surface of the carriage 30 detects contact of the person 16 or other obstacles with the carriage 30. Therefore, when the robot 10 is in contact with an obstacle during its movement, the robot 10 can detect it and immediately stop driving the wheels 32 to suddenly stop the movement of the robot 10. Further, the other contact sensors 58 detect whether or not the respective parts are touched.

  A neck joint 60 is provided at the upper center of the body 42 (a position corresponding to a person's neck), and a head 62 is further provided thereon. Although illustration is omitted, the neck joint 60 has a degree of freedom of three axes, and the angle can be controlled around each of the three axes. A certain axis (yaw axis) is an axis directed directly above (vertically upward) of the robot 10, and the other two axes (pitch axis and roll axis) are axes orthogonal to each other in different directions.

  The head 62 is provided with a speaker 64 at a position corresponding to a human mouth. The speaker 64 is used for the robot 10 to communicate with the surrounding people by voice. A microphone 66R and a microphone 66L are provided at a position corresponding to a human ear. Hereinafter, the right microphone 66R and the left microphone 66L may be collectively referred to as a microphone 66. The microphone 66 captures ambient sounds, in particular, the voice of the person 16 who is a target for communication. Further, a right eyeball portion 68R and a left eyeball portion 68L are provided at positions corresponding to human eyes. The right eyeball portion 68R and the left eyeball portion 68L include a right eye camera 70R and a left eye camera 70L, respectively. Hereinafter, the right eyeball part 68R and the left eyeball part 68L may be collectively referred to as the eyeball part 68. The right eye camera 70R and the left eye camera 70L may be collectively referred to as an eye camera 70.

  The eye camera 70 captures a human face approaching the robot 10, other parts or objects, and captures a corresponding video signal. In this embodiment, the robot 10 detects the line-of-sight directions (vectors) of the left and right eyes of the person based on the video signal from the eye camera 70.

  The eye camera 70 can be the same camera as the omnidirectional camera 46 described above. For example, the eye camera 70 is fixed in the eyeball unit 68, and the eyeball unit 68 is attached to a predetermined position in the head 62 via an eyeball support unit (not shown). Although illustration is omitted, the eyeball support portion has two degrees of freedom, and the angle can be controlled around each of these axes. For example, one of the two axes is an axis (yaw axis) in a direction toward the top of the head 62, and the other is orthogonal to the one axis and goes straight in a direction in which the front side (face) of the head 62 faces. It is an axis (pitch axis) in the direction to be performed. By rotating the eyeball support portion around each of these two axes, the tip (front) side of the eyeball portion 68 or the eye camera 70 is displaced, and the camera axis, that is, the line-of-sight direction is moved. Note that the installation positions of the speaker 64, the microphone 66, and the eye camera 70 described above are not limited to those portions, and may be provided at appropriate positions.

  As described above, the robot 10 of this embodiment includes independent two-axis driving of the wheels 32, three degrees of freedom of the shoulder joint 48 (6 degrees of freedom on the left and right), one degree of freedom of the elbow joint 52 (2 degrees of freedom on the left and right), It has a total of 17 degrees of freedom, 3 degrees of freedom for the neck joint 60 and 2 degrees of freedom for the eyeball support (4 degrees of freedom on the left and right).

  FIG. 3 is a block diagram showing an electrical configuration of the robot 10. Referring to FIG. 3, robot 10 includes a CPU 80. The CPU 80 is also called a microcomputer or a processor, and is connected to the memory 84, the motor control board 86, the sensor input / output board 88 and the audio input / output board 90 via the bus 82.

  Although not shown, the memory 84 includes a ROM, an HDD, and a RAM. Various programs described later are stored in advance in the ROM and the HDD.

  The motor control board 86 is configured by, for example, a DSP, and controls driving of motors for the axes such as the arms, the neck joint 60, and the eyeball unit 68. That is, the motor control board 86 receives control data from the CPU 80, and controls two motors (collectively indicated as “right eyeball motor 92” in FIG. 3) that control the angles of the two axes of the right eyeball portion 68R. Control the rotation angle. Similarly, the motor control board 86 receives control data from the CPU 80, and controls two angles of the two axes of the left eyeball portion 68L (in FIG. 3, collectively referred to as “left eyeball motor 94”). ) To control the rotation angle.

  The motor control board 86 receives control data from the CPU 80, and includes a total of four motors including three motors for controlling the angles of the three orthogonal axes of the shoulder joint 48R and one motor for controlling the angle of the elbow joint 52R. The rotation angle of two motors (collectively indicated as “right arm motor 96” in FIG. 3) is controlled. Similarly, the motor control board 86 receives control data from the CPU 80, and includes three motors for controlling the angles of the three orthogonal axes of the shoulder joint 48L and one motor for controlling the angle of the elbow joint 52L. The rotation angles of a total of four motors (collectively indicated as “left arm motor 98” in FIG. 3) are controlled.

  Further, the motor control board 86 receives control data from the CPU 80, and controls three motors that control the angles of the three orthogonal axes of the neck joint 60 (in FIG. 3, collectively indicated as “head motor 100”). Control the rotation angle. The motor control board 86 receives control data from the CPU 80 and controls the rotation angles of the two motors (collectively indicated as “wheel motor 36” in FIG. 3) that drive the wheels 32.

  A hand actuator 108 is further coupled to the motor control board 86. The motor control board 86 receives control data from the CPU 80 and controls the opening and closing of the hands 56R and 56L.

  Similar to the motor control board 86, the sensor input / output board 88 is configured by a DSP and takes in signals from each sensor and gives them to the CPU 80. That is, data relating to the reflection time from each of the distance sensors 40 is input to the CPU 80 through the sensor input / output board 88. The video signal from the omnidirectional camera 46 is input to the CPU 80 after being subjected to predetermined processing by the sensor input / output board 88 as necessary. Similarly, the video signal from the eye camera 70 is also input to the CPU 80. Further, signals from the plurality of contact sensors 58 described above (collectively indicated as “contact sensors 58” in FIG. 3) are provided to the CPU 80 via the sensor input / output board 88. Similarly, the voice input / output board 90 is also configured by a DSP, and voice or voice in accordance with voice synthesis data provided from the CPU 80 is output from the speaker 64. In addition, voice input from the microphone 66 is given to the CPU 80 via the voice input / output board 90.

  The CPU 80 is connected to the communication LAN board 102 via the bus 82. The communication LAN board 102 is configured by a DSP, for example, and provides transmission data given from the CPU 80 to the wireless communication module 104. The wireless communication module 104 transmits the transmission data to a server (not shown) or the like via the network. . Further, the communication LAN board 102 receives data via the wireless communication module 104 and gives the received data to the CPU 80.

  FIG. 5 is an illustrative view showing one example of a memory map of the memory 16 in the computer 14 shown in FIG. 2, and the memory 16 includes a program storage area 202 and a data storage area 204. The program storage area 202 stores a measurement program 206 disclosed in Japanese Patent Laid-Open No. 2012-215555 described above. The program storage area 202 further includes a group recognition program 208 shown in detail in FIG. 7 and a service providing program 210 shown in detail in FIG. However, the loop recognition program 208 also includes a tuning program for tuning the group recognition program shown in FIG.

  In the data storage area 204, a measurement data buffer 212 is provided. The measurement data buffer 210 is information indicating the measurement result by the human tracking system as described above, the state and attribute / profile of each target, and the position, moving direction, three-dimensional shape and posture, and attributes as the state / Height, gender and adult / child distinction as profiles are stored. However, the group recognition is executed by the group recognition program 208 described above, and the result is stored in the data storage area 204 as group data 214. The group data 214 stores a pair (id1, id2,) of a person identification number id and a group flag (not shown) indicating whether the group is a group. If people id1 and id2 are the same group, group data (id1, id2, true ("1")) is stored as group data 214. If people id1 and id3 are not the same group, (id1, id3, false (“0”)) is stored as group data 214.

  The map data 216 is a map of a place where the service providing robot system 100 is applied, an event venue, a commercial facility such as a shopping center, and the like, and is used to determine the position of the robot 10 or a person.

  Note that the data storage area 204 stores the three-dimensional shape model data shown in FIG. 8 and the like, which is omitted here by citing JP 2012-215555 as an example.

  Referring to FIG. 6, in the first step S <b> 1 of the service providing process, the computer 14 receives the position information (id, x, y, z) of all persons from the measurement data buffer 210 shown in FIG. 5 at that time. To get. Here, id is the identification number of each person, x is the x coordinate, y is the y coordinate, and z is the height of the person's back. In the shopping mall of FIG. 1, the width direction of the paper surface is the x axis (x coordinate), and the depth direction of the paper surface is the Y axis (y coordinate).

  In the next step S3, the computer 14 determines whether each person is an adult or a child based on the height information z included in the position information of each person acquired in step S1. However, in this program, as an example, a person whose z information indicates a numerical value less than “130 cm” is determined as “child”. However, other parameters may be used for the adult / child determination together with the height information z or instead of the height information z.

  In subsequent step S7, the computer 14 identifies the group. That is, the ids of people belonging to the same group are grouped into one group based on the position information of each person. When group information is stored as measurement result data in the measurement data buffer 210 (FIG. 5), this step S7 may be omitted, but in this embodiment, as described above. The group is recognized by executing the group recognition subroutine shown in FIG.

  Here, the group recognition processing will be described with reference to FIG. In the first step S21 of FIG. 7, all person ids (id = person identification number) from the measurement data buffer 212 (FIG. 5) measured and acquired by the person tracking system from the current time t to Δt seconds ago. ) Position information (positions marked with x in FIG. 8) x, y, person height z, person movement speed v, and person movement direction θ.

  Thereafter, in step S23, the computer 14 initializes the group. That is, all people are set as another group, and a set {pid} having the number of elements “1” is formed. Here, pid indicates the id of the person p included in the set.

  Next, in step S25, the computer 14 calculates an average density D (p) of people within a radius of 5 m of each person p1. This average density D (p) is the number of persons existing within a radius of 5 m centering on the person p1 for Δt seconds. The smaller the average density G (p), the higher the possibility that a group is formed. In step S35, which will be described later, this numerical value is used to determine whether the pair (p1, p2) is the same group.

  In steps S27 to S33, the computer 14 uses the positional information (x, y) of each person's pair (p1, p2), and the similarity of the movement trajectory of each person p1 and p2 forming the pair. SimTraj (p1, p2), average of moving direction similarity Simθ (p1, p2), average of moving speed similarity SimV (p1, p2), average of people distance D (p1, p2) between pairs calculate.

  If the movement trajectories are similar, there is a high possibility that they are the same group. Therefore, in step S35, which will be described later, this numerical SimTaj (p1, p2) is used to determine whether the pair (p1, p2) is the same group. It was decided to. That is, one of the features of this embodiment is to use the similarity Tram (p1, p2) of the movement trajectory not considered in the conventional group recognition as the first parameter.

  According to the inventors' experiments, the behavior of the groups is not just the same movement in the vicinity, but for example, one group member precedes and the other member follows it. There are also phenomena (eg, men walking ahead and women chasing behind). The inventors have confirmed that these two people pass almost the same place (even if there is a space around) at the time of such a follow-up phenomenon.

  In this embodiment, the reason why the similarity of the movement trajectory is employed as a parameter for group recognition is to incorporate this tracking phenomenon into group recognition.

  However, on the other hand, the degree of similarity of the movement trajectory may not always give a correct recognition result in a place where the surrounding density D is too high (if the passage is a narrow passage). Therefore, in this embodiment, the average density D (p) calculated in step S25 is used.

  As described above, the similarity of the movement trajectory greatly functions to recognize the group, particularly the tracking phenomenon, but the recognition result is influenced by the average density D (p) around each person as described above. .

  Therefore, in this embodiment, the similarity in the moving direction is further adopted as a parameter for group recognition.

  That is, if the movement directions are similar, there is a high possibility that they are in the same group. Therefore, in step S35, which will be described later, the pair (p1, p2) is set using the second parameter, the numerical value Simθ (p1, p2). Determine if they are the same group. For the same reason, in step S35, which will be described later, a pair using the moving speed similarity SimV (p1, p2) as the third parameter and the average moving distance AvgD (p1, p2) as the fourth parameter. It is determined whether (p1, p2) are the same group.

  However, these second to fourth parameters may also be used in the conventional group recognition method. It is based on the idea that people moving in a certain amount of time (for example, 5 seconds), in a certain degree, in the same direction, with a certain degree of speed, are in groups. ing.

  Note that it is not necessary to use all of the second to fourth parameters, and at least one of them may be used together with the first parameter (similarity of movement trajectory).

  In step S35, the computer 14 calculates SimTraj (p1, p2), Simθ (p1, p2), SimV (p1, p2) calculated as described above to a learning machine such as an SVM (support vector machine), for example. , AvgD (p1, p2) is input to determine whether the pair (p1, p2) is included in the same group. However, in addition to the learning machine, the calculation of the probability of whether or not to form a group may use a calculation table or model a mathematical expression.

  In step S35, if the similarity Tram (p1, p2) of the movement trajectory exceeds a predetermined threshold and the two movement trajectories are dissimilar, it is determined that 2 belongs to another group only by that fact. May be performed. That is, the largest weight (weight) is given to the similarity Tram (p1, p2) of the movement trajectory, and first, it is determined whether or not they are the same group based on the similarity SimTaj (p1, p2) of the movement trajectory. As a result, for pairs recognized as being in the same group, it may be determined whether they are in the same group using the second parameter to the fourth parameter.

  If Pgroup (p1, p2) determined to form the same group in step S35 is included in the same group, the set of p1 and p2 is combined.

  In this way, a group as shown in FIG. 9 is certified. As shown in FIG. 9, pairs in which Pgroup (p1, p2) are determined to be in the same group are connected by a straight line. Moreover, the area | region enclosed with the broken line is recognized as the same group. Such a group and its constituent members and member attributes (adult, child, etc.) are stored as group data in the data storage area 204 shown in FIG.

  In the above description, the average density G (p), the moving path similarity SimTraj (p1, p2), the moving direction similarity Simθ (p1, p2), the moving speed similarity SimV (p1, p2), The average moving distance AvgD (p1, p2) is input to a determination device such as SVM, for example. However, it is not necessary to use all of these numerical values, and the group can be recognized using one or more necessary numerical values. it can. In particular, the movement trajectory similarity SimTraj (p1, p2), the movement direction similarity Simθ (p1, p2), the movement speed similarity SimV (p1, p2), or the average movement distance AvgD (p1, p2) What is necessary is just to determine whether a pair (p1, p2) is the same group based on the time series similarity.

  In this way, after identifying groups and persons belonging to each group, in step S7, the computer 14 refers to the group data 214 in FIG. 5 and performs a group configuration recognition process. That is, for each group, the member label is determined from the information of adults and children included (step S3), and if the group includes adults and children, the group is “family” and all the groups are adults. Is classified as “adult only”, and when all groups are children, “children only”.

  In step S <b> 9, the computer 14 refers to the position information of each person and determines whether or not a person exists within a range where the robot 10 can speak (for example, a range of radius 2-20 m).

  If “NO” is determined in this step S9, the process returns to the previous step S1, and if “YES”, the computer 14 executes the next step S11, for example, a robot such as a person closest to the robot 10 Select the subject to which 10 speaks.

  Then, in step S13, for example, as shown in FIG. 10, the utterance content that matches (appropriate) the attribute of the group recognized in the previous step S7 is selected from the utterance content database 20, and in accordance with the utterance content in step S15. Speak.

  For example, when talking to a group member whose members are all adults, alcohol discount information is uttered as content. When a member speaks to a group member who is traveling with a family, the discount information of the family restaurant is spoken as content. When talking to a group member whose members are all children, the event information of the toy store is spoken as content. When the group is a couple, for example, information on a pair invitation ticket of a cake shop is uttered as content.

  However, it is conceivable that the firing conditions for providing the content include not only the group attributes but also a specific time, a specific place, etc. as the firing conditions.

  Furthermore, the provided service may include not only the utterance according to the utterance content, but also the content expressed using the limbs of the robot, and the content that reduces the cost using a linked device (for example, a display).

  Further, as a service to be provided, there may be a service that distributes discount coupons or coupons for stores and products. In this case, it is also conceivable that a printer (not shown) is mounted on the robot 10 so that the robot 10 prints and distributes discount tickets and coupons on the spot.

  In this embodiment, since a service that matches the attribute of the group is provided, the success rate of service provision is high, and the service can be provided efficiently.

  In addition, as in Group 3 in FIG. 9, when the number of elements is 1, that is, according to attributes such as whether the person is an adult, a child, a man, or a woman, even if the person does not constitute a group. You can also select and provide content.

  FIG. 11 is a flowchart showing an operation for tuning a parameter as to whether the group recognition process shown in FIG. 7 correctly recognizes a group.

  In the first step S41 of the tuning process, the computer 14 acquires position information (id, x, y, z) and movement information (v, θ) of all persons for Δt seconds from the tuning data set. .

  This data set for tuning is data in which the presence / absence of a group-related label is added to the result data of the human position measurement system, and is prepared as a separate database. It consists of a moving trajectory database (corresponding to measurement data 212) in which position information (t, id, x, y, z, v, θ) is described, and a group label database (corresponding to group data 214). . As described above, the group data 214 includes three data, ie, an id pair and group information (flag).

  Thereafter, in steps S43 to S51, processing similar to that of steps S25 to S33 of FIG. 7 described above is executed.

  In step S53, class information C indicating whether or not each person's pair (p1, p2) is included in the same group is read out. Class information C = 0 indicates that the pair (p1, p2) is a different group, and C = 1 indicates that it is the same group.

  In step S55, for example, when each numerical value SimTraj (p1, p2), Simθ (p1, p2), SimV (p1, p2), and AvgD (p1, p2) are input to the SVM, the prediction rate of the class information C is Tune SVM parameters to be the highest.

  As a result, the group recognition can be performed correctly in the group recognition process of FIG.

  More specifically, a learning device such as SVM usually has a plurality of learning parameters (in the case of SVM, kernel function types (linear and Gaussian functions) for determining a recognition method and their basic parameters). Then, after determining the learning parameters, the learning variables and correct answer labels are given to adjust the internal variables (= learning) so that the recognition rate becomes the highest. However, since the recognition rate differs for each learning parameter, it is necessary to search for a number indicating the highest recognition rate, and such work is called “tuning”.

  Furthermore, although an operation for avoiding overlearning is also performed, this is not a point related to the present invention and is well known in the art, so further explanation is omitted here.

  Note that the communication robot that can be used in the present invention is not limited to the robot 10 described in the embodiment of FIGS.

  Furthermore, the human tracking system for detecting the positions and postures of people around the robot 10 is not limited to the system described in the embodiment. Other configurations of person tracking systems may be utilized.

  Further, in the embodiment, the remote operation computer 14 gives a command to the robot 10 to provide the service. However, instead of the computer 14, the robot 10 such as the CPU 80 in FIG. 4 performs the measurement program 206, the group recognition program 208 and / or the service. The providing program 210 (FIG. 5) may be executed.

DESCRIPTION OF SYMBOLS 10 ... Robot 12 ... Distance image sensor 14 ... Computer 100 ... Service provision robot system

Claims (5)

A service providing robot system in which a robot provides a service to a person,
A group recognition unit that recognizes groups according to the attributes of each person acquired by the person tracking system,
A service providing robot system comprising: a content selection unit that selects content that conforms to an attribute of the group when providing a service to a member of the group; and a content provision unit that provides the selected content to the member.   The service providing robot system according to claim 1, wherein the group recognizing unit recognizes whether or not the pair constitutes the same group based on a parameter indicating a time-series similarity between behaviors of two pairs.   The parameter is the similarity between the movement trajectories of the two pairs, and the group recognition unit forms the group based on the average density of each of the two pairs and the similarity of the movement trajectories. The service providing robot system according to claim 2, wherein the service providing robot system is recognized. Whether the group recognizing unit includes at least one of the similarity in the moving direction, the similarity in the moving speed, and the average moving distance of the two persons in the pair as another parameter, and whether the two persons in the pair are included in the same group. The service providing robot system according to claim 3, which recognizes whether or not. A service providing program executed by content of a service providing robot system in which a robot provides a service to a person, the program including the content,
A group recognition unit that recognizes groups according to the attributes of each person acquired by the person tracking system,
A service providing program that functions as a content selection unit that selects content according to an attribute of the group and a content providing unit that provides the selected content to the member when providing a service to the member of the group.

Images