JP2019087047A - Control program, control method and control device - Google Patents

Abstract

To improve the efficiency of attendance to a plurality of visitors.SOLUTION: When a computer detects a first visitor and a second visitor, the computer identifies respective regions of interest based on an image in which the first visitor and the second visitor are imaged, or on the orientation of a part of one of the first visitor and the second visitor which is detected by a sensor. The computer determines whether the identified region of interest of the first visitor is different from the identified region of interest of the second visitor. When the region of interest of the first visitor is different from the region of interest of the second visitor, the computer executes information providing processing using a plurality of information providing resources possessed by an information providing device. The computer executes information providing processing for the first visitor using the information providing resources according to the region of interest of the first visitor. The computer also executes information providing processing for the second visitor using the information providing resources according to the region of interest of the second visitor.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a control program, a control method, and a control device.

  In recent years, introduction of a customer service robot has been tried at an exhibition hall or a business window. The customer service robot responds to customers in turn for visitors who visit the booths and windows in the exhibition hall. However, in the case where a plurality of visitors approach the customer-service robot, for example, in order from the closest visitor to the customer-service robot, a high-interest visitor may be postponed to a low-interest visitor. On the other hand, it is proposed that the degree of interest is determined for visitors in a predetermined area, and customer service is performed based on the determined degree of interest. It has also been proposed to detect the movement vector of the visitor and change the service provided depending on whether the visitor is a group or an individual.

JP, 2009-248193, A JP, 2014-014899, A

  However, when the customer-service robot responds to a specific visitor, other visitors are likely to be dissatisfied because they are kept waiting or the provided service is changed. That is, it is difficult for the customer service robot to improve service efficiency by simultaneously responding to a plurality of visitors.

  An aspect of the present invention is to provide a control program, a control method and a control device capable of improving the efficiency of customer service to a plurality of visitors.

  In one aspect, the control program causes the computer to execute the following processing. When the computer detects the first visitor and the second visitor, the computer detects an image in which the first visitor and the second visitor have been imaged, or the first image detected by a sensor The respective regions of interest of the first visitor and the second visitor based on the orientation of any one part of the visitor and the orientation of any part of the second visitor. Identify. The computer determines whether or not the identified first visitor's region of interest and the second visitor's region of interest are different. The computer executes the information provision process using a plurality of information provision resources possessed by the information provision apparatus, when the first visitor's interest area and the second visitor's interest area are different. That is, the computer executes the information provision processing for the first visitor using the information provision resource according to the region of interest of the first visitor among the plurality of information provision resources. Further, the computer executes an information provision process for the second visitor using an information provision resource corresponding to the region of interest of the second visitor among the plurality of information provision resources.

  It is possible to improve the efficiency of service for multiple visitors.

FIG. 1 is a view showing an example of arrangement of a customer service robot of the embodiment. FIG. 2 is a block diagram showing an example of the configuration of the customer service robot of the embodiment. FIG. 3 is a diagram showing an example of the coordinates of a block set in the service providing space. FIG. 4 is a diagram showing an example of a space of interest information storage unit. FIG. 5 is a diagram showing an example of a space of interest. FIG. 6 is a diagram showing another example of the space of interest. FIG. 7 is a diagram showing an example of the visitor space information storage unit. FIG. 8 is a diagram showing an example of a range in which the approach to the visitor is noticed. FIG. 9 is a diagram illustrating an example of the resource utilization information storage unit. FIG. 10 is a diagram illustrating an example of the resource utilization information storage unit. FIG. 11 is a diagram illustrating an example of the resource utilization information storage unit. FIG. 12 is a diagram illustrating an example of the resource utilization information storage unit. FIG. 13 is a diagram illustrating an example of the resource utilization information storage unit. FIG. 14 is a diagram illustrating an example of the service scenario storage unit. FIG. 15 is a flowchart showing an example of control processing of the embodiment. FIG. 16 is a flowchart illustrating an example of the interest space identification process. FIG. 17 is a flowchart illustrating an example of the identification process. FIG. 18 is a flowchart illustrating an example of the first estimation process. FIG. 19 is a flowchart illustrating an example of the second estimation process. FIG. 20 is a flowchart illustrating an example of the third estimation process. FIG. 21 is a flowchart illustrating an example of the fourth estimation process. FIG. 22 is a flowchart illustrating an example of the fifth estimation process. FIG. 23 is a flowchart showing an example of the resource utilization coefficient calculation process. FIG. 24 is a flowchart showing an example of the calculation process. FIG. 25 is a flowchart illustrating an example of the resource allocation process. FIG. 26 is a diagram showing a specific example of customer service for a plurality of visitors. FIG. 27 is a diagram showing a specific example of a service scenario. FIG. 28 is a diagram illustrating a specific example of a service scenario. FIG. 29 is a diagram showing a specific example of a service scenario. FIG. 30 is a diagram showing a specific example of a service scenario. FIG. 31 is a diagram showing a specific example of customer service for a plurality of visitors. FIG. 32 is a diagram illustrating a specific example of a service scenario. FIG. 33 is a diagram illustrating a specific example of a service scenario. FIG. 34 is a diagram illustrating a specific example of a service scenario. FIG. 35 shows a specific example of a service scenario. FIG. 36 is a diagram illustrating a specific example of a service scenario. FIG. 37 is a diagram showing a specific example of customer service for a plurality of visitors. FIG. 38 is a diagram showing an example of guiding a plurality of children. FIG. 39 is a diagram illustrating an example of a computer that executes a control program.

  Hereinafter, embodiments of the control program, control method, and control device disclosed in the present application will be described in detail based on the drawings. The disclosed technology is not limited by the present embodiment. In addition, the following embodiments may be combined as appropriate as long as no contradiction arises.

FIG. 1 is a view showing an example of arrangement of a customer service robot of the embodiment. First, the arrangement of the customer service robot (hereinafter, also simply referred to as a robot) 100 will be described with reference to FIG. As shown in FIG. 1, the customer robot 100 is disposed, for example, at a booth of the exhibition hall. Around the customer-service robot 100, a service providing space 20 where service can be provided by the customer-service robot 100 is set. In the booth, a table 21, a wall 22, and a screen 23 for image projection are disposed. The customer robot 100 sets a coordinate system in the service providing space 20, and sets, for example, a space block group of 1 cm ³ units. The customer service robot 100 performs various processing such as specifying the region of interest of the visitor using the space block group.

  FIG. 2 is a block diagram showing an example of the configuration of the customer service robot of the embodiment. The customer robot 100 detects the first visitor and the second visitor based on sensor information acquired from the external sensor 10 installed in, for example, a booth. Upon detecting the first visitor and the second visitor, the customer robot 100 detects the first visitor and the second visitor, or the first detected by the sensor. The region of interest of each of the first visitor and the second visitor is identified based on the orientation of any particular region of the visitor and the orientation of any region of the second visitor. The customer service robot 100 determines whether the identified first visitor's region of interest is different from the second visitor's region of interest. When the region of interest of the first visitor and the region of interest of the second visitor differ, the customer service robot 100 selects the region of interest of the first visitor among the plurality of information provision resources possessed by the information provision device. The information provision processing for the first visitor is executed using the corresponding information provision resource. Further, the customer robot 100 executes the information provision processing for the second visitor using the information provision resource corresponding to the region of interest of the second visitor among the plurality of information provision resources. As a result, the customer service robot 100 can improve the service efficiency for a plurality of visitors.

  Next, the functional configuration of the customer service robot 100 will be described with reference to FIG. As shown in FIG. 2, the customer service robot 100 includes a communication unit 110, a head 111, an eye unit 112, a right arm unit 113, a left arm unit 114, a dialogue unit 115, a display unit 116, and a storage unit 120. And a control unit 130. In the following description, when the head 111, the eye unit 112, the right arm unit 113, the left arm unit 114, the dialogue unit 115, and the display unit 116 are collectively referred to, the robot resource 117 is used. That is, in the present embodiment, the information providing apparatus is, for example, the customer service robot 100, and the information providing resource is the robot resource 117. In addition to the functional units shown in FIG. 2, the customer robot 100 may have various functional units of a known computer, for example, functional units such as various input devices. In the following description, a region of interest is also referred to as a space of interest.

  The communication unit 110 is realized by, for example, a network interface card (NIC). The communication unit 110 is a communication interface connected to the external sensor 10 or another information processing apparatus by wire or wirelessly via a network (not shown), and manages communication of information with the external sensor 10 or another information processing apparatus. .

  For the external sensor 10, for example, a camera, an infrared sensor, an ultrasonic sensor, a radar, LIDAR (Laser Imaging Detection and Ranging), a strain gauge, a voice localization microphone, a pressing mat, a contact sensor, or the like can be used. In addition, for example, the external sensor 10 is asked to wear a visitor a wearable terminal such as a headset incorporating a beacon oscillator, a GPS (Global Positioning System) receiver, an acceleration sensor, a direction sensor, etc. The sensor information of may be used. The external sensor 10 transmits sensor information of these various sensors to the customer robot 100 at predetermined intervals, for example, every second.

  The head 111 is a head of the customer-service robot 100, and includes, for example, an eye unit 112 and a dialogue unit 115. The head 111 can perform, for example, an operation of moving the head vertically and horizontally in accordance with an instruction from the control unit 130.

  The eye unit 112 is an eye of the customer service robot 100, and can express an expression or a sight line by moving within a predetermined range according to an instruction from the control unit 130, for example. The eye unit 112 may incorporate a camera to acquire a captured image.

  The right arm 113 and the left arm 114 are the right arm and the left arm of the customer robot 100, respectively. The right arm unit 113 and the left arm unit 114 can perform various operations such as raising an arm and pointing a direction according to an instruction from the control unit 130, for example.

  The dialogue unit 115 has a microphone and a speaker, and realizes dialogue with the visitor. The dialogue unit 115 outputs, for example, the voice of the visitor input by the microphone to the control unit 130, and outputs the voice signal input from the control unit 130 using a speaker. The dialogue unit 115 may be provided not only at the head 111 but also at other places, for example, the body of the customer service robot 100.

  The display unit 116 is a display device for displaying various information. The display unit 116 is realized by, for example, a liquid crystal display as a display device. The display unit 116 displays various screens such as a display screen input from the control unit 130. The display unit 116 can be provided, for example, on the trunk of the customer service robot 100. Also, each of the resources of the robot resource 117 outputs contact information to the control unit 130 when it detects a visitor's contact with each resource. In the following description, the robot resource 117 and each resource may be expressed as a customer service resource.

  The storage unit 120 is realized by, for example, a storage device such as a random access memory (RAM), a semiconductor memory device such as a flash memory, or a hard disk or an optical disk. The storage unit 120 includes a service providing space storage unit 121, an interest space information storage unit 122, a visitor space information storage unit 123, a resource utilization information storage unit 124, and a service scenario storage unit 125. Further, the storage unit 120 stores information used for processing in the control unit 130.

  The service provision space storage unit 121 stores information of space block groups (hereinafter also referred to as block groups) of the coordinate system of the service provision space 20 set around the customer robot 100. In the service providing space storage unit 121, for example, when there is a shielding object such as a desk or a wall, shielding information is set for the target block.

Here, blocks to be set in the service providing space 20 will be described using FIG. 3. FIG. 3 is a diagram showing an example of the coordinates of a block set in the service providing space. As shown in FIG. 3, the block is, for example, a cube of 1 cm ³ , and the coordinates of each vertex (hereinafter also referred to as each end point) of the block are associated with the block ID (Identifier) to be provided in the service provision space storage unit 121. Remember.

  Returning to the explanation of FIG. 2, the space of interest information storage unit 122 stores the space of interest information for each visitor. FIG. 4 is a diagram showing an example of a space of interest information storage unit. As shown in FIG. 4, the space of interest information storage unit 122 has, for example, one space of interest table for each visitor (user). The interest space table includes "user ID", "interest space determination", "interest object determination", "interest object type", "start time", "end time", "block ID", "end point coordinate value", " It has items such as "interest flag value" and "determination flag".

  The "user ID" is an identifier for identifying a visitor (user). The “interest space determination” is information indicating whether or not there is an interest space corresponding to the visitor. The “interested object determination” is information indicating whether or not there is an interest object that the visitor is interested in. The “interest object type” is information indicating the type of the object of interest, for example, information such as a robot or a screen. The “start time” is information indicating the generation start time of the space of interest table. The “end time” is information indicating the generation end time of the space of interest table. The “block ID” is an identifier that identifies each block in the space of interest. The “end point coordinate value” is information indicating a coordinate value in the service providing space 20 of each end point of the block. The "interest flag value" is information to which a numerical value is added when the visitor is interested. The “determination flag” is information that is set as the interest space when the interest flag value is equal to or greater than the threshold.

  Here, the space of interest will be described using FIGS. 5 and 6. FIG. 5 is a diagram showing an example of a space of interest. In the example of FIG. 5, assuming that the visitor A looks at the screen 24, the line of sight of the visitor A is stationary in the direction of the screen 24, and the screen 24 is on the extension of the line of sight. It becomes the space of the screen 24 and its vicinity.

  FIG. 6 is a diagram showing another example of the space of interest. In the example of FIG. 6, the screen 24 and the customer service robot 100 are arranged side by side, and the visitor A looks at the customer service robot 100. In this case, since the line of sight of the visitor A is stationary in the direction of the customer robot 100 and the customer robot 100 is on the extension of the line of sight direction, the space of interest is the customer robot 100 and a space in the vicinity thereof.

  Returning to the explanation of FIG. 2, the visitor space information storage unit 123 stores the probability that the visitor notices at the start of the service and at the end of the service, for each sensor such as the visitor's eyes and ears. FIG. 7 is a diagram showing an example of the visitor space information storage unit. As shown in FIG. 7, the visitor space information storage unit 123 has, for example, a visitor space table 123a corresponding to an eye, and a visitor space table 123b corresponding to an ear. The visitor space table 123a includes items such as "user ID", "user sense device", "robot resource", "probability of noticing at start of service", "probability of noticing at end of service", "start time", "end time" Have. Further, the visitor space table 123a has items such as "sensor sense neighborhood block ID", "end point coordinate value", "disparity indicated by robot resource", "probability of noticing start", and "probability of noticing end".

  The "user ID" is an identifier for identifying a visitor (user). The "user sense" is information for identifying a sense such as a visitor's eye or ear. “Robot resource” is information such as a robot resource 117 used when the customer service robot 100 takes action on the visitor's sensei, for example, an arm indicating the right arm 113 and the left arm 114, and a head indicating the head 111. is there. The “probability of noticing at the start of service” is information indicating the probability that the visitor notices the action taken for the visitor using the robot resource 117. The “probability of noticing at the end of the service” is information indicating the probability of being noticed by the visitor when the action being performed using the robot resource 117 is ended for the visitor. The "start time" is information indicating the generation start time of the visitor space table 123a. The "end time" is information indicating the generation end time of the visitor space table 123a.

  “Sensor proximity block ID” is an identifier for identifying each block present in the vicinity of the user sensation. The “end point coordinate value” is information indicating a coordinate value in the service providing space 20 of each end point of the block. The “parallax indicated by the robot resource” is, for example, information indicating a parallax corresponding to a moving range of the arm when an operation of shaking the arm is performed. Note that the unit of parallax can be radians (rad). The “probability of noticing the start” is information indicating the probability that the visitor notices the action taken for the visitor using the robot resource 117 in each block. The “probability of noticing the end” is information indicating the probability that the visitor notices when the action being performed using the robot resource 117 is finished for the visitor in each block.

  In the visitor space table 123b, "robot resources" is "voice" and "disparity indicated by robot resources" is "sound pressure level indicated by robot resources" compared to the visitor space table 123a. The other items are the same, so the description will be omitted.

  Here, the range in which the approach to the visitor is noticed using FIG. 8, that is, the range corresponding to the “visitor space table 123a” and the “sensor near block ID” of the visitor space table 123b will be described. FIG. 8 is a diagram showing an example of a range in which the approach to the visitor is noticed. As shown in FIG. 8, each block included in the field of view of the visitor A is set to the “sensor near block ID” of the visitor space table 123 a corresponding to the eye. In addition, each block included in the audible range of the visitor A is set to the “sensor near block ID” of the visitor space table 123 b corresponding to the ear. As a result, in the visitor space information storage unit 123, it is possible to set a range to be noticed for each sense of the visitor.

  Returning to the explanation of FIG. 2, the resource utilization information storage unit 124 stores information on how to allocate the robot resource 117 to each visitor. 9 to 13 are diagrams showing an example of the resource utilization information storage unit. The resource utilization information storage unit 124 has, for example, resource utilization tables 124 a to 124 e indicating the distribution of robot resources 117 (service resources) according to the conditions of a plurality of visitors. In the following description, although the case of two visitors A and B will be described for simplicity, the present invention is not limited to this, and the resource utilization information storage unit 124 corresponds to three or more visitors. You may do so. Further, the allocation of each resource utilization table 124 a or the like of the resource utilization information storage unit 124 may be dynamically calculated according to the visitor. In the following description, it is assumed that the resource utilization table 124a is an example, and the visitor A is on the right side of the customer robot 100 and the visitor B is on the left side.

  As shown in FIG. 9, the resource utilization table 124 a includes “condition”, “robot resource”, “allocation of current status”, “unit”, and probability that the visitor A or B corresponding to each action notices “(1)”. ~ Have an item such as "(4)". Also, the resource utilization table 124 a has items such as “value indicating the utility when changing to B” and “allocation after review of resource allocation”.

  "Situation" is information indicating the situations of the visitors A and B. Also, "status" includes information indicating the status of changing the allocation. In the example of the resource utilization table 124a, the visitor A who is the predecessor is interested in only the screen 23 shown in FIG. 1, for example, and the visitor B who visited later looks at the customer robot 100 at a glance. In the resource utilization table 124a, in this situation, the visitor A is the preceding service beneficiary (the visitor A looks almost at the screen 23 away from the customer service robot 100), and the robot appears for the visitor B who appears later This is a case of considering reassignment of the resource 117.

  The “robot resource” is information indicating the robot resource 117 used when the customer service robot 100 takes action on the visitor. The “current allocation” is information indicating the current allocation of the robot resource 117. In the example of FIG. 9, of the robot resource 117, the head 111 to the dialogue unit 115 are allocated to the visitor A, and the display unit 116 indicates that the robot resource 117 is not allocated to either the visitor A or B. The “unit” is a unit in the case of allocating each resource of the robot resource 117.

  In “(1)”, when each resource of the robot resource 117 starts an action for the visitor A and the visitor A receives a benefit, that is, the service robot 100 starts an action for the visitor A (right side Show the probability that the visitor A notices. In (2), when each resource of the robot resource 117 stops the action for the existing visitor A and the visitor A suffers an obstruction, that is, the customer service robot 100 stops the action for the existing visitor A And show the probability that visitor A (not to the right) notices. In “(3)”, when each resource of the robot resource 117 starts an action for a new visitor B, and the visitor B receives a benefit, that is, the customer service robot 100 starts an action for a new visitor B And the probability that visitor B (not on the left) notices. In “(4)”, when each resource of the robot resource 117 starts an action for a new visitor B, and the visitor A receives a hindrance, that is, the customer service robot 100 starts an action for a new visitor B And show the probability that visitor A (not to the right) notices.

  For example, if the “value indicating the effect of changing to B” is a positive value, changing the distribution of the robot resource 117 to the visitor B is expected to be useful to cause the customer-service robot 100 to take action. Show the case. If the value is a negative value, it indicates that a decrease in satisfaction of the customer service for the visitors A and B can be suppressed if the allocation of the robot resource 117 is not changed to the visitor B. That is, “value indicating utility when changing to B” is a positive value, changes the allocation of robot resource 117 to visitor B, and a negative value changes the allocation of robot resource 117. Leave Visitor A. The “value indicating the utility when changing to B” can be calculated, for example, based on the probability “(1)” to “(4)” that the visitor A or B corresponding to each action notices. The head 111 to the dialogue unit 115 of the robot resource 117 can calculate “the value indicating the effect when changing to B” in “(3) − (2) − (4)”. Since the display unit 116 of the robot resource 117 is not allocated to any of the visitors A and B in the "current allocation", the display unit 116 changes to "B" in "(3)-(1)-(4)". A value indicating the utility of the case can be calculated.

  “Distribution after review of resource allocation” indicates a value in which each resource of the robot resource 117 is allocated to either of the visitors A and B based on “the value indicating the utility when changing to B”. In the example of FIG. 9, all the resources of the robot resource 117 are allocated to the visitor B. That is, in the example of FIG. 9, since the visitor A is not interested in the customer service robot 100, all resources of the robot resource 117 can be allocated to the visitor B.

  In the resource utilization table 124b shown in FIG. 10, the "status" indicates that the visitor A who is the predecessor has an interest in the customer robot 100 and also an interest in the screen 23. In the resource utilization table 124b, in this situation, the visitor A is the preceding service beneficiary (looking at the customer robot 100 and the screen 23 in half), and the robot resource 117 is replaced by the visitor B who appears later It is a case to consider. In the resource utilization table 124 b, in the “current allocation”, all the robot resources 117 are allocated to the visitor A. In the resource utilization table 124b, as in the resource utilization table 124a, when "a value indicating the utility when changing to B" is calculated, the left arm 114 is a positive value and the remaining is a negative value. Therefore, in the resource utilization table 124b, the allocation after the review of the resource allocation is changed from the visitor A to the visitor B in the left arm part 114. That is, in the example of FIG. 10, since the visitor A also has an interest in the customer service robot 100, the resource of the robot resource 117 allocated to the visitor B is only the left arm 114.

  The resource utilization table 124c illustrated in FIG. 11 is a state in which the “status” is such that the visitor A who is the predecessor occupies most of the customer service robot 100. In the resource utilization table 124c, in this situation, the visitor A is a precedent service beneficiary (with almost all of the customer service robot 100 occupied), and is considering a change of the robot resource 117 to the visitor B who appears later . In the resource utilization table 124 c, in the “current allocation”, all the robot resources 117 are allocated to the visitor A. In the resource utilization table 124c, all values of the robot resource 117 become negative values when “a value indicating the effect when changing to B” is calculated as in the resource utilization table 124a. Therefore, in the resource utilization table 124c, in the allocation after the review of the resource allocation, all the resources of the robot resource 117 remain allocated to the visitor A, and there is no robot resource 117 allocated to the visitor B . That is, in the example of FIG. 11, since visitor A occupies service robot 100, there is no resource of robot resource 117 allocated to visitor B.

  In the resource utilization table 124 d shown in FIG. 12, the “condition” is a situation where the visitor A is not the predecessor and both the visitor A and the visitor B simultaneously access the customer service robot 100. The resource utilization table 124 d is a case where, in this situation, both the visitor A and the visitor B simultaneously appear to consider to which of the robot resources 117 is to be allocated. In the resource utilization table 124d, in the “current allocation”, the robot resource 117 is not allocated to either of the visitors A and B. In the resource utilization table 124 d, among the probabilities “(1)” to “(4)” that the visitor A or B corresponding to each action notices, “(2)” to “(4)” are the resource utilization table 124 a and It is different.

  In “(2)”, when each resource of the robot resource 117 starts an action for the visitor A and the visitor B suffers an obstruction, that is, the service robot 100 starts an action for the visitor A (left side Show the probability that the visitor B notices. In “(3)”, when each resource of the robot resource 117 starts an action for the visitor B and the visitor B receives a benefit, that is, the service robot 100 starts an action for the visitor B (left side Show the probability that the visitor B notices. In “(4)”, when each resource of the robot resource 117 starts an action for the visitor B, and the visitor A receives a hindrance, that is, the customer service robot 100 starts an action for the visitor B (right side Show the probability that the visitor A notices.

  For example, if “value indicating the utility based on A direction” is a positive value, the utility of the customer robot 100 is expected to take action by changing the distribution of the robot resource 117 to the visitor A. Show the case. If the value is a negative value, changing the distribution of the robot resource 117 to the visitor B indicates a case where the service robot 100 is expected to be effective in causing an action. That is, “value indicating utility when based on A” is a positive value, changing the allocation of robot resource 117 to visitor A, and a negative value, allocating robot resource 117 Is changed to visitor B. The “value indicating the utility based on A direction” is, for example, the probability that the visitor A or B corresponding to each action notices the same value as the “value indicating the utility when changing to B direction” described above. It can be calculated based on “(1)” to “(4)”. For each resource of the robot resource 117, “((1) − (2)) − ((3) − (4))” can be calculated “a value indicating the effect based on A direction”.

  “Distribution after review of resource allocation” indicates a value obtained by allocating each resource of the robot resource 117 to any one of the visitors A and B based on “value indicating utility when using A as a standard”. .

  In the resource utilization table 124 d, when “a value indicating the utility based on A direction” is calculated, the head 111 and the eye 112, the right arm 113, and the dialogue among the resources of the robot resource 117 are calculated. The part 115 has a positive value and is distributed to the visitor A. Further, in the resource utilization table 124 d, the left arm portion 114 and the display unit 116 have negative values, and are distributed to the visitor B. That is, in the example of FIG. 12, since visitor A is not a predecessor and visitor A and B receive service simultaneously, allocation of each resource of robot resource 117 is made according to the position and interest degree of visitor A and B. change. In the example of FIG. 12, the visitor A is allocated three resources, and the visitor B is allocated two resources.

  The resource utilization table 124e shown in FIG. 13 is an example of a resource utilization table in a case where the customer-handling robot 100 can simultaneously use the left and right eyes separately, for example, like a dolphin robot. In the resource utilization table 124e, the “condition” is a situation in which both the visitor A and the visitor B simultaneously access the reception robot 100, and the reception robot 100 simultaneously uses the left and right eyes separately. In the resource utilization table 124e, in this situation, the visitor A and the visitor B simultaneously appear from the left and right, and the customer service robot 100 eventually guides the display 116 of the chest, but until then, it is a simultaneous and parallel customer service There is a case. In the resource utilization table 124e, among the resources of the robot resource 117, the eye 112 is divided into a right eye and a left eye.

  In the resource utilization table 124e, in the “current allocation”, the robot resource 117 is not allocated to either of the visitors A and B. In the resource utilization table 124e, as in the resource utilization table 124d, when the “value indicating the utility based on A direction” is calculated, the head 111, the dialogue unit 115, and the display unit 116 become zero, and Resources are not allocated to either visitor A or B. The right eye and right arm 113 have positive values and are distributed to the visitor A. Also, the left eye and the left arm 114 have negative values, and are distributed to the visitor B. That is, in the example of FIG. 13, since visitor A and B are not the precedents, and visitors A and B receive service simultaneously, allocation of each resource of robot resource 117 is made according to the positions and interests of visitors A and B. change. In the example of FIG. 13, each resource of the robot resource 117 is allocated to the visitors A and B in a symmetrical manner.

  Returning to the explanation of FIG. 2, the service scenario storage unit 125 stores an operation or the like corresponding to the scenario of the service provided by the customer service robot 100. FIG. 14 is a diagram illustrating an example of the service scenario storage unit. As shown in FIG. 14, the service scenario storage unit 125 has items such as “phase”, “service development space”, “utterance”, and “operation (standard)”. The service scenario storage unit 125 stores, for example, one record for each phase.

  The “phase” is information indicating a service scenario aspect. The “service development space” is information indicating a space where the customer robot 100 provides a service in the phase. The "utterance" is information indicating the content that the customer-service robot 100 utters in the phase. When the phase is “provision of core value”, the utterance content corresponds to the content of the core value, so FIG. 14 shows an example of the major items. “Operation (standard)” is information indicating the standard content for the customer-service robot 100 to operate the head 111, the right arm 113, the left arm 114, and the like in this phase. In addition, depending on the phase, there may be no movement of the customer-service robot 100. For example, in the case of only speech, the content as the operation is omitted.

  Returning to the explanation of FIG. 2, the control unit 130 executes, for example, a program stored in the internal storage device by using a RAM as a work area by a central processing unit (CPU), a micro processing unit (MPU) or the like. Is realized by Further, the control unit 130 may be realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

  The control unit 130 includes a detection unit 131, a specification unit 132, a determination unit 133, and an information provision unit 134, and implements or executes the function and action of the information processing described below. Note that the internal configuration of the control unit 130 is not limited to the configuration illustrated in FIG. 2, and any other configuration may be used as long as the information processing described later is performed.

  The detection unit 131 first sets, as an initial setting, a service providing space 20 for the customer service robot 100 to service the customer. That is, the detection unit 131 sets the coordinate system of the service providing space 20 and the block group. Next, the detection unit 131 sets shielding information of the service providing space 20 for shielding objects such as the table 21, the wall 22, and the screen 23 disposed in the service providing space 20.

  The detection unit 131 acquires sensor information from, for example, the external sensor 10 installed in the booth via the communication unit 110. The detection unit 131 detects a visitor based on the acquired sensor information. That is, the detection unit 131 determines whether a visitor is in the service providing space 20 based on the acquired sensor information. If it is determined that there is no visitor in the service providing space 20, the detecting unit 131 continues to determine whether there is a visitor in the service providing space 20 based on the acquired sensor information.

  If the detection unit 131 determines that there is a visitor in the service providing space 20, the detection unit 131 determines whether there are a plurality of visitors. If there are not a plurality of visitors, the detection unit 131 outputs, to the information providing unit 134, visitor information indicating that the visitor is a single visitor. When there are a plurality of visitors, the detection unit 131 outputs, to the identification unit 132, a specific instruction including, among sensor information, a captured image in which a visitor is imaged and the number N of visitors. In addition, when there are a plurality of visitors, the detection unit 131 outputs detection information including voice, a captured image, and the number N of visitors to the determination unit 133. In other words, the detection unit 131 detects the first visitor and the second visitor based on the sensor information.

  When the specifying unit 132 receives a specifying instruction from the detecting unit 131, the specifying unit 132 executes an interest space specifying process. First, when the identification unit 132 starts the interest space identification process, the identification unit 132 initializes a variable i for identifying a plurality of visitors, for example, to i = 1. The identifying unit 132 detects the number N of visitors in the service providing space 20 based on the identifying instruction. When the identification unit 132 detects the number N of visitors, the identification unit 132 generates an interest space table for N persons. The identifying unit 132 stores the generation start time of the space of interest table in the space of interest table. In addition, the interest space table for N persons can be represented by an equation such as {(N × number of blocks × number of flag types) + (N × flag for table type determination)}.

  The identifying unit 132 determines whether or not there is an unprocessed visitor. When it is determined that there is an unprocessed visitor, the identifying unit 132 sets the visitor i as an identification target. The identifying unit 132 executes a identifying process for the visitor i.

  When the identification unit 132 starts the identification process, first, the position of the visitor i, the head orientation, and the line of sight are acquired based on the captured image included in the identification instruction. The specifying unit 132 may acquire the position of the visitor i, the direction of the head, and the line of sight based on the information such as the radar, the strain gauge, and the sound localization microphone included in the specifying instruction. When acquiring the position of the visitor i, the direction of the head, and the line of sight, the identifying unit 132 executes a first estimation process.

  When the identification unit 132 starts the first estimation process, it determines whether the position of the visitor i, the head orientation or the direction vector of the sight line is stationary in the same direction. When it is determined that the direction vector is not stationary in the same direction, the identification unit 132 ends the identification processing as non-interest space identification, and returns to the interest space identification processing. If the identification unit 132 determines that the direction vector is stationary in the same direction, the spatial information storage unit for interest is based on the stationary position of the visitor i, the head direction, and the direction vector of the sight line. An interest flag value is added to the corresponding blocks of the 122 interest space table. The block group to which the interest flag value is added is, for example, a circumference of 160 degrees ahead of the direction vector of the position of the visitor i, a circumference of 160 degrees ahead of the direction vector of the head, a viewing direction based on the direction vector of the eye Can be mentioned. In the following description, the addition of the interest flag value is performed on the interest space table of the interest space information storage unit 122. Further, the specifying unit 132 may store an index value of the reliability of the stationary time, for example, an average value, a deviation value, and the like in the space of interest table.

  When the specifying unit 132 adds the interest flag value based on the direction vector, the specifying unit 132 refers to the service provision space storage unit 121 and determines whether there is a candidate of interest on the extension of each direction vector. If the identification unit 132 determines that there is no candidate of interest on the extension of each direction vector, the identification unit 132 ends the identification processing as non-interest identification, and returns to the interest space identification processing. If the identification unit 132 determines that there is a candidate of interest on the extension of each direction vector, the smaller one of the view angle vector space of the visitor i and the space based on the outer edge of the candidate of interest Add the interest flag value to the blocks in The identifying unit 132 may add the interest flag values of the neighboring block group of the interest target candidate on a priority basis. Further, the identifying unit 132 adds the interest flag value to a block group in the vicinity of the interest target candidate, ends the first estimation processing, and returns to the identification processing.

  When the first estimation process ends, the identifying unit 132 acquires, for example, the timing of information presentation, an operation, a dialogue, and the like of an object of interest candidate from the external sensor 10 or the information providing unit 134 and operation information of the visitor i. . That is, the identifying unit 132 acquires the motion information of the interest object candidate and the visitor i. When acquiring the motion information of the object of interest candidate and the visitor i, the identifying unit 132 executes a second estimation process.

  When the second estimation process is started, the identifying unit 132 determines, based on the acquired motion information, whether the reaction operation timing of the visitor i matches the operation timing from the interest target candidate. The identifying unit 132 adds the interest flag value to the block group in the vicinity of the candidate of interest, when the reaction operation timing of the visitor i matches the timing of action from the candidate of interest. The identifying unit 132 proceeds to the next process without adding the interest flag value when the reaction operation timing of the visitor i does not match the action timing from the interest target candidate.

  The identifying unit 132 determines, based on the acquired motion information, whether or not the reaction content of the visitor i matches the content of the action from the candidate of interest. The identifying unit 132 adds the interest flag value to a block group in the vicinity of the candidate of interest when the reaction action content of the visitor i matches the content of the action from the candidate of interest. The identifying unit 132 proceeds to the next process without adding the interest flag value when the reaction action content of the visitor i does not match the action content from the interest object candidate.

  For example, the identifying unit 132 acquires, from the information providing unit 134, the information presentation log of the object of interest candidate and the operation log of the visitor i. The identifying unit 132 may acquire the information presentation log and the operation log from the external sensor 10 or another information processing apparatus via the communication unit 110. The identifying unit 132 determines whether the operation timing of the visitor i matches the information presentation timing from the interest target candidate based on the acquired information presentation log and the operation log. When the operation timing of the visitor i matches the information presentation timing from the candidate of interest, the identifying unit 132 adds the interest flag value to the block group in the vicinity of the candidate of interest. If the operation timing of the visitor i does not match the information presentation timing from the candidate of interest, the identifying unit 132 proceeds to the next process without adding the interest flag value.

  The identifying unit 132 determines, based on the acquired information presentation log and the operation log, whether the operation content of the visitor i matches the information presentation content from the interest target candidate. When the operation content of the visitor i matches the information presentation content from the interest object candidate, the identification unit 132 adds the interest flag value to the block group in the vicinity of the interest object candidate, and returns to the original processing. If the operation content of the visitor i does not match the information presentation content from the interest target candidate, the identification unit 132 returns to the original processing without adding the interest flag value.

  When the second estimation process ends, the identifying unit 132 acquires, for example, interaction information of the candidate of interest and the visitor i from the information providing unit 134. When acquiring the dialogue information of the candidate of interest and the visitor i, the identifying unit 132 executes the third estimation process.

  When the identification unit 132 starts the third estimation process, it determines whether the dialog timing of the visitor i matches the dialog timing from the interest target candidate based on the acquired dialog information. The identifying unit 132 adds the interest flag value to blocks near the candidate of interest when the interaction timing of the visitor i matches the timing of interaction from the candidate of interest. If the dialogue timing of the visitor i does not match the dialogue timing from the candidate of interest, the identifying unit 132 proceeds to the next process without adding the interest flag value.

  The identifying unit 132 determines, based on the acquired dialogue information, whether the dialogue information of the visitor i matches the dialogue information from the candidate of interest. If the dialog information of the visitor i matches the dialog information from the candidate of interest, the identifying unit 132 adds the flag value of interest to blocks in the vicinity of the candidate of interest and returns to the original processing. If the dialog information of the visitor i does not match the dialog information from the candidate of interest, the identifying unit 132 returns to the original processing without adding the flag value of interest.

  When the third estimation process ends, the identifying unit 132 acquires, for example, from the external sensor 10 via the communication unit 110, the image of the candidate of interest, who is a person, and the image of the visitor i and audio information. That is, the specifying unit 132 acquires the action information of the object of interest candidate who is a person and the visitor i. When acquiring the behavior information of the candidate of interest which is a person and the visitor i, the identifying unit 132 executes a fourth estimation process.

  When the identification unit 132 starts the fourth estimation process, the behavior of the visitor i and the candidate of interest which is a person is, for example, for a predetermined period, in a predetermined space such as an exhibition booth, based on the acquired behavior information. It is determined whether or not tuning is continued for one minute continuously. In other words, the identifying unit 132 determines whether the interaction history (action tuning, speech alternation tuning, voice meaning tuning) of the visitor i with the candidate of interest which is a person continues to occur in the fixed space for a certain period. Determine If the identifying unit 132 determines that the behavior of the visitor i and the candidate of interest is continuously synchronized in the predetermined space for a predetermined period, the block of the space between the visitor i and the candidate of interest Add interest flag value to group and return to original processing. If the identification unit 132 determines that the behavior of the visitor i and the candidate of interest is not synchronized in a predetermined space continuously for a predetermined period, the process returns to the original processing without adding the interest flag value.

  When the fourth estimation process ends, for example, when the interest target candidate is the customer service robot 100, the specifying unit 132 acquires contact information from each resource of the robot resource 117. That is, the identifying unit 132 acquires contact information (including the contact site and the contact duration time) on the candidate of interest. When acquiring the contact information on the target of interest, the identifying unit 132 executes a fifth estimation process.

  When the identifying unit 132 starts the fifth estimation process, the identifying unit 132 determines whether the contact between the visitor i and the candidate of interest continues for a predetermined period, based on the acquired contact information. When it is determined that the contact between the visitor i and the candidate of interest continues for a predetermined period, the identifying unit 132 adds the flag value of interest to the group of blocks in the vicinity of the contact portion of the candidate of interest and Return to processing. When it is determined that the contact between the visitor i and the candidate of interest does not continue for a predetermined period, the identifying unit 132 returns to the original process without adding the interest flag value.

  When the fifth estimation process ends, the identification unit 132 sets an interest space flag for each block in the interest space table of the interest space information storage unit 122 in a block where the interest flag value is equal to or higher than the threshold. The threshold may be, for example, "6" or more. When the setting of the interest space flag is completed, the identification unit 132 ends the identification processing and returns to the interest space identification processing.

  When the identification processing ends, the identification unit 132 increments a variable i for identifying a visitor, and returns to the determination of whether or not there is an unprocessed visitor. When it is determined that there is no unprocessed visitor, the identification unit 132 stores the generation end time of the interest space table in the interest space table, and ends the interest space identification process. When the identification unit 132 ends the interest space identification process, the identification unit 132 outputs a calculation instruction to the determination unit 133.

  In other words, the identifying unit 132 determines each of the first visitor and the second visitor based on the captured image in which the first visitor and the second visitor included in the sensor information are imaged. Identify a region of interest (space of interest). Alternatively, the identifying unit 132 may be configured to set the first visitor based on the orientation of any one of the first visitor's particular sites detected by the sensor included in the sensor information and the orientation of any one of the second visitors. Identify the region of interest (space of interest) of the visitor and the second visitor.

  Detection information is input to the determination unit 133 from the detection unit 131. When the calculation instruction is input from the identification unit 132, the determination unit 133 executes resource utilization coefficient calculation processing.

  When the resource utilization coefficient calculation process is started, the determination unit 133 detects the number N of visitors in the service providing space 20 based on the detection information. The determination unit 133 generates a visitor space table for N persons in the visitor space information storage unit 123 based on the detected number N of visitors. Note that the determination unit 133 generates a table according to the number of customer service resources M, which is the number of resources of the robot resource 117, as a visitor space table for one person. In addition, when there are a plurality of the number of senses of the visitor corresponding to one resource, the determination unit 133 may generate a table according to the number of the senses. The size of the visitor space table can be expressed, for example, by the following equation: N number of visitors × M number of service resources × M × (Σ (K number of sensory organs × L number of sensory organ neighboring blocks)). Further, the number of representative values for each visitor space table can be expressed, for example, by the formula {N number of visitors N number of service resources M number of sensory organs K number of K}. Furthermore, the determination unit 133 stores the generation start time of the visitor space table in the visitor space table.

  The determination unit 133 determines whether there is an unprocessed visitor. When it is determined that there is an unprocessed visitor, the determination unit 133 sets the visitor i as a calculation target. The determination unit 133 executes a calculation process for the visitor i.

  When the calculation process is started, the determination unit 133 first acquires the positions of the eyes and ears of the visitor i based on the captured image included in the detection information. The determination unit 133 may acquire the positions of the eyes and ears of the visitor i based on the information such as the radar, the strain gauge, and the voice localization microphone included in the detection information. The determination unit 133 refers to the space-of-interest information storage unit 122 based on the acquired eye and ear positions, and acquires the space of interest of the visitor i. The determination unit 133 estimates the parallax and the sound pressure level in the surrounding space of the visitor i based on the acquired space of interest of the visitor i. That is, the determination unit 133 estimates the visibility and audibility of the visitor i based on the positional relationship between the customer service robot 100 and the visitor i. Here, when the space of interest of the visitor i can be specified, the determination unit 133 fixes the direction of the field of view and the head of the visitor i, and estimates the parallax and the sound pressure level. If the space of interest of the visitor i can not be identified, the determination unit 133 can turn the field of view of the visitor i forward of the head, that is, expand the space to estimate the parallax and the sound pressure level. The determination unit 133 stores the estimated disparity and sound pressure level in the surrounding space of the visitor i in each visitor space table of the visitor space information storage unit 123 for each block.

  The determination unit 133 refers to the service providing space storage unit 121 and information on the movable range of each customer service resource of the robot resource 117 set in advance, and calculates the operation space of each customer service resource. Based on the calculated operation space of each customer service resource and the visitor space table corresponding to the visitor i, the determination unit 133 determines the parallax due to the operation of each customer resource in the operation space of the visitor i at the eye position. calculate.

  The determination unit 133 refers to the information of the sound source of each customer service resource of the robot resource 117 set in advance, and calculates the sound pressure level (dB) of the sound emitted by each customer service resource. The determination unit 133 calculates the sound pressure level of the sound of each customer service resource at the position of the visitor i's ear based on the calculated sound pressure level and the visitor space table corresponding to the visitor i.

  The determination unit 133 compares the parallax and sound pressure level corresponding to the calculated position of the eye and ear of the visitor i with the parallax and sound pressure level in the surrounding space of the visitor i estimated, and the visitor i The probability of noticing and the probability of noticing are calculated for the operation and sound of each customer service resource. The determination unit 133 determines whether the calculation result, that is, the parallax and sound pressure level corresponding to the positions of the eyes and ears of the visitor i, the probability of noticing the operation and sound of each customer service resource, and the probability of not being noticed It stores in the visitor space table corresponding to i and returns to the original processing.

  When the calculation process ends, the determination unit 133 increments a variable i for identifying a visitor, and returns to the determination of whether or not there is an unprocessed visitor. When it is determined that there is no unprocessed visitor, the determination unit 133 stores the generation end time of the visitor space table in the visitor space table. The determination unit 133 refers to the visitor space table of all the target visitors, generates the resource utilization table of the resource utilization information storage unit 124, and returns to the original processing. The resource utilization table to be generated includes a coefficient table (M × N) in which the influence on the number of visitors (N persons) is described based on the service resource (M) of the robot resource 117. The resource utilization table is, for example, a table corresponding to the status of a plurality of visitors, as shown in FIGS. 9 to 13. In addition, although a resource utilization table demonstrated the resource which works for a visitor's vision and hearing in a present Example, it is not limited to this. For example, the resource utilization table may include effects (reach coefficients) on other sense organs such as olfactory sense to cope with odor, tactile sense to cope with wind, etc., taste.

When the resource utilization coefficient calculation process ends, the determination unit 133 executes the resource allocation process. When the resource allocation process is started, the determination unit 133 refers to the resource utilization information storage unit 124 and acquires a resource utilization table corresponding to the number N of visitors and the number M of service resources. The determination unit 133 determines whether or not the service is being provided to some of the visitors N ₁ based on the acquired resource utilization table. That is, the determination unit 133 determines, by referring to the column "allocation" basis of resource utilization table, whether it is part of a visitor N serving during _1. Determining section 133, if it is determined not to be part of the visitor N serving during ₁ allocate customer resource number M with respect to visitors number N. In the distribution, the determination unit 133 calculates the allocation of the number M of customer service resources to N, and thus calculates the allocation of M to the Nth power. The determination unit 133, for each resource of the resource utilization table, for example, adopts allocation of a combination in which the total value of values indicating utility is the highest. The determination unit 133 stores the distributed result in the corresponding resource utilization table of the resource utilization information storage unit 124. After storing the allocation result in the resource utilization table, the determination unit 133 outputs the allocation completion information to the information providing unit 134 and returns to the original processing.

Determination unit 133, when it is determined that the part of the visitor N serving during ₁ determines whether the visitor N ₂ of the visitor N ₁ is not receiving appropriate services. When the judging unit 133 judges that the visitor N ₂ among the visitors N ₁ receives the service properly, the judging unit 133 responds to the group of the visitor N ₁ and the visitor (N-N ₁ ). Distribute the number M of customer service resources. That is, in this case, the visitor N ₂ is a person that has delayed relative visitor N _1, a state in which properly received the visitor N ₁ and the same service. Since the determination unit 133 calculates the allocation of the number M of customer service resources to (1+ (N−N ₁ )) in the distribution, the determination unit 133 calculates an allocation of (1+ (N−N ₁ )) times M. Do. The determination unit 133, for each resource of the resource utilization table, for example, adopts allocation of a combination in which the total value of values indicating utility is the highest. In this case, the determination unit 133 are the in providing services to visitors N _1, also considered when notice visitors N _1. The determination unit 133 stores the distributed result in the corresponding resource utilization table of the resource utilization information storage unit 124. After storing the allocation result in the resource utilization table, the determination unit 133 outputs the allocation completion information to the information providing unit 134 and returns to the original processing.

Determining section 133, when the visitor _{N 2} of the visitor _{N 1} is determined to not receiving proper services, and a group of visitors _(N 1 -N _2), the visitor (N-N _The number M of customer service resources is allocated to ₁ + N ₂ ). That is, in this case, for example, visitor N ₂ is the visitor N ₁ identical to the group, which is different from the existing service, a state in which an interest in, for example, exhibit the like. As the determination unit 133, in the allocation, multiply _{(1+ (N-N 1 +} N 2)) is calculated. Therefore the assignment of the number of customer resources to human M, of _{M (1+ (N-N 1} + N 2)) Calculate the allocation of The determination unit 133, for each resource of the resource utilization table, for example, adopts allocation of a combination in which the total value of values indicating utility is the highest. In this case, since the determination unit 133 is providing a service to the visitor (N ₁ -N ₂ ), the determination unit 133 also considers the case where the visitor (N ₁ -N ₂ ) is noticed. The determination unit 133 stores the distributed result in the corresponding resource utilization table of the resource utilization information storage unit 124. After storing the allocation result in the resource utilization table, the determination unit 133 outputs the allocation completion information to the information providing unit 134 and returns to the original processing.

  In other words, the determination unit 133 determines whether or not the identified first visitor's region of interest (interesting space) is different from the second visitor's region of interest (interesting space). In addition, the determination unit 133 determines whether there is a common region of interest between the first visitor's identified region of interest (interesting space) and the second visitor's region of interest (interesting space).

  In addition, the determination unit 133 identifies a region of interest (space of interest) based on the position of the first visitor or the second visitor, the orientation of the head, and the line of sight. In addition, the determination unit 133 is based on one or more of a reaction response to an action on the first visitor or the second visitor, a dialogue, a relationship with another person, and a contact with the information providing apparatus. And identify a region of interest (space of interest).

  Visitor information is input to the information providing unit 134 from the detection unit 131, and distribution completion information is input from the determination unit 133. When the visitor information is input, the information provision unit 134 distributes all resources of the robot resource 117 to the visitor, since the visitor is one person. When the allocation completion information is input, the information providing unit 134 refers to the resource utilization information storage unit 124, and the robot for a plurality of visitors according to the information on allocation of each resource (allocation after review) Allocate each resource of resource 117. The information provision unit 134 refers to the service scenario storage unit 125, and executes the service scenario based on the resource allocation. Depending on the resource allocation, there may be visitors who do not provide services.

  When the information providing unit 134 executes the service scenario, the information providing unit 134 determines whether a re-determination event has occurred. The re-determination event may include, for example, the elapse of a predetermined time, a change in the state of the visitor such as the visitor leaving, or a change in the state of the customer service robot 100 such as no response to the inquiry to the visitor. If the information providing unit 134 determines that the re-determination event has not occurred, the information providing unit 134 continues to determine the re-determination event.

  If the information providing unit 134 determines that the re-determination event has occurred, the information providing unit 134 determines whether to end the service, for example, by the operation of the administrator of the customer service robot 100 or the like. When it is determined that the service is not ended, the information providing unit 134 outputs an instruction to the detecting unit 131 to determine whether a visitor is in the service providing space. When it is determined that the service is to be ended, the information providing unit 134 ends the provision of the service by the customer service robot 100.

  In other words, when the region of interest of the first visitor and the region of interest of the second visitor are different, the information provision unit 134 has a plurality of information provision resources (robot resources) possessed by the information provision device (service robot 100). An information provision process is performed using 117). That is, the information provision unit 134 executes the information provision process for the first visitor using the information provision resource corresponding to the region of interest of the first visitor among the plurality of information provision resources. In addition, the information providing unit 134 executes an information providing process for the second visitor using the information providing resource according to the region of interest of the second visitor among the plurality of information providing resources.

  In addition, when the information provision resource of each of the plurality of information provision resources executes the information provision process, the information provision unit 134 notices that the first visitor and the second visitor perform the information provision process. Find the probability. The information provision unit 134 distributes the plurality of information provision resources to the first visitor and the second visitor based on the obtained notice probability.

  In addition, the information provision unit 134 changes the allocation of information provision resources when the region of interest of the second visitor is specified while the information provision process for the first visitor is being executed. That is, the information provision unit 134 reduces the allocation of the information provision resource in execution of the information provision process to the first visitor, and increases the allocation of the information provision resource according to the second visitor's region of interest.

  Further, when there is a common region of interest for the first visitor's region of interest and the second visitor's region of interest, the information provision unit 134 uses the information provision resource for the common region of interest as the first information resource. The visitor and the second visitor share the information provision process.

  In addition, the information providing unit 134 determines whether there is a common region of interest for the first visitor's region of interest and the second visitor's region of interest. When there is a common region of interest, the information provision unit 134 uses the information provision resource corresponding to the common region of interest among the plurality of information provision resources possessed by the information provision device, to use the first visitor and the second visitor. Execute information provision processing for visitors.

  Next, the operation of the customer service robot 100 of the embodiment will be described. FIG. 15 is a flowchart showing an example of control processing of the embodiment.

  The detection unit 131 of the customer robot 100 sets the coordinate system and block group of the service providing space 20 (step S1). Next, the detection unit 131 sets shielding information of the service providing space 20 for the shielding disposed in the service providing space 20 (step S2).

  The detection unit 131 acquires sensor information from the external sensor 10. The detection unit 131 determines whether there is a visitor in the service providing space 20 based on the acquired sensor information (step S3). If the detection unit 131 determines that there are no visitors in the service providing space 20 (step S3: negative), the determination in step S3 is repeated.

  When the detecting unit 131 determines that there is a visitor in the service providing space 20 (Step S3: Yes), the detecting unit 131 determines whether there are a plurality of visitors (Step S4). If there are not a plurality of visitors (step S4: negative), the detection unit 131 outputs visitor information indicating that the visitor is a single person to the information providing unit 134, and the process proceeds to step S8. The detecting unit 131 outputs a specifying instruction to the specifying unit 132 when there are a plurality of visitors (Step S4: Yes). The detection unit 131 also outputs detection information to the determination unit 133.

  When the specification instruction is input from the detection unit 131, the specification unit 132 executes a space of interest specification process (step S5). Here, the space of interest identification process will be described with reference to FIG. FIG. 16 is a flowchart illustrating an example of the interest space identification process.

  The identifying unit 132 initializes a variable i for identifying a plurality of visitors. The identifying unit 132 detects the number N of visitors in the service providing space 20 based on the identifying instruction (step S51). When the identification unit 132 detects the number N of visitors, the identification unit 132 generates an interest space table for N persons (step S52). The identifying unit 132 stores the generation start time of the space of interest table in the space of interest table (step S53).

  The identifying unit 132 determines whether there is an unprocessed visitor (step S54). When it is determined that there is an unprocessed visitor (Step S54: Yes), the identifying unit 132 sets the visitor i as the identification target (Step S55). The identifying unit 132 executes a identifying process for the visitor i (step S56).

  Here, the identification processing will be described with reference to FIG. FIG. 17 is a flowchart illustrating an example of the identification process. The identifying unit 132 acquires the position of the visitor i, the direction of the head, and the line of sight based on the captured image included in the identification instruction (step S561). When acquiring the position of the visitor i, the direction of the head, and the line of sight, the identifying unit 132 executes a first estimation process (step S562).

  Here, the first estimation process will be described with reference to FIG. FIG. 18 is a flowchart illustrating an example of the first estimation process. The identifying unit 132 determines whether the position vector of the visitor i, the direction of the head, or the direction vector of the sight line is stationary in the same direction (step S101). When it is determined that the direction vector is not stationary in the same direction (step S101: negative), the identification unit 132 ends the identification processing as non-interest space identification (step S102), and returns to the interest space identification processing.

  When it is determined that the direction vector is stationary in the same direction (Step S101: Yes), the identifying unit 132 is based on the stationary position of the visitor i, the head orientation, and the direction vector of the sight line. Then, the interest flag value is added to the corresponding block group (step S103). The identifying unit 132 refers to the service provision space storage unit 121 and determines whether there is a candidate of interest on the extension of each direction vector (step S104). If the identification unit 132 determines that there is no candidate of interest on the extension (step S104: negative), it ends the identification processing as non-interest identification (step S105), and returns to the interest space identification processing.

  If the identification unit 132 determines that there is a candidate of interest on the extension (step S104: affirmative), the smaller one of the view angle vector space of the visitor i and the space based on the outer edge of the candidate of interest The interest flag value is added to blocks in the space of (step S106). Further, the identifying unit 132 adds the interest flag value to the block group in the vicinity of the interest target candidate (step S107), ends the first estimation processing, and returns to the identification processing.

  Returning to the description of FIG. 17, when the first estimation process ends, the identifying unit 132 acquires operation information of the object of interest candidate and the visitor i (step S563). When acquiring the motion information of the object of interest candidate and the visitor i, the identifying unit 132 executes a second estimation process (step S564).

  Here, the second estimation process will be described with reference to FIG. FIG. 19 is a flowchart illustrating an example of the second estimation process. The identifying unit 132 determines whether the reaction operation timing of the visitor i matches the operation timing from the target candidate based on the acquired operation information (step S111). The identifying unit 132 adds an interest flag value to a group of blocks in the vicinity of the candidate of interest, when the reaction operation timing of the visitor i matches the action timing from the candidate of interest (Step S111: Yes) ( Step S112).

  If the reaction operation timing of the visitor i does not match the operation timing from the candidate of interest (step S111: negative), the identifying unit 132 proceeds to step S113 without adding the interest flag value.

  The identifying unit 132 determines, based on the acquired motion information, whether or not the reaction motion content of the visitor i matches the challenge content from the interest target candidate (step S113). The identifying unit 132 adds the interest flag value to a block group in the vicinity of the candidate of interest when the reaction action content of the visitor i matches the content of the action from the candidate of interest (Step S113: Yes) ( Step S114).

  If the reaction operation content of the visitor i does not match the action content from the interest target candidate (step S113: negative), the identification unit 132 proceeds to step S115 without adding the interest flag value.

  The identifying unit 132 acquires the information presentation log of the object of interest candidate and the operation log of the visitor i (step S115). The identifying unit 132 determines whether the operation timing of the visitor i matches the information presentation timing from the interest target candidate based on the acquired information presentation log and the operation log (step S116). When the operation timing of the visitor i matches the information presentation timing from the candidate of interest (Step S116: Yes), the identifying unit 132 adds the interest flag value to the group of blocks near the candidate of interest (A). Step S117).

  If the operation timing of the visitor i does not match the information presentation timing from the candidate of interest (step S116: negative), the identifying unit 132 proceeds to step S118 without adding the interest flag value.

  The identifying unit 132 determines whether the operation content of the visitor i matches the information presentation content from the interest target candidate based on the acquired information presentation log and the operation log (step S118). When the operation content of the visitor i matches the information presentation content from the target of interest (Step S118: Yes), the identifying unit 132 adds the interest flag value to the block group in the vicinity of the target of interest Step S119), return to the original process.

  If the operation content of the visitor i does not match the information presentation content from the interest target candidate (step S118: negative), the identification unit 132 returns to the original processing without adding the interest flag value.

  Returning to the description of FIG. 17, when the second estimation process ends, the identifying unit 132 acquires dialogue information of the candidate of interest and the visitor i (step S 565). When acquiring the dialogue information of the candidate of interest and the visitor i, the identifying unit 132 executes the third estimation process (step S566).

  Here, the third estimation process will be described with reference to FIG. FIG. 20 is a flowchart illustrating an example of the third estimation process. The identifying unit 132 determines whether the dialog timing of the visitor i matches the dialog timing from the candidate of interest based on the acquired dialog information (step S121). The identifying unit 132 adds the interest flag value to a block group in the vicinity of the candidate of interest when the timing of interaction of the visitor i matches the timing of interaction from the candidate of interest (Step S121: affirmative) S122).

  If the dialogue timing of the visitor i does not match the dialogue timing from the candidate of interest (step S121: negative), the identifying unit 132 proceeds to step S123 without adding the interest flag value.

  The identifying unit 132 determines whether the dialog information of the visitor i matches the dialog information from the candidate of interest based on the acquired dialog information (step S123). When the dialogue information of the visitor i matches the dialogue information from the candidate of interest (Step S123: Yes), the identifying unit 132 adds the interest flag value to the block group in the vicinity of the candidate of interest (Step S124), return to the original processing.

  If the dialogue information of the visitor i does not match the dialogue information from the candidate of interest (step S123: negative), the identifying unit 132 returns to the original process without adding the interest flag value.

  Returning to the explanation of FIG. 17, when the third estimation process ends, the identifying unit 132 acquires action information of a person of interest object candidate and a visitor i (step S 567). When acquiring the behavior information of the candidate of interest and the visitor i who are persons, the identifying unit 132 executes a fourth estimation process (step S568).

  Here, the fourth estimation process will be described with reference to FIG. FIG. 21 is a flowchart illustrating an example of the fourth estimation process. The identifying unit 132 determines, based on the acquired behavior information, whether the behavior of the visitor i and the candidate of interest is continuously synchronized in a predetermined space for a predetermined period (step S131). When the identification unit 132 determines that the behavior of the visitor i and the candidate of interest is continuously synchronized in the predetermined space for a predetermined period (Yes at step S131), the visitor i and the candidate of interest The interest flag value is added to the blocks in the space in between (step S132), and the process returns to the original process.

  If the identification unit 132 determines that the behavior of the visitor i and the candidate of interest is not synchronized in the predetermined space continuously for a predetermined period (step S131: negative), the interest flag value is not added. Return to the original process.

  Returning to the description of FIG. 17, when the fourth estimation process ends, the identifying unit 132 acquires contact information on a target of interest (step S569). When acquiring the contact information on the target of interest, the identifying unit 132 executes a fifth estimation process (step S570).

  Here, the fifth estimation process will be described with reference to FIG. FIG. 22 is a flowchart illustrating an example of the fifth estimation process. The identifying unit 132 determines whether the contact between the visitor i and the candidate of interest continues for a predetermined period based on the acquired contact information (step S141). When the identifying unit 132 determines that the contact between the visitor i and the candidate of interest continues for a predetermined period (Step S141: Yes), the interest flag value is set to blocks near the contact portion of the candidate of interest. Are added (step S142), and the process returns to the original process.

  When the identification unit 132 determines that the contact between the visitor i and the candidate of interest does not continue for a predetermined period (step S141: negative), the process returns to the original process without adding the interest flag value.

  Returning to the description of FIG. 17, when the fifth estimation process ends, the identification unit 132 sets the interest space flag in each block of the interest space table in the interest space information storage unit 122 to a block whose interest flag value is equal to or higher than the threshold. (Step S571). When the setting of the interest space flag is completed, the identification unit 132 ends the identification processing and returns to the interest space identification processing.

  Returning to the description of FIG. 16, when the identification processing ends, the identification unit 132 increments a variable i for identifying a visitor (step S57), and returns to step S54. When it is determined that there is no unprocessed visitor (No at Step S54), the identifying unit 132 stores the generation end time of the space of interest table (Step S58), and ends the space of interest identification process. When the identification unit 132 ends the interest space identification process, the identification unit 132 outputs a calculation instruction to the determination unit 133.

  Returning to the explanation of FIG. 15, when the calculation instruction is input from the specification unit 132, the determination unit 133 executes resource utilization coefficient calculation processing (step S6).

  Here, resource utilization coefficient calculation processing will be described with reference to FIG. FIG. 23 is a flowchart showing an example of the resource utilization coefficient calculation process. The determination unit 133 detects the number N of visitors in the service providing space 20 based on the detection information (step S61). The determination unit 133 generates a visitor space table for N people in the visitor space information storage unit 123 based on the detected number N of visitors (step S62). The determination unit 133 stores the generation start time of the visitor space table (step S63).

  The determination unit 133 determines whether there is an unprocessed visitor (step S64). If the determination unit 133 determines that there is an unprocessed visitor (Step S64: affirmative), the determination unit 133 sets the visitor i as a calculation target (Step S65). The determination unit 133 executes calculation processing for the visitor i (step S66).

  Here, the calculation process will be described with reference to FIG. FIG. 24 is a flowchart showing an example of the calculation process. The determination unit 133 acquires the positions of the eyes and ears of the visitor i based on the captured image included in the detection information (step S661). The determination unit 133 refers to the space-of-interest information storage unit 122 based on the acquired positions of the eyes and ears, and acquires the space of interest of the visitor i (step S662). The determination unit 133 estimates the parallax and the sound pressure level in the space around the visitor i based on the acquired space of interest of the visitor i (step S663). The determination unit 133 stores the parallax and the sound pressure level in the space around the visitor i in the visitor space information storage unit 123.

  The determination unit 133 refers to the service providing space storage unit 121 and the information of the movable range of each customer service resource of the robot resource 117 set in advance, and calculates the operation space of each customer service resource (step S664). Based on the calculated operation space of each customer service resource and the visitor space table corresponding to the visitor i, the determination unit 133 determines the parallax due to the operation of each customer resource in the operation space of the visitor i at the eye position. Calculate (step S665).

  The determination unit 133 refers to the information of the sound source of each customer service resource of the robot resource 117 set in advance, and calculates the sound pressure level of the sound emitted by each customer service resource (step S666). The determination unit 133 calculates the sound pressure level of the sound of each customer service resource at the position of the visitor i's ear based on the calculated sound pressure level and the visitor space table corresponding to the visitor i (step S667) ).

  The determination unit 133 compares the parallax and sound pressure level at the positions of the eyes and ears of the visitor i with the parallax and sound pressure level in the space around the visitor i, and the visitor i operates the respective customer resources and For the sound, the probability of noticing and the probability of noticing are calculated (step S668). The determination unit 133 stores the calculation result in the visitor space table (step S669), and returns to the original process.

  Returning to the explanation of FIG. 23, when the calculation process is completed, the determination unit 133 increments a variable i for identifying a visitor (step S67), and returns to step S64. When it is determined that there is no unprocessed visitor (No at Step S64), the determination unit 133 stores the generation end time of the visitor space table (Step S68). The determination unit 133 refers to the visitor space table of all the targeted visitors, generates the resource utilization table of the resource utilization information storage unit 124 (step S69), and returns to the original processing.

  Returning to the explanation of FIG. 15, when the resource utilization coefficient calculation process ends, the determination unit 133 executes the resource allocation process (step S7). Here, resource allocation processing will be described with reference to FIG. FIG. 25 is a flowchart illustrating an example of the resource allocation process. The determination unit 133 refers to the resource utilization information storage unit 124, and acquires a resource utilization table corresponding to the number N of visitors and the number M of customer service resources (step S71).

The determination unit 133 determines whether or not the service is being provided to some of the visitors N ₁ based on the acquired resource utilization table (step S72). Determining section 133, when it is determined that the part of the visitor N ₁ is not in service provision (step S72: No), allocate customer resource number M with respect to visitors number N (step S73). The determination unit 133 stores the distributed result in the resource utilization table, outputs the distribution completion information to the information providing unit 134, and returns to the original process.

If the determination unit 133 determines that the service is being provided to some of the visitors N ₁ (Step S 72: Yes), is the visitor N ₂ not properly receiving the service among the visitors N ₁ ? It is determined whether or not it is (step S74). Determining section 133, when it is determined that the visitor _{N 2} of the visitor _{N 1} is under appropriate service (step S74: No), 1 and group visitors _{N 1,} visitor (N- The number M of customer service resources is allocated to N ₁ ) (step S75). The determination unit 133 stores the distributed result in the resource utilization table, outputs the distribution completion information to the information providing unit 134, and returns to the original process.

If the determination unit 133 determines that the visitor N ₂ is not properly receiving the service among the visitor N ₁ (Step S 74: affirmative), one group of the visitors (N ₁ -N ₂ ), against the visitor _{(N-N 1 + N 2} ), to allocate the customer resource count M (step S76). The determination unit 133 stores the distributed result in the resource utilization table, outputs the distribution completion information to the information providing unit 134, and returns to the original process.

  Returning to the explanation of FIG. 15, when the visitor information is input, the information providing unit 134 allocates all resources of the robot resource 117 to the visitor. When the allocation completion information is input, the information providing unit 134 refers to the resource utilization information storage unit 124, and each resource of the robot resource 117 is transmitted to a plurality of visitors according to the information on the allocation of each resource. Distribute. The information provision unit 134 refers to the service scenario storage unit 125, and executes the service scenario based on the resource allocation (step S8).

  After executing the service scenario, the information providing unit 134 determines whether a re-determination event has occurred (step S9). If the information providing unit 134 determines that the re-determination event has not occurred (No at Step S9), the determination at Step S9 is repeated. If the information providing unit 134 determines that a re-determination event has occurred (Yes at Step S9), the information providing unit 134 determines whether to end the service (Step S10).

  When it is determined that the service is not ended (Step S10: No), the information providing unit 134 outputs an instruction to the detection unit 131 to perform the determination of Step S3, and the process returns to Step S3. When it is determined that the service is to be ended (Step S10: Yes), the information provision unit 134 ends the provision of the service by the customer-service robot 100. As a result, the customer service robot 100 can improve the service efficiency for a plurality of visitors. In addition, the service robot 100 lowers the quality of service to the extent that the first visitor (A) is less likely to be dissatisfied, secures the resources, and secures the second visitor (B) in the range where the effect is easily obtained. The above resources are diverted to serving the second visitor (B), so that the quality of serving can be maintained in total. Also, the customer robot 100 can improve the interaction failure of the visitor.

  Subsequently, a specific example of the customer service will be described with reference to FIGS.

  FIG. 26 is a diagram showing a specific example of customer service for a plurality of visitors. As shown in FIG. 26, for example, the customer service robot 100 is installed at a booth of the exhibition hall and receives the visitor A. When receiving the visitor A, the customer robot 100 guides the visitor A to a location. The customer service robot 100, for example, guides the visitor A to a position where the screen 25 on which the presentation screen is displayed can be easily viewed. The customer robot 100 provides the visitor A with the core value, that is, starts the explanation using the presentation screen displayed on the screen 25. At this time, it is assumed that the visitor B appears behind the booth and keeps looking at the customer service robot 100. Then, the customer service robot 100 detects the appearance of the visitor B, and determines that the space of interest of the visitor B is directed to itself.

  Further, the customer robot 100 detects a state in which the visitor A faces the screen 25 and does not look at the customer robot 100. That is, the customer robot 100 determines that the space of interest of the visitor A is fixed to the screen 25 for a long time. In order to provide services to the visitors A and B in parallel, the customer service robot 100 determines whether the mutual incoherence of the respective interest spaces of the visitors A and B is high or not. That is, since the visitor A looks at the screen 25 and the visitor B looks at the customer robot 100, the customer robot 100 determines that the mutual incoherence is high. That is, the customer service robot 100 determines that each resource of the robot resource 117 can be distributed to the visitors A and B.

  The customer service robot 100 allocates each resource of the robot resource 117 so as to increase the service effect for each of the visitors A and B. The robot 100 distributes the right arm 113 and the dialogue 115 to the visitor A, and distributes the head 111, the left arm 114, and the display 116 to the visitor B. The customer service robot 100 executes a plurality of services in parallel according to the allocation. At this time, the customer-service robot 100 enhances the controllability of the presentation given on the screen 25. That is, the customer service robot 100 secures time to perform the service in parallel by advancing the moving image service in the presentation and setting the question acceptance behind. The customer service robot 100 points the screen 25 to the visitor A using the right arm 113 and explains with voice while pointing the head 111 to the visitor B and waving the left arm 114, and displays the advance notice information 26 on the display unit 116. indicate. By distributing each resource of the robot resource 117 in this manner, the customer service robot 100 can service both the visitor A who sees the moving image and the visitor B who is waiting for the response, thereby improving the service efficiency. it can.

  FIGS. 27 to 30 are diagrams showing specific examples of service scenarios. 27 to 30 show an example of a service scenario in the specific example of the presentation service shown in FIG. The service scenario of FIG. 27 to FIG. 30 is stored in the service scenario storage unit 125 in the example of FIG. In the service scenario, for example, seven more sub-phases are set for the phase "providing core value" for presentation. In the service scenario, resource allocation can be performed in sub-phase units, thereby increasing the opportunity to provide services to a plurality of visitors (visitors A and B) in parallel.

  FIG. 31 is a diagram showing a specific example of customer service for a plurality of visitors. As shown in FIG. 31, for example, the customer-service robot 100 a is installed at the window of the F city hall and receives the visitor A. When receiving the visitor A, the customer-service robot 100a guides the visitor A to a location. The customer service robot 100a, for example, guides the visitor A to a desk on which a robot resource 117 capable of completing the application form can be installed. The customer service robot 100a provides the core value to the visitor A, that is, displays the explanation 27 for the application form and starts the filling instruction. At this time, it is assumed that the visitor B appears behind the booth and stands in front of the customer service robot 100a. Then, the customer robot 100a detects the appearance of the visitor B, and determines that the space of interest of the visitor B is directed to itself.

  Further, the customer robot 100a detects a state in which the visitor A gazes at the desk and does not look at the customer robot 100a. That is, the robot 100a determines that the space of interest of the visitor A is fixed on the desk for a long time. In order to provide services to the visitors A and B in parallel, the customer service robot 100 a determines whether the mutual incoherence of the respective interest spaces of the visitors A and B is high or not. That is, since the visitor A looks at the desk and the visitor B looks at the customer robot 100a, the customer robot 100a determines that the mutual incoherence is high. That is, the customer service robot 100a determines that each resource of the robot resource 117 can be distributed to the visitors A and B. In addition, in order to confirm the interest space of the visitor A, the customer-service robot 100a may perform a waving motion using the left arm 114 on the visitor A side.

  The customer service robot 100a redistributes each resource of the robot resource 117 so as to increase the service effect for each of the visitors A and B. The robot 100a distributes the left arm 114 to the visitor A, and distributes the head 111, the right arm 113, and the dialogue 115 to the visitor B. The customer service robot 100a executes a plurality of services in parallel according to the allocation. At this time, the customer-service robot 100a enhances the controllability of the writing instruction given on the desk. That is, the customer-service robot 100a secures a time for performing the service in parallel by advancing the self-confirmation request of the entry content in the entry instruction and advancing the question acceptance afterward. The robot 100a points to the visitor A with the left arm 114, bows to the visitor B with the head 111 facing, and uses the right arm 113 to "visit" the visitor B with another desk after the "reception utterance". Instruct the applicant to complete the application form. By distributing each resource of the robot resource 117 in this manner, the customer service robot 100a can service both the visitor A who is beginning to fill in the application form and the visitor B who comes later, Service efficiency can be improved.

  32 to 36 illustrate specific examples of service scenarios. FIGS. 32 to 36 illustrate an example of a service scenario in the specific example of the application support service shown in FIG. The service scenarios of FIGS. 32 to 36 are stored in the service scenario storage unit 125 in the example of FIG. In the service scenario, for example, seven more sub-phases are set for the “filling core value” phase for providing application filling support. In the service scenario, resource allocation can be performed in sub-phase units, thereby increasing the opportunity to provide services to a plurality of visitors (visitors A and B) in parallel.

FIG. 37 is a diagram showing a specific example of customer service for a plurality of visitors. As shown in FIG. 37, the customer service robot 100b is installed, for example, in an aquarium or the like, and provides a guidance service to a plurality of visitors. First, for example, and at the same time visitor A on the left and right of the customer robot 100b at a certain time _{t 0,} and B appeared. At this time, when the customer service robot 100b detects the visitors A and B, for example, the service robot 100b refers to the resource utilization table 124e shown in FIG. The right eye of the eye 113 and the eye 112 is distributed. Further, the customer robot 100b distributes the left eye portion of the left arm portion 114 and the eye portion 112 to the visitor B who appears from the right side toward the customer robot 100b.

  In other words, if the head 111 of the customer robot 100b is directed to either the left or the right visitor, the opposite visitor is likely to notice (as not being the other party), but the eyes are left and right separately And B, it is difficult for the other visitor to notice. In addition, when the dialogue unit 115 (for example, the mouth) or the display unit 116 is directed to either the left or the right visitor, the visitor on the opposite side is also easily noticed (as a state in which the other party is not the other party).

  The customer service robot 100b moves each resource of the robot resource 117 according to the allocation, and guides the visitors A and B in the front direction of themselves by hand gesture, eye contact, or the like. The customer robot 100b starts an information display service which is a core value for the visitors A and B guided in the front direction of the robot. By distributing the resources of the robot resource 117 in this manner, the customer service robot 100b can guide the visitors A and B that have simultaneously appeared, and can improve the service efficiency.

  FIG. 38 is a diagram showing an example of guiding a plurality of children. As shown in FIG. 38, in a certain kindergarten, it is assumed that a customer service robot 100c is introduced. The children C and D play at a place far from the customer service robot 100c. The children E and F play near the customer service robot 100c. When the predetermined time is reached, the customer robot 100c guides the children C to F in the direction of the story teacher 32.

  The customer service robot 100 c confirms the possibility of the hand gesture reach 28, 29 to the four children C to F and the sight reach 30, 31. The customer service robot 100c distributes the right arm 113 to the children C and D, and distributes the left arm 114 and the display 116 to the children E and F based on the confirmation result. In addition, since the dialogue unit 115 utters the guidance message to the children C to F, the dialogue unit 115 allocates the message to both.

  The customer service robot 100c moves the right arm 113 according to the allocation, and utters a guidance message using the dialogue unit 115 to guide the children C and D in the direction of the story teacher 32. Further, the customer-service robot 100c moves the left arm 114 according to the distribution, displays the guidance arrow 33 on the display unit 116, utters a guidance message using the dialogue unit 115, and causes the children E and F to Guide in the direction of the story teacher 32. By distributing the resources of the robot resource 117 in this manner, the customer service robot 100c can execute tasks with different designation destinations in parallel, even for services of the same type, that is, direction indication, and the customer service efficiency can be improved. It can be improved.

  As described above, when the customer robot 100 detects the first visitor and the second visitor, the customer robot 100 detects the first visitor and the second visitor by the captured image or the sensor in which the first visitor and the second visitor are imaged. Based on the orientation of any specific part of the first visitor and the orientation of any part of the second visitor, the respective regions of interest of the first visitor and the second visitor are Identify. Further, the customer service robot 100 determines whether the identified first visitor's region of interest is different from the second visitor's region of interest. In addition, when the service area of the first visitor is different from the area of interest of the second visitor, the customer robot 100 is interested in the first visitor among the plurality of information provision resources possessed by the information providing apparatus. The information provision process according to the area is performed to the first visitor using the information provision resource. Further, the customer robot 100 executes the information provision processing for the second visitor using the information provision resource corresponding to the region of interest of the second visitor among the plurality of information provision resources. As a result, the customer service robot 100 can improve the service efficiency for a plurality of visitors.

  In addition, when the information provision resource of each of the plurality of information provision resources executes the information provision process, the customer robot 100 may notice that the first visitor and the second visitor notice the execution of the information provision process. Ask for Further, the customer service robot 100 distributes a plurality of information provision resources to the first visitor and the second visitor based on the obtained notice probability. As a result, the customer service robot 100 lowers the quality of customer service within a range in which the first visitor is less likely to be dissatisfied, secures the resource, and secures the resource within the range where the second visitor can easily obtain the second effect. Since it is turned into customer service to visitors, it is possible to maintain the quality of customer service in total.

  Also, when the service robot 100 executes the process of providing information to the first visitor, if the region of interest of the second visitor is specified, the distribution of information resources is changed. That is, the customer-service robot 100 reduces the allocation of the information provision resource in execution of the information provision process for the first visitor, and increases the allocation of the information provision resource according to the second visitor's region of interest. As a result, the customer service robot 100 lowers the quality of customer service within a range in which the first visitor is less likely to be dissatisfied, secures the resource, and secures the resource within the range where the second visitor can easily obtain the second effect. Since it is turned into customer service to visitors, it is possible to maintain the quality of customer service in total.

  Further, the customer robot 100 specifies the region of interest based on the position of the first visitor or the second visitor, the orientation of the head, and the line of sight. As a result, the customer robot 100 can provide a service according to the region of interest (space of interest).

  In addition, the customer service robot 100 is based on one or more of a reaction response to an action on the first visitor or the second visitor, a dialogue, a relationship with another person, and a contact to the information providing apparatus. Identify the region of interest. As a result, the customer robot 100 can provide a service according to the region of interest (space of interest).

  Also, if there is a region of interest common to the region of interest of the first visitor and the region of interest of the second visitor, the customer robot 100 visits the information providing resource for the common region of interest as the first visit. Sharing information between the person and the second visitor to execute the information providing process. As a result, the customer service robot 100 can improve the service efficiency for a plurality of visitors.

  Further, when the customer robot 100 detects the first visitor and the second visitor, the customer robot 100 detects the first visitor and the second visitor by a captured image or a sensor in which the first visitor and the second visitor are imaged. Identify the respective regions of interest of the first visitor and the second visitor based on the orientation of any particular region of the first visitor and the orientation of any region of the second visitor . Further, the customer service robot 100 determines whether there is a common region of interest for the first visitor's identified region of interest and the second visitor's region of interest. Furthermore, when there is a common region of interest, the customer robot 100 uses the information provision resource corresponding to the common region of interest among the plurality of information provision resources possessed by the information provision device, and the first visitor and the second visitor Execute information provision processing for the visitors of As a result, the customer service robot 100 can improve the service efficiency for a plurality of visitors.

  In addition, although the external sensor 10 was used for the detection of a visitor in the said Example, it is not limited to this. For example, the customer robot 100 may be provided with a camera, an infrared sensor, an ultrasonic sensor, a radar, a LIDAR, a voice localization microphone or the like to detect a visitor. In addition, a visitor's interest space or the like may be detected using these various sensors.

  Moreover, in the said Example, although the region of interest of each of a 1st visitor and a 2nd visitor was specified using the captured image, it is not limited to this. For example, the first visitor and the second visitor based on the direction of any one specific part of the first visitor detected by various sensors and the direction of any part of the second visitor. Each region of interest may be identified. Here, the direction of the visitor, for example, has the visitor wear a headset incorporating a GPS receiver and an acceleration sensor, and the visitor is visited based on the GPS position information and the integration of the acceleration information. The position of the person and the direction of the head can be identified. In addition, for the direction of the visitor, for example, by placing a pressing mat on the floor and detecting which direction the foot form of the visitor is facing, specify the position of the visitor and the direction of the foot form Can. Further, the orientation of the visitor can be determined by, for example, having the visitor wear a headset having a built-in direction sensor and combining it with the pressing mat to specify the visitor's position, foot type and head direction. it can.

  Further, each component of each unit shown in the drawings does not necessarily have to be physically configured as shown in the drawings. That is, the specific form of the dispersion and integration of each part is not limited to the illustrated one, and all or a part thereof is functionally or physically dispersed or integrated in any unit according to various loads, usage conditions, etc. Can be configured. For example, the identification unit 132 and the determination unit 133 may be integrated. Further, the illustrated processes are not limited to the above-described order, and may be performed simultaneously within the range in which the process contents do not contradict each other, or may be performed by changing the order.

  Furthermore, all or any part of various processing functions performed by each device may be executed on a CPU (or a microcomputer such as an MPU or an MCU (Micro Controller Unit)). In addition, various processing functions may be executed in whole or any part on a program analyzed and executed by a CPU (or a microcomputer such as an MPU or MCU) or on hardware by wired logic. It goes without saying that it is good.

  The various processes described in the above embodiments can be realized by executing a prepared program on a computer. So, below, an example of a computer which runs a program which has the same function as each above-mentioned example is explained. FIG. 39 is a diagram illustrating an example of a computer that executes a control program.

  As shown in FIG. 39, the computer 200 has a CPU 201 that executes various types of arithmetic processing, an input device 202 that receives data input, and a monitor 203. In addition, the computer 200 is connected by wire or wirelessly to the medium reading device 204 which reads a program or the like from a storage medium, the interface device 205 for connecting to various devices, the external sensor 10, and other information processing devices. And the communication device 206 of The computer 200 also has a RAM 207 for temporarily storing various information, and a hard disk drive 208. Each of the devices 201 to 208 is connected to the bus 209.

  The hard disk drive 208 stores control programs having the same functions as the processing units of the detection unit 131, the identification unit 132, the determination unit 133, and the information providing unit 134 illustrated in FIG. In the hard disk drive 208, the service providing space storage unit 121, the interest space information storage unit 122, the visitor space information storage unit 123, the resource utilization information storage unit 124, the service scenario storage unit 125, and the control program are realized. Various data for storing are stored. The input device 202 receives, for example, an input of various information such as operation information from a visitor to which the information providing device controlled by the computer 200 corresponds. The monitor 203 displays, for example, various screens such as a display screen to the visitor corresponding to the information providing apparatus controlled by the computer 200. The interface device 205 is connected to, for example, an information providing device and a printing device. The communication device 206 has, for example, the same function as the communication unit 110 shown in FIG. 2 and is connected to the external sensor 10 and a network not shown, and exchanges various information with the external sensor 10 and other information processing devices not shown Do.

  The CPU 201 reads out each program stored in the hard disk device 208, develops the program in the RAM 207, and executes the program to perform various processes. Also, these programs can cause the computer 200 to function as the detection unit 131, the identification unit 132, the determination unit 133, and the information providing unit 134 illustrated in FIG.

  The above control program does not necessarily have to be stored in the hard disk drive 208. For example, the computer 200 may read out and execute a program stored in a storage medium readable by the computer 200. The storage medium readable by the computer 200 corresponds to, for example, a CD-ROM, a DVD disk, a portable recording medium such as a USB (Universal Serial Bus) memory, a semiconductor memory such as a flash memory, a hard disk drive, or the like. Alternatively, the control program may be stored in a device connected to a public line, the Internet, a LAN or the like, and the computer 200 may read out the control program from these and execute it.

10 external sensor 100 customer service robot 110 communication unit 111 head 112 eye unit 113 right arm unit 114 left arm unit 115 dialogue unit 116 display unit 117 robot resource 120 storage unit 121 service providing space storage unit 122 interest space information storage unit 123 visitor space information Storage unit 124 Resource utilization information storage unit 125 Service scenario storage unit 130 Control unit 131 Detection unit 132 Identification unit 133 Determination unit 134 Information provision unit

Claims (11)

When the first visitor and the second visitor are detected, a captured image in which the first visitor and the second visitor are imaged, or the first visit detected by a sensor Identifying the respective regions of interest of the first visitor and the second visitor based on the orientation of any part of the person and the orientation of any part of the second visitor,
It is determined whether or not the identified first visitor's region of interest and the second visitor's region of interest are different.
In the case where the region of interest of the first visitor and the region of interest of the second visitor differ, according to the region of interest of the first visitor among a plurality of information provision resources possessed by the information provision apparatus The information providing process is performed using the information providing resource, and the information providing process according to the region of interest of the second visitor among the plurality of information providing resources is performed using the information providing resource. Execute information provision processing for the second visitor,
A control program causing a computer to execute a process. In the process of executing the information provision process, the first visitor and the second visitor provide the information when the information provision resource of each of the plurality of information provision resources executes the information provision process. Allocating the plurality of information provision resources to the first visitor and the second visitor based on the probability of noticing the execution of the process;
The control program according to claim 1, characterized in that: The process for executing the information provision process is the process for providing information to the first visitor when the region of interest of the second visitor is specified while the information provision process for the first visitor is performed. Reduce the allocation of informational resources in progress to increase the allocation of informational resources according to the area of interest of the second visitor,
The control program according to claim 1 or 2, characterized in that: The identifying process identifies the region of interest based on the position of the first visitor or the second visitor, the head orientation, and the line of sight.
The control program as described in any one of the Claims 1-3 characterized by the above-mentioned. The identifying process may be one or more of a reaction to an action on the first visitor or the second visitor, an interaction, a relationship with another person, or a contact with the information providing apparatus. Identifying the region of interest based on
The control program according to claim 4, characterized in that: When the processing for executing the information provision processing has a common region of interest for the first visitor's region of interest and the second visitor's region of interest, the information provision resource for the common region of interest is The information providing process is executed by sharing the first visitor and the second visitor;
The control program according to claim 1, characterized in that: When the first visitor and the second visitor are detected, a captured image in which the first visitor and the second visitor are imaged, or the first visit detected by a sensor Identifying the respective regions of interest of the first visitor and the second visitor based on the orientation of any part of the person and the orientation of any part of the second visitor,
It is determined whether there is an area of interest common to the identified area of interest of the first visitor and the area of interest of the second visitor,
When there is a common region of interest for the first visitor's region of interest and the second visitor's region of interest, the common region of interest among the plurality of information provision resources possessed by the information provision apparatus Executing an information provision process for the first visitor and the second visitor using the specified information provision resource;
A control program causing a computer to execute a process. When the first visitor and the second visitor are detected, a captured image in which the first visitor and the second visitor are imaged, or the first visit detected by a sensor Identifying the respective regions of interest of the first visitor and the second visitor based on the orientation of any part of the person and the orientation of any part of the second visitor,
It is determined whether or not the identified first visitor's region of interest and the second visitor's region of interest are different.
In the case where the region of interest of the first visitor and the region of interest of the second visitor differ, according to the region of interest of the first visitor among a plurality of information provision resources possessed by the information provision apparatus The information providing process is performed using the information providing resource, and the information providing process according to the region of interest of the second visitor among the plurality of information providing resources is performed using the information providing resource. Execute information provision processing for the second visitor,
A control method, characterized in that a computer executes a process. When the first visitor and the second visitor are detected, a captured image in which the first visitor and the second visitor are imaged, or the first visit detected by a sensor Identifying the respective regions of interest of the first visitor and the second visitor based on the orientation of any part of the person and the orientation of any part of the second visitor,
It is determined whether there is an area of interest common to the identified area of interest of the first visitor and the area of interest of the second visitor,
When there is a common region of interest for the first visitor's region of interest and the second visitor's region of interest, the common region of interest among the plurality of information provision resources possessed by the information provision apparatus Executing an information provision process for the first visitor and the second visitor using the specified information provision resource;
A control method, characterized in that a computer executes a process. When the first visitor and the second visitor are detected, a captured image in which the first visitor and the second visitor are imaged, or the first visit detected by a sensor Specifying the respective regions of interest of the first visitor and the second visitor based on the orientation of any part of the person and the orientation of any part of the second visitor Department,
A determination unit that determines whether the identified first visitor's region of interest is different from the second visitor's region of interest;
In the case where the region of interest of the first visitor and the region of interest of the second visitor differ, according to the region of interest of the first visitor among a plurality of information provision resources possessed by the information provision apparatus The information providing process is performed using the information providing resource, and the information providing process according to the region of interest of the second visitor among the plurality of information providing resources is performed using the information providing resource. An information providing unit that executes an information providing process for a second visitor;
A control device characterized by having. When the first visitor and the second visitor are detected, a captured image in which the first visitor and the second visitor are imaged, or the first visit detected by a sensor Specifying the respective regions of interest of the first visitor and the second visitor based on the orientation of any part of the person and the orientation of any part of the second visitor Department,
A determination unit that determines whether there is a region of interest common to the identified region of interest of the first visitor and the region of interest of the second visitor;
When there is a common region of interest for the first visitor's region of interest and the second visitor's region of interest, the common region of interest among the plurality of information provision resources possessed by the information provision apparatus An information providing unit that executes an information providing process for the first visitor and the second visitor using the specified information providing resource;
A control device characterized by having.

Images