JP2019128557A - Information output device, method, and program - Google Patents

Abstract

To control the output of information on the basis of the distance to a pedestrian as well as the face direction of the pedestrian and determine whether or not a call to the pedestrian by output of information is successful.SOLUTION: Face direction data pertaining to one or more pedestrians is chronologically acquired from a face recognition sensor 2 under control of a face direction data acquisition unit 111, and distance data for the distance between an information output device 1 and the one or more pedestrians is acquired from a distance measurement sensor 3 under control of a distance data acquisition unit 112. Thereafter, a gaze determination process and a proximity determination process are executed on the basis of the face direction data and the distance data under control of a gaze determination unit 114 and a proximity determination unit 115. Output control of image information or voice information is exercised on the basis of the determination results of the gaze determination and proximity determination processes under control of an output control unit 116. Thereafter, determination is made under control of a call success determination unit 117 as to whether or not a call to the pedestrians by output of the image or voice information was successful.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an information output device, method, and program for outputting information to a passerby.

  Conventionally, a movement measurement of a passerby is calculated using a distance measurement sensor, and a customer having an ability to operate to affect the real world based on the movement movement (hereinafter simply referred to as an agent) Research has been done to In such research, when passersby did not approach the agent, passersby passing while looking at the agent, and those passing by without looking at the agent were equally excluded from attracting customers (for example, , Non-Patent Document 1).

Niklas Bergstrom, et al., “Modeling of Natural Human-Robot Encounters”, IEEE / RSJ International Conference on Intelligent Robots and Systems, 2008

  However, in the technology described in Non-Patent Document 1, a passerby who is not attracting customers and who looks at the agent without approaching the agent is likely to be interested in the agent. Therefore, if the customer-collecting operation is executed for such a passer-by, the probability that the call by the customer-collecting operation is successful may be high. Also, for example, a person who passes by the agent by chance without looking at the agent is likely to be attracted by the technique as described in Non-Patent Document 1, but there is a high possibility that the agent is not particularly interested. Therefore, even if the customer attraction operation is executed for such a passer-by, the probability that the call by the customer attraction operation will be successful may be low.

  That is, in the technique described in Non-Patent Document 1 for determining whether or not an agent performs a customer attraction operation based only on data measured by a distance measuring sensor, a passerby who is interested in the agent However, there may be a situation in which the attracting operation is performed for a passerby who is not interested in the agent without performing the attracting operation. Therefore, it is preferable to use other determination criteria in addition to the distance between the agent and the passerby for determining whether or not to perform the customer attraction operation.

  Further, the technique described in Non-Patent Document 1 has the following problem.

  For example, the agent's customer attracting operation is continued even if the call by the customer attracting operation is not successful and there are no passersby around. This wastes agent power.

  Furthermore, for example, when the sound environment around the agent changes, the sound output from the agent may be drowned out by the surrounding noise and may not reach the passerby. As described above, when the environment in which the agent is installed is changed, even if the agent is performing the customer collection operation under the same conditions, the probability that the call by the customer collection operation is successful may decrease. In the technique described in Non-Patent Document 1, even if such a situation occurs, the agent repeatedly executes the customer-collection operation, which makes it difficult for the call to succeed, under the same conditions.

  For these problems, measures may be taken by detecting the success or failure of the call due to the attracting customers.

  The present invention has been made focusing on the above circumstances, and the purpose of the invention is to control the output of information based on the distance between the passing person and the face direction of the passing person, and to output the information It is an object to provide an information output device, method, and program capable of determining success or failure of a call to a passerby.

  In order to solve the above-described problem, the first aspect of the present invention provides image information, audio information, and drive control information for driving an object to the passer-by in response to detection of the passer-by. A face direction data acquisition unit that acquires face direction data relating to the passer by time from the face recognition sensor, and the information from the distance measurement sensor. A distance data acquisition unit for acquiring distance data between an output device and the passer with time and a predetermined range in which the face of the passer includes the information output device based on the acquired face direction data The passerby is within a predetermined distance from the information output device based on the gaze determination unit that determines whether or not the time facing is equal to or longer than the first time and the acquired distance data The time is more than the second time Whether or not to output at least one of the image information, the audio information, and the drive control information based on the determination result in the proximity determination unit that determines whether or not, and the determination result in the gaze determination unit and the proximity determination unit And determining that at least one of the image information, the audio information, and the drive control information is output, at least one of the image information, the audio information, and the drive control information. A determination result obtained by the gaze determination unit and the proximity determination unit during or after an output period of at least one of the image information, the audio information, and the drive control information, the output control unit configured to output one A call for determining whether the call by the output of at least one of the image information, the audio information, and the drive control information is successful based on It is obtained by such and a determination unit.

  According to a second aspect of the present invention, in the output control unit, a determination result in the gaze determination unit and the proximity determination unit, the image information, the audio information, and the drive control information associated with the determination result. And at least one of the plurality of types of operations for outputting at least one and the action value table storing the value of the action value of each of the operations, the determination result in the gaze determination unit and the proximity determination unit A selection unit that selects the operation having the highest action value among the plurality of types of operations, and at least one of the image information, the audio information, and the drive control information according to the selected operation. And a control unit for outputting the.

  According to a third aspect of the present invention, the output control unit includes a determination result in the gaze determination unit and the proximity determination unit, the image information associated with the determination result, the audio information, and the drive control information. Are associated with determination results in the gaze determination unit and the proximity determination unit based on a behavior value table that stores a plurality of types of operations that output at least one of the behaviors and a behavior value of each of the operations. A control unit that randomly selects an operation from the plurality of types of operations, and a control that outputs at least one of the image information, the audio information, and the drive control information according to the selected operation. Part.

  According to a fourth aspect of the present invention, the face direction data acquisition unit acquires, from the face recognition sensor, face direction data relating to each of a plurality of passers over time, and the distance data acquisition unit includes the distance data Distance data between the information output device and each of the plurality of passers is acquired over time from the measurement sensor, and the gaze determination unit is configured to determine the passersby for each of the plurality of passers-by. It is determined whether or not the time when the face is facing a predetermined range including the information output device is equal to or longer than the first time, the proximity determination unit, for each of the plurality of passers-by It is determined whether or not the time during which the passerby exists within a predetermined distance from the information output device is the second time or more, and the output control unit corresponds to each of the plurality of passersby, The determination result in the gaze determination unit and the proximity determination unit, and the A behavior value table that stores a plurality of types of operations that output at least one of the image information, the audio information, and the drive control information associated with a fixed result, and behavior value values of the respective operations Of the action value from among combinations of the plurality of passersby and the plurality of types of actions associated with the determination results in the gaze determination unit and the proximity determination unit for each of the plurality of passers A selection unit that selects a combination of a passer-by and an action corresponding to the action having the highest value; and for the selected passer-by, the image information, the audio information, and the drive corresponding to the selected action And a control unit that outputs at least one of the control information.

  According to a fifth aspect of the present invention, the face direction data acquisition unit acquires, from the face recognition sensor, face direction data related to each of a plurality of passers, and the distance data acquisition unit acquires the distance. Distance data between the information output device and each of the plurality of passers is acquired over time from the measurement sensor, and the gaze determination unit is configured to determine the passersby for each of the plurality of passers-by. It is determined whether or not the time when the face is facing a predetermined range including the information output device is equal to or longer than the first time, the proximity determination unit, for each of the plurality of passers-by It is determined whether or not the time during which the passerby exists within a predetermined distance from the information output device is the second time or more, and the output control unit corresponds to each of the plurality of passersby, The determination result in the gaze determination unit and the proximity determination unit, and the A behavior value table that stores a plurality of types of operations that output at least one of the image information, the audio information, and the drive control information associated with a fixed result, and behavior value values of the respective operations Based on the combination of the plurality of passers and the plurality of types of movements associated with the determination results in the gaze determination unit and the proximity determination unit for each of the plurality of passers. And selecting a passer with the highest value of the action value of the selected action among the plurality of passers, and selecting the passer based on the action value table. A passerby is randomly selected from a combination of the plurality of passers and the plurality of types of movements associated with the determination results in the gaze determination unit and the proximity determination unit for each. A second selection unit that selects the operation having the highest action value associated with the selected passerby among the plurality of types of actions, and the plurality of actions based on the action value table. Selecting a motion and a passer at random from a combination of the plurality of passers and the plurality of types of actions associated with the determination results in the gaze determination unit and the proximity determination unit for each passerby of At least one of a third selection unit, and the output control unit selects the passerby selected by the first selection unit, the second selection unit, or the third selection unit. A control unit that outputs at least one of the image information, the audio information, and the drive control information according to the operation selected by the unit.

  According to a sixth aspect of the present invention, when the information output device determines that the call is successful, the information output device increases the value of the action value of the selected action in the action value table, and the passerby When it is determined that the call to is unsuccessful, the behavior value table updating unit that decreases the value of the behavior value of the selected action in the behavior value table is further provided.

  According to the first aspect of the present invention, the face orientation data relating to the passerby is acquired over time from the face recognition sensor, and the distance data between the information output device and the passerby is acquired from the distance measurement sensor over time. Is acquired. A gaze determination process in which it is determined whether or not the time during which the face of the passerby faces the predetermined range including the information output device is the first time or more based on the acquired face direction data. Executed. Further, based on the acquired distance data, proximity determination processing is performed in which it is determined whether the time during which the passerby is present within the predetermined distance from the information output device is equal to or longer than the second time. The Whether or not to output at least one of image information, voice information, and drive control information is determined based on the determination result of the gaze determination process for the passerby and the determination result of the proximity determination process, and image information and When it is determined to output at least one of audio information and drive control information, at least one of the image information, audio information, and drive control information is output. Based on the determination result obtained by the gaze determination process and the proximity determination process during or after the output period of at least one of the image information, the audio information, and the drive control information, the image information, the audio information, and the drive control information It is determined whether the call with at least one of the outputs is successful.

  Face orientation is roughly the same as gaze direction, so is the passer's face orientation similar to the distance between the information output device and the passer, is the passer showing interest in the information output device? It is useful to determine whether or not. Therefore, by using the face direction of the passer as a reference in addition to the distance between the information output device and the passer as described above, for example, a passerby who is certainly interested in the information output device is targeted. It is possible to output information as

  Further, when it is determined whether or not the call is successful, the following effects can be obtained. That is, if the call is successfully determined during the information output period, for example, if there is no passerby that satisfies the information output condition, the output of the information is stopped and wasted. Power consumption can be reduced. Furthermore, the information output condition can be reviewed based on the information on whether or not the call is successful, and it can be determined that the operator changes the environment in which the information output device is installed.

  According to the second aspect of the present invention, at least one of the determination results of the gaze determination processing and the proximity determination processing, and the image information, audio information, and drive control information associated with the determination results is output. The action value of the plurality of types of actions associated with the determination results of the gaze determination process and the proximity determination process based on the action value table storing the plurality of types of actions and the action value values of the respective actions. The operation with the highest value is selected. Thereafter, at least one of image information, audio information, and drive control information corresponding to the selected operation is output. As described above, if an action is selected based on a behavior value value, there is an advantage that the selected passerby may be stopped with accuracy of a success rate guaranteed in advance.

  According to the third aspect of the present invention, based on the behavior value table, an action is randomly selected from the plurality of types of actions associated with the determination results of the gaze determination process and the proximity determination process. Thereafter, at least one of image information, audio information, and drive control information corresponding to the selected operation is output.

  For example, when an action is selected on the basis of the action value, it is possible that an action with a low action value is not selected at the initial stage. However, among the operations whose action value is set to be low, there may be actually those that are not so low in the probability of successful call by the operation. As described above, by selecting an action at random, it is possible to obtain information on whether or not the call by such an action is successful. It can be used to review whether the value setting is appropriate.

  According to the fourth aspect of the present invention, face direction data relating to each of a plurality of passers is acquired from the face recognition sensor over time, and the information output device and the plurality of passers are acquired from the distance measurement sensor. Distance data between each is acquired over time. A gaze determination process in which it is determined for each of the plurality of passersby whether or not the time when the face of the passer is facing a predetermined range including the information output device is equal to or longer than the first time. Is executed. In addition, for each of the plurality of passers-by, a proximity determination process for determining whether or not the time during which the passer-by exists within a predetermined distance from the information output device is equal to or longer than the second time. Executed. Then, corresponding to each of the plurality of passers-by, at least one of the determination result of the gaze determination process and the proximity determination process, and the image information, audio information, and drive control information associated with the determination result Are associated with the determination results of the gaze determination process and the proximity determination process for each of the plurality of passersby, based on the action value table storing the plurality of types of actions for outputting and the action value values of the respective actions. The combination of the passer-by and the action corresponding to the action having the highest action value is selected from the combination of the plurality of passers-by and the plurality of kinds of actions. Thereafter, at least one of image information, audio information, and drive control information corresponding to the selected operation is output to the selected passerby.

  For this reason, it is possible to make a call by outputting information to passersby by a method that is most likely to stop at least one of a plurality of passersby based on the value of action value. .

  Further, for example, the display angle is adjusted so that the front of the display faces the direction of the selected passer-by and the image information is output, or the front of the speaker faces the direction of the selected passer-by. Audio information may be output by adjusting the direction, or the movable part may be driven to present a gesture toward the direction of the selected passerby. In this way, the selected passer-by becomes more susceptible to the provision of the image information or the voice information and the gesture performed by the movable unit, and the probability that the passer's call by the output of the information is successful is high. Become.

  According to the fifth aspect of the present invention, face direction data relating to each of a plurality of passersby is acquired sequentially from the face recognition sensor, and the information output device and the plurality of passersby are obtained from the distance measurement sensor. The distance data between each of these is acquired over time. A gaze determination process in which it is determined for each of the plurality of passersby whether or not the time when the face of the passer is facing a predetermined range including the information output device is equal to or longer than the first time. Is executed. In addition, for each of the plurality of passers-by, a proximity determination process for determining whether or not the time during which the passer-by exists within a predetermined distance from the information output device is equal to or longer than the second time. Executed. Then, corresponding to each of the plurality of passers-by, at least one of the determination result of the gaze determination process and the proximity determination process, and the image information, audio information, and drive control information associated with the determination result Are associated with the determination results of the gaze determination process and the proximity determination process for each of the plurality of passersby, based on the action value table storing the plurality of types of actions for outputting and the action value values of the respective actions. The action is randomly selected from the combination of the plurality of passers and the plurality of types of actions, and the passer with the highest action value of the selected action among the plurality of passers is selected. Selected. Alternatively, based on the action value table, a passer is randomly selected from among the plurality of passersby and the combination of the plurality of types of actions, and further, the selected passer of the plurality of types of actions The action having the highest action value associated with is selected. Alternatively, based on the action value table, an action and a passer-by are selected at random from a combination of the plurality of passers-by and the plurality of kinds of actions. Thereafter, at least one of image information, audio information, and drive control information corresponding to the selected operation is output to the selected passerby.

  In this way, one of the action and the passer can be randomly selected, and the other can include the selection such that the value of the action value is the highest, in the random choice between the action and the passer. When an action is performed based on the selection and information on whether the call by the action is successful is acquired, for example, based on a combination of a passerby and the action corresponding to the action having the highest action value. Thus, it is possible to analyze the rate of contribution to the change in the value of action value by changing the passerby, the rate of contribution to the change in the value of action value by changing the action, and the like.

  According to the sixth aspect of the present invention, when it is determined that the call has succeeded, the value of the action value of the selected action in the action value table is increased, and it is determined that the call has failed. In this case, the action value of the selected action in the action value table is decreased. For this reason, when selecting an operation to be executed by the information output device based on the value of action value, an operation with a successful call is easily selected after the next time, and an operation with a failed call is less likely to be selected after the next time. . That is, the information output device learns autonomously and sequentially so that it can cope with the installed environment.

  In other words, according to the present invention, information output control is performed based on the face direction of the passer in addition to the distance to the passerby, and the information output that can determine the success or failure of the call to the passer by the output of the information Devices, methods and programs can be provided.

The figure which shows an example of the information output device which outputs image information or audio | voice information with respect to the said passerby according to the detection of a passerby according to 1st Embodiment of this invention. FIG. 2 is a block diagram showing a functional configuration of the information output device shown in FIG. 1; FIG. 2 is a view showing an example of distance data acquired by the distance measurement sensor shown in FIG. 1; FIG. 6 is a view showing an example of face direction data acquired by the face recognition sensor shown in FIG. 1; The figure which shows an example of the operation | movement which outputs the image information or audio | voice information which can be performed by the information output device shown in FIG. The flowchart which shows an example of the action value table initialization process performed by the control unit of the information output device shown in FIG. FIG. 3 is a flow chart showing an example of a measurement value acquisition process executed by a control unit of the information output device shown in FIG. 2; The flowchart which shows an example of the user state determination process performed by the control unit of the information output device shown in FIG. FIG. 3 is a view showing an example of a head status log table H stored in a storage unit of the information output device shown in FIG. 2; The figure which shows an example of the body condition log table B memorize | stored in the memory | storage unit of the information output device shown in FIG. The figure which shows an example of head state log table H '' produced | generated from head state log table H shown in FIG. The figure which shows an example of the body state log table B '' produced | generated from the body state log table B shown in FIG. The figure which shows an example of the integrated log table M produced | generated combining the head state log table H "shown in FIG. 11, and the body state log table B" shown in FIG. FIG. 14 is a view showing an example of generating a target-specific log table from the integrated log table M shown in FIG. 13; The figure which shows an example of the proximity determination process performed in the user state determination process shown in FIG. The figure which shows an example of the gaze determination process performed in the user state determination process shown in FIG. FIG. 7 is a flow chart showing an example of output control processing executed by a control unit of the information output device shown in FIG. 2; The figure which shows an example of the action value table memorize | stored in the memory | storage unit of the information output device shown in FIG. The flowchart which shows an example of the action value table update process performed by the control unit of the information output device shown in FIG. The conceptual diagram of the tracking process using the nearest neighbor method. A flow showing an example of processing that is executed along with the user state determination processing shown in FIG. 8 to generate a head state log table H ′ and a body state log table B ′ to which an id value is added by executing tracking processing Figure. The flowchart which shows an example of the process which produces | generates the match list | wrist which matches the id _H value and id _B value which show the same passer, and is performed with the user state determination process shown in FIG. An example of processing for generating the integrated log table M by combining the head state log table H ′ and the body state log table B ′ using the match list, which is executed in accordance with the user state determination processing shown in FIG. FIG. The figure which shows an example of the operation | movement which outputs the drive control information or audio | voice information which can be performed by the information output device based on other Embodiment of this invention.

Embodiments according to the present invention will be described below with reference to the drawings.
First Embodiment
(Constitution)
FIG. 1 is a view showing an example of an information output apparatus 1 for outputting image information or voice information to a passer in accordance with the detection of a passer according to the first embodiment of the present invention.

  The information output device 1 is, for example, virtual robot interactive signage or the like which outputs image information or voice information to a passerby and calls for use of the service. Although FIG. 1 shows an example in which the information output device 1, the face recognition sensor 2, the distance measurement sensor 3, the display 4 and the speaker 5 are integrated, each of them is separated as a separate device. And any combination of these may be integrated.

  The information output device 1 acquires face direction data relating to a passerby over time from a face recognition sensor 2 such as a camera. Further, the information output device 1 temporally acquires distance data between the information output device 1 and a passerby from the distance measurement sensor 3 which is, for example, an ultrasonic distance sensor or an infrared distance sensor. Based on the acquired face orientation data and distance data, the information output device 1 determines whether to output image information or audio information. And when it determines with outputting, image information and audio | voice information are output using the display 4 and the speaker 5, respectively.

  Furthermore, the information output device 1 determines whether the call by the output of the image information or the sound information is successful or not during or after the output period of the image information or the sound information based on the face direction data and the distance data. judge.

FIG. 2 is a block diagram showing a functional configuration of the information output apparatus 1 shown in FIG.
The information output device 1 includes a control unit 11, a storage unit 12, and an input / output interface unit 13 as hardware.

  The input / output interface unit 13 includes, for example, one or more wired or wireless communication interface units. The input / output interface unit 13 acquires face direction data relating to a passerby from the face recognition sensor 2, acquires distance data between the information output device 1 and the passerby from the distance measurement sensor 3, Face direction data and distance data are output to the control unit 11. Further, the input / output interface unit 13 outputs image information and audio information output from the control unit 11 to the display 4 and the speaker 5, respectively.

  The storage unit 12 uses, as a storage medium, a non-volatile memory such as a hard disc drive (HDD) or a solid state drive (SSD) that can be written and read as needed, and a head state to realize this embodiment. A log table H storage unit 121, a body state log table B storage unit 122, a subject-specific log table storage unit 123, and an action value table storage unit 124 are provided.

  The head status log table H storage unit 121 stores a head status log table H. The head state log table H stores face orientation data relating to one or more passers-by acquired from the face recognition sensor 2.

  The physical condition log table B storage unit 122 stores a physical condition log table B. The body state log table B stores distance data acquired from the distance measurement sensor 3 between the information output device 1 and one or more passers-by.

  The target-specific log table storage unit 123 is used to store one or more target-specific log tables for each passer by integrating face direction data and distance data for each passer-by.

  The behavior value table storage unit 124 stores a behavior value table. The action value table corresponds to each of one or more passers-by, and the gaze state of the passer-by to the information output device 1, the proximity state between the information output device 1 and the passer-by, the gaze state and the proximity state A plurality of types of operations for outputting image information or audio information associated with a combination of the values and action value values of the respective operations are stored.

  The control unit 11 includes a hardware processor such as a CPU (Central Processing Unit) and a program memory. In order to execute the processing functions in the present embodiment, the face orientation data acquisition unit 111, the distance data acquisition unit 112, , The target log table generation unit 113, the gaze determination unit 114, the proximity determination unit 115, the output control unit 116, the challenge success / failure determination unit 117, the behavior value table initialization unit 118, and the behavior value table update unit 119. And. All of the processing functions in these units are realized by causing the hardware processor to execute a program stored in the program memory.

  The face direction data acquisition unit 111 sequentially acquires face direction data relating to one or more passersby from the face recognition sensor 2 through the input / output interface unit 13, and acquires the acquired face direction data as a head state log. Execute processing to write to table H.

  The distance data acquisition unit 112 acquires distance data between the information output device 1 and one or more passers from the distance measurement sensor 3 via the input / output interface unit 13 over time, and acquires the acquired distance data. , The process of writing in the body state log table B is executed.

  The target log table generation unit 113 first reads face orientation data and distance data from the head state log table H and the body state log table B. Then, the target-specific log table generating unit 113 integrates the face direction data and distance data for each passerby to generate one or more target-specific log tables for each passer-by, and generates one or more target-specific log tables. A process of storing the log table in the by-target log table storage unit 123 of the storage unit 12 is executed.

  The gaze determination unit 114 reads one or more target-specific log tables for each passerby from the target-specific log table storage unit 123 of the storage unit 12. Then, the gaze determination unit 114 determines, for each of the one or more passersby, based on the face direction data stored in the read one or more target-specific log tables for each passerby. A process of determining whether a face is facing a predetermined range including the information output device 1 is performed to determine whether or not it is the first time or more to determine the gaze state of one or more passersby.

  The proximity determination unit 115 also reads one or more target log tables for each passerby from the target log table storage unit 123 of the storage unit 12. Then, the proximity determination unit 115 sends the information output device 1 to each of the one or more passers, based on the distance data stored in the one or more subject-specific log tables that are read. A process of determining whether the time during which the passer is present within the predetermined distance is the second time or more is executed to determine the proximity state of one or more passers.

  The output control unit 116 refers to the behavior value table, and based on the determination results in the gaze determination unit 114 and the proximity determination unit 115 for each of one or more passers-by, information from the operations stored in the behavior value table By selecting an operation to be executed by the output device 1, it is determined whether to output image information or audio information. Then, when it is determined that the image information or the audio information is to be output, the output control unit 116 executes a process of causing the display 4 or the speaker 5 to output the image information or the audio information via the input / output interface unit 13. .

  The call success / failure determination unit 117 is based on the determination result obtained by the gaze determination unit 114 and the proximity determination unit 115 during or after the output period of the image information or the audio information, to the passerby by outputting the image information or the audio information. The process of determining whether or not the call is successful is executed.

  The behavior value table initialization unit 118 executes processing for initializing the behavior value table stored in the behavior value table storage unit 124 of the storage unit 12.

  The behavior value table updating unit 119 executes processing for increasing or decreasing the behavior value of the action selected by the output control unit 116 in the behavior value table based on the determination result in the challenge success / failure determination unit 117.

(Operation)
Next, the operation of the information output apparatus 1 configured as described above will be described.
(1) Specific Example of Measurement Data First, face orientation data and distance data acquired by the information output apparatus 1 will be described.

FIG. 3 is a diagram illustrating an example of distance data acquired by the distance measurement sensor 3 illustrated in FIG. 1. As a reference axis of the world coordinate system, for example, a positive direction is a direction from the position where the distance measuring sensor 3 is installed on the display 4 of the information output apparatus 1 toward the right side along the plane of the display 4. An axis, a Y-axis whose positive direction is the downward direction along the plane of the display 4, and a Z-axis whose positive direction is the frontward direction of the display 4 from the plane of the display 4 .
As shown in FIG. 3, the information output device 1 uses the distance measuring sensor 3 as the distance data at time t and an angle θ from the positive direction of the X axis to the direction of the center coordinates of the passers centering on the Y axis. obtaining a _p, the distance R _p between the position and the passerby center coordinates of the distance measuring sensor 3 on the display 4 is installed. The measurement of the distance data by the distance measuring sensor 3 is executed continuously or irregularly.

FIG. 4 is a diagram showing an example of face orientation data acquired by the face recognition sensor 2 shown in FIG. The face recognition sensor 2 is, for example, one in which imaging devices are arranged on a plane, and it is assumed that a plane on which the imaging devices are arranged is installed so as to be perpendicular to the Z axis.
FIG. 4A illustrates an angle of view θ _c that can be captured by the face recognition sensor. FIG. 4B shows the width W of the image captured by the face recognition sensor 2, the coordinates (x, y) at the upper left of the face detection frame based on the upper left point of the captured image, The width w of the face detection frame is illustrated. In addition, the said magnitude | size and coordinate are a pixel unit, for example. FIG. 4C illustrates an angle θ _h from the positive direction of the X axis to the front direction of the passer's face centered on a straight line passing through the center coordinates of the passer and parallel to the Y axis. . Note that θ _h is a positive value when the face of the passer is facing the side opposite to the display 4 and a negative value when the face of the passer is facing the display 4 side.

From the image captured by the face recognition sensor 2, the information output device 1 uses the time t, the upper left coordinates (x, y) of the face detection frame, and the size w of the width of the face detection frame as face direction data. , The angle θ _h .

(2) Specific Example of Operation for Outputting Image Information or Audio Information Next, each operation for outputting image information or audio information by the information output apparatus 1 will be described.

FIG. 5 is a diagram illustrating an example of an operation for outputting image information or audio information that can be executed by the information output apparatus 1 illustrated in FIG. 1 in response to detection of a passerby. FIG. 5 illustrates three types of operations a ₁ , a ₂ , and a ₃ that can be executed by the information output apparatus 1.

Operating a ₁ is, on the display 4, and outputs the image information of the person to induce while beckoning, from the speaker 5, is an operation to output the audio information corresponding to the words of the interrogation that "Please to here".
The action a ₂ outputs the image information of the person who is guided while beckoning on the display 4, and from the speaker 5, the voice information corresponding to the calling word “Please come here” and the attention of the passerby This is an operation for outputting sound information corresponding to the sound effect for drawing. Note that the volume of the audio information corresponding to the sound effect is, for example, larger than the volume of the above-described two types of audio information corresponding to the calling word.
The operation a ₃ is an operation for outputting the image information of the waiting person on the display 4.

(3) Operation of Information Output Device 1 Initializing Action Value Table FIG. 6 is a flow chart showing an example of action value table initialization processing executed by the control unit 11 of the information output device 1 shown in FIG. is there.

  The information output device 1 selects an operation for outputting image information or audio information based on the behavior value table. The action value table corresponds to each of one or more passers-by, and the gaze state of the passer-by to the information output device 1, the proximity state between the information output device 1 and the passer-by, the gaze state and the proximity state A plurality of types of operations for outputting image information or audio information associated with a combination of the values and action value values of the respective operations are stored.

  First, in step S101, the control unit 11 accesses a predetermined setting storage area of the storage unit 12 under the control of the action value table initialization unit 118, and the initialization flag flag of the action value table is an undefined value. It is determined whether or not. When it is determined that the initialization flag flag is an indefinite value, the processing from step S102 to step S105 is executed to initialize the behavior value table.

  In step S102, the control unit 11 accesses a predetermined setting storage area of the storage unit 12 under the control of the behavior value table initialization unit 118, and writes 20 as the maximum value N of the number of persons to be measured. Thus, the action value table can be handled until the number of passersby obtained from the face orientation data and distance data acquired by the information output device 1 reaches 20.

In step S103, the control unit 11 accesses a predetermined setting storage area of the storage unit 12 under the control of the behavior value table initialization unit 118, and writes, for example, 5 as the value of each element of the learning frequency vector K. Note that K = {K ₁ , K ₂ ,..., K _N }, K ₁ indicates the number of times of learning used when the number of measurement persons is 1, and K ₂ indicates that the number of measurement persons is 2 Indicates the number of learnings to be used at the same time, and so on. For example, each element of the learning frequency vector K defines a scale for increasing or decreasing the value of the behavior value when the behavior value table is updated under the control of the behavior value table update unit 119. For example, as the value of each element of the learning frequency vector K increases, the increase or decrease in the value of the behavior value when the behavior value table is updated becomes small.

  In step S <b> 104, the control unit 11 initializes a behavior value Q in the behavior value table under the control of the behavior value table initialization unit 118. The initialization may be based on initialization data stored in the storage unit 12, for example.

  In step S105, the control unit 11 accesses a predetermined setting storage area of the storage unit 12 under the control of the behavior value table initialization unit 118, and writes True in the initialization flag flag of the behavior value table. That the initialization flag flag of the action value table is True indicates that the action value table has already been initialized.

(4) Operation in which the information output device 1 detects a passerby and determines the state of the passerby In the following, the operation of the information output device 1 will be described from FIG. 9 to FIG. This will be described with reference to the example of the log table shown in FIG.

FIG. 7 is a flowchart showing an example of a measurement value acquisition process executed by the control unit 11 of the information output apparatus 1 shown in FIG.
First, in step S201, the control unit 11 initializes the head state log table H and the body state log table B, respectively.

  Next, in step S <b> 202, the control unit 11 acquires the current time t ′ and stores it in a predetermined area of the storage unit 12.

In step S203, under the control of the face orientation data acquisition unit 111, the control unit 11 receives, for example, measured values t, m, x, y, and the like as face orientation data relating to one or more passers-by from the face recognition sensor 2. Get w, θ _h . The measurement value m is stored by dividing the parameter set of the measurement values x, y, w, θ _h according to the value of m according to the value of m when face direction data pertaining to a plurality of passers are acquired at the same time. This is the same index below.

In step S 204, the control unit 11 stores the measurement values t, m, x, y, w, θ _h acquired in the head state log table H storage unit 121 under the control of the face direction data acquisition unit 111. Add a row to the log state log table H and write it. FIG. 9 shows an example of the head state log table H stored in the head state log table H storage unit 121.

On the other hand, in step S205, under the control of the distance data acquisition unit 112, the control unit 11 uses the distance measurement sensor 3 as distance data between the information output device 1 and one or more passersby, for example, as a measurement value t. , M, R _p and θ _p are obtained.

In step S206, the control unit 11 stores the measured values t, m, R _p , and θ _p acquired under the control of the distance data acquisition unit 112 in the body condition log table B storage unit 122. Add row to table B and write. FIG. 10 shows an example of the body state log table B stored in the body state log table B storage unit 122.

  The processing in step S203 and step S204 and the processing in step S205 and step S206 are continuously or periodically performed.

  In step S207, the control unit 11 determines whether or not t−t ′, which is the time elapsed from time t ′ to the current time t, is greater than a predetermined time T. When the elapsed time t−t ′ does not exceed T, the measurement value acquisition process in step S203 and step S205 is continued. If the elapsed time t-t 'exceeds T, one or more target-specific log tables are created for each passer, as shown in FIG. 8, in which face direction data and distance data are integrated for each passer, A process of determining a passerby state based on the target log table is executed. It is known that the time Ta when a human recognizes and reacts to a visual stimulus is about 200 milliseconds. As the predetermined time T, for example, a time of about 0.5 seconds, which is longer than Ta, is used. Use.

FIG. 8 is a flowchart showing an example of a user state determination process executed by the control unit 11 of the information output apparatus 1 shown in FIG.
First, in step S301, the control unit 11 reads the head state log table H stored in the head state log table H storage unit 121 under the control of the target log table generation unit 113, and the read head state log. Give id _H column to table H. In the assignment of the id _H column, the same id _H value is given to the parameter set of the same passerby for each parameter set of the measured values t, m, x, y, w, θ _h , and different passing Different id _H values are assigned to the parameter set of the person.

In step S302, the control unit 11 reads the physical condition log table B stored in the physical condition log table B storage unit 122 under the control of the object-specific log table generation unit 113, and the read physical condition log table B Give id _B column to In the assignment of the id _B column, the same id _B value is given to the same passer's parameter set for each parameter set of the measurement values t, m, R _p , and θ _p , and the parameters of different passers Different id _B values are assigned to the set. Note that the processing in step S301 and the processing in step S302 may be executed in an arbitrary order.

Next, in step S303, the control unit 11, under the control of the target-specific log table generating unit 113, id _H column and head state log table H which is applied, id _B columns granted body state log table B And the id _H value indicating the same passer-by and the id _B value are associated with each other.

In step S304, the control unit 11 controls the head status log table H under the control of the object-specific log table generation unit 113, and the head status log table H ′ to which the id _H column is assigned, and the body status log table B id _B in the column body state log table granted B', based on the correspondence between id _H value and id _B value, replacing the id _H value and id _B value, the parameter set of the same passer is The same value id _M value is given, different parameter id _M values are given to different passer parameter sets, and a head condition log table H ′ ′ and a body condition log table B ′ ′ are generated. Thereafter, the control unit 11 combines the head status log table H ′ ′ and the body status log table B ′ ′ under the control of the subject-specific log table creation unit 113 to create an integrated log table M.

FIG. 11 shows an example of the head state log table H ″ generated from the head state log table H shown in FIG. 9, and FIG. 12 shows the example generated from the body state log table B shown in FIG. An example of physical condition log table B '' is shown. Note that the head condition log table H ′ ′ shown in FIG. 11 has all the values of the R _p column and the θ _p column filled with NA and added, and the body status log table B ′ ′ shown in FIG. , y column, w column, θ _h columns have been added to fill in all the value NA. Further, FIG. 13 shows an example of the integrated log table M generated by combining the head state log table H ′ ′ shown in FIG. 11 and the body state log table B ′ ′ shown in FIG.

In step S305, the control unit 11 divides the integrated log table M based on the id _M value under the control of the target log table generation unit 113, and integrates the face direction data and the distance data for each passerby. Create one or more target-specific log tables M1, M2, ... for each passerby, and store the generated one or more target-specific log tables M1, M2, ... in the target-specific log table storage unit 123 . FIG. 14 shows an example of the target log tables M1 and M2 generated from the integrated log table M shown in FIG.

Finally, in step S306, the control unit 11 reads and reads one or more target log tables M1, M2,... Stored in the target log table storage unit 123 under the control of the gaze determination unit 114. For each passerby corresponding to the id _M value, the face of the passer's face outputs the information output device 1 based on the face direction data stored in the one or more target-specific log tables M1, M2,. A gaze determination process for determining whether or not the time in which the predetermined range is included is equal to or longer than the first time is executed. Further, under the control of the proximity determination unit 115, the control unit 11 reads one or more target-specific log tables M1, M2,... Stored in the target-specific log table storage unit 123, Based on the distance data stored in the subject log tables M1, M2,..., The time that the passerby exists within a predetermined distance from the information output device 1 for each passerby corresponding to the id _M value. The proximity determination process for determining whether or not is equal to or longer than the second time is executed.

(5) Specific Example of Gaze Determination Process and Proximity Determination Process As described above, the state of the passer is determined from two types of viewpoints by the gaze determination process and the proximity determination process. That is, the state "s _id " of the passerby is represented by the following expression by the determination result "isNear (M _id )" of the proximity determination process and the determination result "is Looking (M _id )" of the gaze determination process.

FIG. 15 is a diagram illustrating an example of the proximity determination process executed in step S306 of FIG.
As described above, the proximity determination process corresponds to the id _M value based on the distance data acquired from the distance measurement sensor 3 and stored in any of the object-specific log tables M1, M2,. This is a process for determining for each passerby whether or not the time during which the passerby is within a predetermined distance from the information output device 1 is equal to or longer than the second time.

As the predetermined distance R _B, for example, it may be used 100cm known as boundary personal distance and social distance in the robot and humans. Further, as the second time T _2, for example, may be used half the time of the measurement time. The predetermined distance R _B and the time T ₂ are, may be any value.

The above-mentioned proximity determination process is expressed by the following equation, for example, which returns True if the passerby has a time T ₂ or more within the distance R _B from the display 4 of the information output device 1 and returns False otherwise. Is performed using the The function is a function defining a condition related to the measurement value R _p of the distance between the information output apparatus 1 obtained from the distance measuring sensor 3 and the passerby. In the following equation, the time T ₂ is illustrates the case where a half hours of time as a reference of the measurement time (t-t') / 2. Also, in the following equation, the integral symbol represents a numerical integral such as trapezoidal rule, and the missing value is ignored as a value. If R _p (t) is all missing values, the otherwise condition is applied. The same applies to the following.

Note that U in the above equation is a unit step function and is expressed as follows.

FIG. 16 is a diagram illustrating an example of gaze determination processing executed in step S306 of FIG.
As described above, the gaze determination process uses the id _M value based on the face direction data acquired from the face recognition sensor 2 and stored in any of the subject-specific log tables M1, M2,. This is a process for determining for each corresponding passer whether or not the time when the face of the passer is facing a predetermined range including the information output device 1 is equal to or longer than the first time.

In the present embodiment, for example, an angle is used as a criterion for determining whether or not the face of the passerby faces the predetermined range including the information output device 1. It is known that the viewing angle 2θa at which _a human can capture a stimulus without moving his head is 22.0 degrees. Thus, for example, as the reference angle of the gaze determination, it may be used 22.0 degrees viewing angle 2 [Theta] _a. Further, as the time T ₁ of the aforementioned first, for example, may be used half the time of the measurement time. Incidentally, the reference angle and the time T ₁ may be of any value.

The above-mentioned gaze determination process is, for example, True when a pedestrian is facing the center side of the display 4 of the information output device 1 for a time T ₁ or more, and returns False otherwise, a function represented by the following equation Is performed using The function is a function that defines the conditions regarding the measurement values x, w, and θ _h described above acquired from the face recognition sensor 2. The following equation shows the case where the time T ₁ is (t−t ′) / 2, which is half of the time used as the reference of the measurement time. In the following equation, θ _h , θ _a , and θ _p (t) are expressed in radians.

Note that θ _p (t) is expressed as follows.

From the distance measurement sensor 3 instead of executing the gaze determination process from the measurement values x, w, θ _h from the face recognition sensor 2 using the θ _p calculated by the above equation using the measurement values x, w using measurements theta _p it may execute the attention judgment process.

(6) Operation of Output Control of Information Output Device 1 FIG. 17 is a flow chart showing an example of the output control process executed by the control unit 11 of the information output device 1 shown in FIG.

  First, in step S401, the control unit 11 accesses a predetermined setting area of the storage unit 12 under the control of the output control unit 116, calculates and writes a random number in the range of 0 to 1.

  Next, in step S402, the control unit 11 determines whether or not the written random number is smaller than ε under the control of the output control unit 116.

  When it is determined that the random number written in step S402 is not smaller than ε, in step S403, the control unit 11 accesses a predetermined setting area of the storage unit 12 under the control of the output control unit 116, and Calculate and write a random number in the range of 0 to 1.

  In step S404, the control unit 11 determines whether or not the written random number is smaller than ε under the control of the output control unit 116.

If it is determined in step S404 that the random number written is not smaller than ε, then in step S405, the control unit 11 stores the action value table stored in the action value table storage unit 124 under the control of the output control unit 116. Referring to FIG. 8, the action value Q is the highest among the combinations of the one or more passers-by and the plurality of types of actions associated with the determination results of the gaze determination process and the proximity determination process in step S306 of FIG. A combination of the passer's id _M value and the action _ai corresponding to the action is selected.

If it is determined that the random number written in step S404 is smaller than ε, the control unit 11 associates the determination result of the gaze determination process and the proximity determination process in step S306 of FIG. 8 under the control of the output control unit 116. In step S406, an action a _i is randomly selected from the combination of the one or more passers and the plurality of actions, and in step S407, the selected action a of the one or more passers is selected. Select the id _M value of the passer whose value Q of _i 's action value is the highest.

  On the other hand, when it is determined that the random number written in step S402 is smaller than ε, in step S408, the control unit 11 accesses a predetermined setting area of the storage unit 12 under the control of the output control unit 116, and Calculate and write a random number in the range of 0 to 1.

  In step S409, the control unit 11 determines whether or not the written random number is smaller than ε under the control of the output control unit 116.

If it is determined in step S409 that the random number written is not smaller than ε, the control unit 11 determines the result of the gaze determination process and the proximity determination process in step S306 of FIG. 8 under the control of the output control unit 116. In step S410, a passer's id _M value is randomly selected from the associated one or more passers-by and a plurality of types of actions, and in step S411, the selection is made among the plurality of types of actions. The action a _i having the highest action value Q associated with the passer's id _M value is selected.

When it is determined that the random number written in step S409 is smaller than ε, the control unit 11 associates the determination result of the gaze determination process and the proximity determination process in step S306 of FIG. 8 under the control of the output control unit 116. From the combination of the one or more passersby and the plurality of types of actions, the action _ai is randomly selected in step S412, and the id _M value of the passer is randomly selected in step S413.

In step S405, in steps S406 and S407, in steps S410 and S411, or in steps S412 and S413, after operation a _i and the id _M value of the passer are selected, control unit 11 is selected in step S414. outputs operation _{a i} selected, a signal indicating the selected id _M value, the target-specific log table _{M id} for the selected id _M values. Thus, for example, by using the positional information and the like in the target-specific log table M _id for the selected id _M value for passerby corresponding to id _M value selected, corresponding to the selected operation a _i Image information or audio information can also be output.

FIG. 18 is a diagram illustrating an example of an action value table referred to in the selection process of the action a _i and the passer's id _M value. The value of the action value in the said action value table has shown the example used as the initial value set in the action value table initialization process of FIG.6 S104, for example. The operation _{a 1,} a 2, _{a 3} stored in the action-value table shown in FIG. 18 coincides with the operation _{a 1,} a 2, _{a 3} defined in FIG.

  The action value table is determined according to the number of passersby obtained from the face orientation data and distance data acquired by the information output device 1, for example. (A) of FIG. 18 shows an example of the action value table to be referred to when the number of measured persons of the passer is 0, and (b) of FIG. 18 is referred to when the number of measured persons of the passer is 1 An example of the action value table to be performed is shown.

For example, as shown in FIGS. 18A and 18B, in the behavior value table, for each passerby, the determination result “isNear (M _id )” of the proximity determination process for the passerby and the gaze Three types of operations a ₁ , a ₂ , a ₃ for outputting image information or audio information for each combination of determination results “isLooking (M _id )” of the determination processing, and the respective operations a ₁ , a ₂ , A ₃ action value values are stored.

That is, the action value table includes the number of passers, the id _M value of the passerby, the determination result of the proximity determination process for the passer, "isNear (M _id )", and the determination result of the gaze determination process for the passer This is a table that returns the value Q of the action value of the operation according to the type of isLooking (M _id ) ”and the operation a _i . That is, the action value Q can be expressed by Q (n, id _M , isNear (M _id ), isLooking (M _id ), a _i ).

Here, an example of the behavior value table in which the value of the behavior value is represented by the initial value when the number of measured people is 0 or 1 is shown, but for example, a table with the number of measured people from 2 to N is determined by the following equation. The action value value Q can be used as an initial value.

Incidentally, the proximity determination processing of the determination result for passerby "isNear _{(M id)"} and for each combination of attention judgment processing of the judgment result of "isLooking _{(M id),} each operation is associated _a 1, _{a 2} , by adjusting the value of the activation level of _{a 3,} executed in accordance with each combination of passerby proximity determination processing of the determination result "isNear _{(M id)"} and the attention judgment processing of the judgment result of "isLooking _{(M id)} It is possible to define an action that is easy to be done.

For example, the proximity determination processing of the determination result "isNear _{(M id)"} is False, and, for the combination of the determination result of the attention judgment processing "isLooking _{(M id)"} is False, the operation _{a 3} is not executed induction in order to increase the probability of being selected, the value of the activation level of the operation a ₃ may be set to a higher value for any passerby.

Also, the proximity determination processing of the determination result "isNear _{(M id)"} is False, and, for the combined determination result of the attention judgment processing "isLooking _{(M id)"} is True, operating _{a 2} to perform induction There in order to increase the probability of being selected, the value of the activation level of the operation a ₂ may be set to a higher value for any passerby.

Furthermore, the proximity determination processing decision result "isNear _{(M id)"} is True, and the operation determination result of the attention judgment processing "isLooking _{(M id)"} is for the combination is True or False, to perform induction However, in order to increase the probability of selecting the action a ₁ whose volume is low because the output sound does not include a sound effect for attracting the passer's attention, an arbitrary passerby can be selected. the value of the activation level of the operation a ₁ may be set to a higher value for.

Note that the setting of the value of the action value described above is merely an example, and for example, the determination result of the proximity determination processing “isNear (M _id )” is True, and the determination result of the fixation determination processing “isLooking (M _id )” is The action value of each operation may be set so that the probability that the operations a ₁ and a ₂ for performing guidance are selected is high only for the combination that is True. Furthermore, although each of the operations a ₁ , a ₂ , and a ₃ is defined as outputting image information in FIG. 5, for example, an operation may be such that neither image information nor audio information is output. .

(7) Operation of Information Output Device 1 Updating Action Value Table FIG. 19 is a flowchart showing an example of action value table update processing executed by the control unit 11 of the information output device 1 shown in FIG.

First, in step S501, the control unit 11 determines whether or not the number n of passers-by when the image information or audio information corresponding to the operation a _i is output by the processing in step S414 in FIG. judge.

If it is determined that the number n of passers-by is greater than 0, in step S502, the control unit 11 sets the id _M value set used in the output control process of the information output apparatus 1 shown in FIG. _It is stored in a predetermined area of the storage unit 12 as a set of _M 'values.

  Further, in step S503, the control unit 11 sets the value of the number n of passersby used in the output control process of the information output apparatus 1 shown in FIG. Remember me.

  Next, in step S504, the control unit 11 stops the measurement value acquisition process for a predetermined time. The predetermined time may correspond to, for example, the reproduction time of image information or audio information output by the information output device 1. For example, when the reproduction time is 10 seconds or more, by setting the predetermined time to 10 seconds, the time at which the information output device 1 starts the next measurement value acquisition process after stopping the measurement value acquisition process. However, it may be the time during the output period of the image information or audio information. The predetermined time is set such that the time when the information output device 1 starts the measurement value acquisition process next after the measurement value acquisition process stops corresponds to the time when the reproduction time of the image information or the audio information is over. It may be

  In step S <b> 505, the control unit 11 acquires the face direction data and distance data related to the passerby over time under the control of the face direction data acquisition unit 111 and the distance data acquisition unit 112. The acquisition process may be performed based on, for example, the same predetermined time T as the process in step S207 of FIG. 7 or may be performed based on a time different from the process in step S207 of FIG. . The acquisition process is executed during or after the output period of the image information or audio information.

Next, in step S506, the control unit 11 performs the same processing as the processing in steps S301 to S306 in FIG. That is, under the control of the gaze determination unit 114, the control unit 11 performs, for each passerby corresponding to the newly assigned id _M value, the passerby based on the face orientation data acquired in the process in step S505. It is determined whether the time during which the face of the person faces the predetermined range including the information output device 1 is equal to or longer than the first time. Further, under the control of the proximity determination unit 115, the control unit 11 sets a predetermined distance from the information output device 1 for each passerby corresponding to the id _M value based on the distance data acquired in the process in step S505. It is determined whether or not the time during which the passerby is present is equal to or longer than the second time.

  In step S507, the control unit 11 controls the passerby by outputting the image information or the audio information based on the determination result obtained by the gaze determination process and the proximity determination process in step S506 under the control of the call success / failure determination unit 117. It is determined whether or not a passerby has used the device in response to the call.

Whether or not the passer-by has used the device in response to a call to the passer-by by outputting the image information or the audio information is determined by the following determination formula.

  That is, for at least one passerby, the time when the face of the passerby faces the predetermined range including the information output device 1 is the first time or more, and the predetermined distance from the information output device 1 When it is determined that the time when the passer is present is equal to or more than the second time, the call to the passer by the output of the image information or the voice information is successful and the passerby uses the device It is judged. In the above description of step S506 and step S507, in the gaze determination process and the proximity determination process, the process is performed based on the same time as the first time and the second time which are the reference in step S306 of FIG. Although described as being executed, the process may be executed based on times different from the first time and the second time as the reference in step S306 of FIG.

If it is determined in step S507 that calling to the passerby by output of image information or voice information is successful and it is determined that the passerby has used the device, in step S508, the control unit 11 determines that the action value table updating unit 119 Under the control, the action value Q of the executed action a _i is increased.

On the other hand, if it is determined in step S507 that the passerby has not been successfully called out by outputting the image information or the voice information and it is determined that the passerby did not use the device, the control unit 11 performs action value in step S509. Under the control of the table updating unit 119, the action value Q of the executed action a _i is decreased.

In addition, the process which increases / decreases the value Q of the action value in step S508 and step S509 is performed according to the following recurrence formula, for example.

That is, when it is determined that the call to the passer is successful, in the action value table corresponding to the number n of passers in the output control process, for each passer, The action value Q of the executed action a _i associated with the determination results in the gaze determination process and the proximity determination process is increased.

On the other hand, if it is determined that the call to the passer has failed, in the action value table corresponding to the number n of passers in the output control process, for each passer, The action value Q of the executed action a _i associated with the determination results in the gaze determination process and the proximity determination process is decreased.

  The above recurrence formula has a meaning of a ratio (the number of times of use / the number of calls) used according to the call when the number n ′ of passersby calls.

Finally, in step S510, the control unit 11 accesses a predetermined setting storage area of the storage unit 12 and increments the value of the element K _n 'of the learning number vector K by one.

(8) Specific Example of Integrated Log Table Generation Processing Here, the head state log table H and the body state log table B, which are executed in steps S301 to S304 in FIG. The process of generating the integrated log table M will be described in detail.

  FIG. 20 is a conceptual diagram of a tracking process of adding passer information to each parameter set of measured values of face direction data and distance data acquired in the information output device 1 using the nearest neighbor method. For each parameter set, the distance to each temporally adjacent parameter set is calculated, and it is determined that the parameter set with the smallest calculated distance is the parameter set for the same passerby. However, even if the distance is the closest parameter set, if the distance is greater than a predetermined value, it is determined that the parameter set relates to a different passerby.

In the present embodiment, in the face direction data related to one or more passersby stored in the head status log table H, tracking processing is performed to add passer information using the nearest neighbor method, and the same passer The same id _H value is assigned to the parameter set, and different id _H values are assigned to different passer parameter sets. The same process is executed in the body state log table B, and an id _B value is assigned to each parameter set.

FIG. 21 shows an example of processing for generating the head state log table H ′ by assigning id _H values that are passer-by information to each parameter set of the head state log table H, which is executed in step S301 of FIG. 8 is a flow chart showing an example of processing executed at step S302 in FIG. 8 to generate body condition log table B ′ by giving id _B value which is passerby information to each parameter set of body condition log table B. is there.

Hereinafter, as an example, an example of generating the head state log table H ′ from the head state log table H will be described in detail.
The generation process of the head state log table H ′ and the body state log table B ′ is executed by the tracking process illustrated in FIG.

  In step S <b> 601, the control unit 11 reads the head state log table H stored in the head state log table H storage unit 121 under the control of the target log table generation unit 113.

In step S602, the control unit 11 adds an id _H value to the parameter set stored together with the measurement start time in the read head state log table H under the control of the by-target log table generation unit 113. When there are a plurality of parameter sets stored together with the measurement start time, different id _H values are assigned to the respective parameter sets.

In step S _< b> 603, the control unit 11 first selects a measurement start time as a reference time for the id _H value assignment process under the control of the target log table generation unit 113.

Each In step S604, the control unit 11, under the control of the target-specific log table generating unit 113, as a reference id _H value of grant processing id _H value, the parameter sets stored together with the selected measurement start time selecting a starting id _H values from the granted id _H value. Here, as an example, it is assumed that 1 is selected as the start id _H value.

Next, in step S605, the control unit 11 stores the temporally adjacent parameter set after the reference time stored in the head status log table H under control of the subject-specific log table generation unit 113 (x _{Among track} , y _track and w _track ), a distance to a parameter set (x _row , y _row , w _row ) stored with the above reference time and to which 1 which is a reference id _H value is given is a predetermined space Determine if there is something in the distance. The predetermined spatial distance may be, for example, a fixed value or proportional to the time difference between parameter sets. The spatial distance is calculated using the following equation.

If it is determined in step S605 that there is a parameter set within the predetermined spatial distance, the control unit 11 determines the parameter set within the predetermined spatial distance under the control of the target-specific log table generation unit 113 in step S606. Among them, 1 which is the same value as the reference id _H value is assigned as the id _H value to the parameter set having the smallest spatial distance.

If it is determined in step S605 that there is no parameter set within the predetermined spatial distance, or after the processing in step S606, the control unit 11 performs the reference under the control of the target log table generation unit 113 in step S607. based all id _H values that are assigned to each parameter set stored together with the time, whether or not the processing of steps S605 and step S606 has been completed is determined. If it is determined that the processing in step S605 and step S606 is not completed on the basis of all id _H values assigned to each parameter set stored together with the reference time, in step S608, the control unit 11 Under the control of the target log table generation unit 113, the reference id _H value is incremented to the next id _H value in the range of the id _H value assigned to the parameter set stored together with the reference time, and from step S605 The process is repeated.

If it is determined that the processing in step S605 and step S606 is completed on the basis of all id _H values assigned to each parameter set stored together with the reference time, in step S609, the control unit 11 Under the control of the log table generation unit 113, one of the parameter sets to which the id _H value has not been assigned yet among the parameter sets that are temporally adjacent after the reference time is already assigned to any parameter set. to impart a different new id _H value is the id _H values are.

  In step S610, the control unit 11 determines under the control of the by-target log table generation unit 113 whether all the measured times are used as the reference time and whether the processing from step S604 to step S609 is completed. If it is determined that the processing from step S604 to step S609 is not completed with all the measured times as the reference time, in step S611, the control unit 11 performs control under the control of the object-specific log table generation unit 113. The reference time is incremented, and the processing from step S604 to step S609 is repeated.

When it is determined that the processing from step S604 to step S609 has been completed using all the measured times as reference times, in step S612, the control unit 11 controls the head state under the control of the target log table generation unit 113. A log table H ′ in which id _H values are assigned to all parameter sets stored in the log table H is output and stored in a predetermined area of the storage unit 12.

  Although the example which produces head state log table H 'from head state log table H was explained in detail in the above, body state log table B' is generated from body state log table B by the same method.

However, when generating the body state log table B ′, the spatial distance in step S605 is calculated using the parameter set (R _p , θ _p ).
Is calculated by

FIG. 22 is a flowchart illustrating an example of a process for generating a match list that associates id _H values and id _B values indicating the same passers, which are executed in step S303 of FIG.

  In step S <b> 701, the control unit 11 reads the head state log table H ′ and the body state log table B ′ stored in the storage unit 12 under the control of the target log table generation unit 113.

In step S702, the control unit 11 stores the time-series data of the parameter set with the same id _H value stored in the head state log table H ′ under the control of the target log table generation unit 113, the body The DTW (Dynamic Time Warping) distance between the time-series data of the parameter set assigned with the same id _B value and stored in the state log table _{B ′} is set to each combination of id _H value and id _B value. To calculate.

  In step S <b> 703, the control unit 11 selects the smallest one of the calculated DTW distances under the control of the target log table generation unit 113.

  In step S704, the control unit 11 determines whether or not the value of the selected DTW distance is smaller than a predetermined value under the control of the target log table generation unit 113.

When it is determined that the selected DTW distance value is smaller than the predetermined value, in step S705, the control unit 11 controls the id _H corresponding to the selected DTW distance under the control of the target log table generation unit 113. Associate the value with the id _B value.

In step S706, the control unit 11 determines, under the control of the by-target log table generation unit 113, whether or not there is any id _H value and id _B value that are not associated with each other.

When it is determined that there is a case in which the above association is not made in both the id _H value and the id _B value, in step S 707, the control unit 11 has already taken action under the control of the object-specific log table generation unit 113. It marked with been id _H values and selects the smallest of the DTW distance calculated for each combination of id _H value and id _B values except id _B values. The process from step S704 is repeated for the selected DTW distance.

Here, if it is determined in step S704 that the value of the DTW distance selected as the smallest value is greater than a predetermined value, or in step S706, all the id _H values or all id _B values are handled as described above. If it is determined that the ID _H value has been added, the matching between the id _H value and the id _B value is completed, and in step S 708, the control unit 11 executes the id processing under the control of the object-specific log table generation unit 113. A match list indicating the association between the _H value and the id _B value is output and stored in a predetermined area of the storage unit 12.

  FIG. 23 shows an example of processing for generating the integrated log table M by combining the head status log table H ′ and the body status log table B ′ using the match list, which is executed in the step S304 of FIG. FIG.

  In step S801, the control unit 11 reads out the head status log table H ′, the body status log table B ′, and the match list stored in the storage unit 12 under the control of the by-target log table generation unit 113.

  In step S802, the control unit 11 duplicates the read head status log table H ′ and the body status log table B ′ under the control of the subject-specific log table generation unit 113, and the head status log table H ′ ′ And a body state log table B ″.

In step S803, the control unit 11 selects one of the correspondence between the id _H value and the id _B value from the match list under the control of the by-target log table generation unit 113.

In step S804, the control unit 11 is given an id _H value related to the selected association among the parameter sets stored in the head state log table H ″ under the control of the target log table generation unit 113. The id that has already been assigned to one of the parameter sets and one of the parameter sets stored in the physical condition log table B ′ ′ to which the id _B value related to the selected association has been assigned. to impart a different new id _M value and _M value.

In step S805, the control unit 11 determines whether or not the process in step S804 has been completed for all associations between id _H values and id _B values in the match list under the control of the target log table generation unit 113. To do. If it is determined that the processing in step S804 has not been completed for all the associations between the id _H value and the id _B value, in step S806, the control unit 11 controls the target log table generation unit 113 under the control. Select one of the correspondences between the unprocessed id _H value and id _B value from the match list. Thereafter, the process from step S804 is repeated.

If it is determined in step S805 that the processing in step S804 has been completed for all associations between id _H values and id _B values, the control unit 11 performs control of the target log table generation unit 113 in step S807. Below, the id _H column is deleted in the head state log table _{H ″} , and the id _B column is deleted in the body state log table _{B ″} .

In step S 808, the control unit 11 adds all values in the column R _{p and the} column θ _p in the head state log table H ′ ′ under the control of the object-specific log table generation unit 113 and adds the values. The head state log table H ″ after the processing corresponds to the example of the head state log table H ″ illustrated in FIG. On the other hand, in step S 809, the control unit 11 controls all the x columns, y columns, w columns, and θ _h columns in the body state log table B ′ ′ under the control of the object-specific log table generation unit 113. Fill in and add. Note that the post-processing body state log table B ″ corresponds to the example of the body state log table B ″ illustrated in FIG. In addition, the order of the process of step S808 and step S809 does not matter.

  In step S810, the control unit 11 generates an integrated log table M by combining the head state log table H ″ and the body state log table B ″ under the control of the target log table generation unit 113.

  In step S811, the control unit 11 deletes m columns from the integrated log table M under the control of the target log table generation unit 113.

  In step S 812, the control unit 11 outputs the integrated log table M under the control of the target log table generation unit 113 and stores it in a predetermined area of the storage unit 12. The integrated log table M corresponds to the example of the integrated log table M shown in FIG.

(effect)
As described in detail above, the first embodiment of the present invention has the following effects.
(1) Under the control of the face orientation data acquisition unit 111, the head orientation data related to one or more passers is acquired over time from the face recognition sensor 2 and stored in the head state log table H storage unit 121. The status log table H is written. On the other hand, under the control of distance data acquisition unit 112, distance data between information output device 1 and one or more passersby is acquired from distance measurement sensor 3 and stored in physical condition log table B storage unit 122. It is written to the physical condition log table B. Thereafter, under the control of the target log table generation unit 113, one or more targets for each passerby that integrate face direction data and distance data for each passerby from the head state log table H and the body state log table B Another log table is generated. Thereafter, the face of the passer is facing a predetermined range including the information output device 1 for each passerby, based on one or more target-specific log tables for each passerby under the control of the gaze determination unit 114. A gaze determination process in which it is determined whether or not the time is equal to or longer than the first time is executed. Further, under the control of the proximity determination unit 115, the time during which the passer is present within a predetermined distance from the information output device 1 is determined for each passer based on one or more target-specific log tables for each passer. Proximity determination processing for determining whether or not the time is equal to or longer than 2 is executed. Thereafter, the action value table is referred to under the control of the output control unit 116, and the information output device 1 outputs the image information or the voice information based on the determination result of the gaze determination process and the proximity determination process for each passer. and operating a _i to be executed by being subject to passerby id _M value and is selected to be of the operation a _i, selected operating a _i, a signal indicating the selected id _M values, selected id _M value The per-target log table M _id is output.

  The face direction corresponds to the line of sight, so that the passer's face direction is similar to the distance between the information output device 1 and the passerby, the passerby is interested in the information output device 1 It is useful to determine whether the Therefore, by using the face orientation of the passer as a reference in addition to the distance between the information output device 1 and the passer-by as described above, for example, a passer-by who is certainly interested in the information output device 1 It is possible to output image information or audio information for

The signal indicating the selected id _M values, using the target-specific log table M _id for the selected id _M value, for example, in the direction of the passerby that id _M value is selected front of the display 4 Output the image information by adjusting the angle of the display 4 so that it faces, or output the audio information by adjusting the direction of the speaker 5 so that the front of the speaker 5 faces the passerby whose id _M value is selected It may be done. In this way, it becomes easier for the passerby whose id _M value is selected to be provided with the image information or the voice information, and the probability that the passerby's call by the output of the image information or the voice information is successful becomes high.

(2) Based on the determination result obtained by the gaze determination process and the proximity determination process during or after the output period of the image information or the audio information corresponding to the selected operation a _i under the control of the call success / failure determination unit 117. Thus, it is determined whether or not the call by the output of the image information or audio information is successful. Under the control of the behavior value table update unit 119, based on the determination result in the challenge success / failure determination unit 117, the value of the behavior value of the action a _i selected in the output control unit 116 in the behavior value table is increased or decreased. .

  If it is determined during the output period of the image information or audio information that the call has been successful, for example, if there is no passerby that satisfies the output condition of the image information or audio information, the image information or The output of audio information can be stopped to reduce wasteful power consumption. Furthermore, the output condition of the image information or the audio information can be reviewed or the operator can determine that the environment in which the information output device is installed can be changed depending on the information as to whether the calling is successful or not.

  Further, when the value of the action value in the action value table is updated as described above, when selecting the action to be executed by the information output device 1 based on the value of the action value, the action for which the call is successful is selected from next time onwards. This makes it easier to select an action for which the call failed, from the next time. That is, the information output device 1 learns autonomously and sequentially so that it can cope with the installed environment.

(3) In the control of the output control unit 116, the action value table is referred to. As a result, among the combinations of the one or more passersby and the plurality of types of actions associated with the determination result in the gaze determination process and the proximity determination process for each passerby corresponding to the id _M value, the value Q of action value The combination of the passer's id _M value and the action a _i corresponding to the action with the highest is selected. Alternatively, the action _ai is randomly selected from the combination of the one or more passersby and the plurality of kinds of actions, and the value Q of the action value of the selected action _ai among the one or more passersby is selected. The highest pedestrian id _M value is selected. Alternatively, from the combinations of said one or more passerby and a plurality of types of operation, id _M value of passersby is randomly selected, corresponding to the id _M value of the selected passerby among the plurality of types of operation The action a _i having the highest value Q of the attached action value is selected. Alternatively, the action a _i is randomly selected from the combination of the one or more passers and a plurality of kinds of actions, and the passer's id _M value is randomly selected.

  As described above, if the action is selected based on the value of the action value, there is an advantage that the selected passerby may be blocked, for example, with a certainty of the success rate guaranteed in advance. In addition, it is possible to make a call by outputting image information or audio information to a passer-by by a method in which at least one of a plurality of passers-by is most likely to be stopped.

  Also, for example, when selecting an action based on the value of the action value, it may happen that the action whose action value is set low in the initial stage is hardly selected. However, among the operations whose action value is set to be low, there may be actually those that are not so low in the probability of successful call by the operation. As described above, by selecting an operation at random, it is possible to obtain information as to whether or not the invocation by such an operation is successful, and the information is, for example, of the action value of the initial stage. It can be used to review whether the value setting is appropriate.

  Furthermore, in the random selection between the action and the passerby, one of the action and the passer is randomly selected, and the other includes the selection in which the value of the action value is the highest. When the action is executed based on the selection and information on whether the call by the action is successful is acquired, for example, based on the combination of the passerby and the action corresponding to the action having the highest action value, It is possible to analyze the rate of contribution to the change in the value of action value by changing the passerby, the rate of contribution to the change in the value of action value by changing the action, and the like.

(4) In the action value table, for example, when the determination result of the proximity determination processing “isNear (M _id )” is False, and the determination result of the gaze determination processing “isLooking (M _id )” is False, in order to increase the probability that the operation a ₃ is not executed induction is selected, the value of the activation level of the operation a ₃ may be set to a higher value for any passerby. Also, the proximity determination processing of the determination result "isNear _{(M id)"} is False, and, for the combined determination result of the attention judgment processing "isLooking _{(M id)"} is True, operating _{a 2} to perform induction There in order to increase the probability of being selected, the value of the activation level of the operation a ₂ may be set to a higher value for any passerby. Furthermore, the proximity determination processing decision result "isNear _{(M id)"} is True, and the operation determination result of the attention judgment processing "isLooking _{(M id)"} is for the combination is True or False, to perform induction However, in order to increase the probability of selecting the action a ₁ whose volume is low because the output sound does not include a sound effect for attracting the passer's attention, an arbitrary passerby can be selected. the value of the activation level of the operation a ₁ may be set to a higher value for.

  A passerby who looks at the information output device 1 without approaching the information output device 1 may be likely to be interested in the information output device 1, but whether or not to output information is based on the distance alone. According to the prior art which was determined to be, it could not be the target of information output. However, the configuration of the action value table described above can increase the probability that image information or audio information is output even for such passers-by.

  In addition, by performing the above-described volume adjustment, for example, it is possible to prevent an unpleasant feeling from being output by outputting voice information of a large volume to a passerby near the information output device 1.

[Other embodiments]
The present invention is not limited to the first embodiment. For example, in the first embodiment, the example in which the information output apparatus is an apparatus that outputs image information or audio information has been described. However, the information output device may be a device that outputs drive control information or audio information for driving an object. For example, the information output device is a humanoid robot that outputs drive control information for driving the movable part of the left-hand part as an object, and rotates the left-hand part accordingly to present a gesture to the passerby It may be a device capable of Note that there may be a plurality of the target objects, and the drive control information may be information for driving any one of the plurality of target objects.

FIG. 24 is a diagram illustrating an example of an operation of outputting drive control information or audio information that can be executed in response to detection of a passerby by the information output device. Each three operating _{a 1,} a 2, _{a 3} illustrated in FIG. 24 corresponds to the operation _{a 1,} a 2, _{a 3} as shown and described in connection with FIG.

The operation a ₁ is an operation in which a moving part of the left hand portion is driven to perform a gesture that guides the user while beckoning, and voice information corresponding to a call-to-call message is output from the speaker. .
Operating a _2, by driving the movable portion of the left hand part, perform a gesture to induce while beckoning, from the speaker, and audio information that corresponds to the word of the interrogation "Please come here", passerby The voice information corresponding to the sound effect for drawing the attention is output. Note that the volume of the audio information corresponding to the sound effect is, for example, larger than the volume of the above-described two types of audio information corresponding to the calling word.
Operation a ₃ is an operation to wait for the information output device without driving the movable portion of the left hand portion. That is, it is an operation that outputs neither drive control information nor audio information.

In the case of the example of FIG. 24, the behavior value table shown in FIG. 18 includes, for each passerby, the determination result “isNear (M _id )” of the proximity determination process and the determination result of the gaze determination process “ For each combination of isLooking (M _id ) ′ ′, three types of operations a ₁ , a ₂ , a ₃ that output drive control information or voice information and actions of the respective operations a ₁ , a ₂ , a ₃ It suffices to store the value of the value. The operations a ₁ , a ₂ and a ₃ stored in the action value table may be made to coincide with the operations a ₁ , a ₂ and a ₃ defined in FIG.

Note that the information output device is not limited to the device that outputs image information or sound information as described above, or the device that outputs drive control information or sound information. That is, the information output device may be a device capable of outputting at least one of image information, audio information, and drive control information, and capable of outputting the information capable of being output in any combination. In this case, the actions a ₁ , a ₂ , and a ₃ may be defined as arbitrary combinations of information that can be output as described above, and the action value table may be appropriately designed as described above. For example, as described above, if the movable unit is driven to present a gesture in the direction of the selected passerby, the gesture executed by the movable unit is selected. It becomes even easier for a person to receive, and the probability that the call by the passer by the output of at least one of the image information, the sound information, and the drive control information will be higher.

Also, the measurement data obtained from the face recognition sensor and the distance measurement sensor may be different from those described above, in which case the function for gazing determination processing and the proximity determination processing also differs from that described above. It may be performed using Also, in the above description, the tracking processing when giving an id value to the head status log table H and the physical status log table B has been described as using the nearest neighbor method, but the tracking processing is not limited to this. Absent. Furthermore, although the process of generating the match list that associates the id _H value with the id _B value has been described as using the DTW distance, the process of generating the match list is not limited to this.

  In addition, the type and configuration of the information output device, the configuration of each log table and action value table, the details of the gaze determination processing and the proximity determination processing, and the like can be variously modified without departing from the gist of the present invention. It is possible.

  In short, the present invention is not limited to the first embodiment as it is, and can be embodied by modifying the components without departing from the scope of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the first embodiment. For example, some components may be deleted from all the components shown in the first embodiment. Furthermore, you may combine suitably the component covering different embodiment.

  DESCRIPTION OF SYMBOLS 1 ... Information output device, 11 ... Control unit, 111 ... Face direction data acquisition part, 112 ... Distance data acquisition part, 113 ... Log table production part according to object, 114 ... Gaze determination part, 115 ... Proximity determination part, 116 ... Output Control unit 117: calling success / failure determination unit 118: action value table initialization unit 119: action value table updating unit 12: storage unit 121: head state log table H storage unit 122: body state log table B storage , 123 ... Subject-specific log table storage unit, 124 ... Action value table storage unit, 13 ... Input / output interface unit, 2 ... Face recognition sensor, 3 ... Distance measurement sensor, 4 ... Display, 5 ... Speaker

Claims (8)

An information output device capable of outputting at least one of image information, voice information, and drive control information for driving an object to the passerby according to detection of the passer,
A face direction data acquisition unit that acquires face direction data relating to the passer-by from a face recognition sensor over time;
A distance data acquisition unit that acquires distance data between the information output device and the passer-by from a distance measurement sensor over time;
Based on the acquired face orientation data, a gaze determination unit that determines whether or not the time when the passer's face is facing a predetermined range including the information output device is a first time or more,
A proximity determination unit that determines whether or not the time during which the passerby is within a predetermined distance from the information output device is a second time or more based on the acquired distance data;
Based on the determination results in the gaze determination unit and the proximity determination unit, it is determined whether to output at least one of the image information, the audio information, and the drive control information, and the image information and the An output controller that outputs at least one of the image information, the audio information, and the drive control information when it is determined to output at least one of the audio information and the drive control information;
Based on the determination result obtained by the gaze determination unit and the proximity determination unit during or after the output period of at least one of the image information, the audio information, and the drive control information, the image information and the audio An information output device comprising: a call success / failure determination unit that determines whether or not a call using information and at least one of the drive control information is successful. The output control unit
A plurality of types of operations for outputting at least one of the determination result in the gaze determination unit and the proximity determination unit, the image information associated with the determination result, the audio information, and the drive control information; The value of the action value is the highest among the plurality of types of actions associated with the determination results in the gaze determination unit and the proximity determination unit based on the action value table storing the value of the action value of each operation A selection unit for selecting an operation;
The information output device according to claim 1, further comprising: a control unit that outputs at least one of the image information, the audio information, and the drive control information according to the selected operation. The output control unit
A plurality of types of operations for outputting at least one of the determination result in the gaze determination unit and the proximity determination unit, the image information associated with the determination result, the audio information, and the drive control information; Based on a behavior value table storing the behavior value of each motion, a motion is randomly selected from the plurality of types of motions associated with the determination results in the gaze determination unit and the proximity determination unit. A selection section;
The information output device according to claim 1, further comprising: a control unit that outputs at least one of the image information, the audio information, and the drive control information according to the selected operation. The face orientation data acquisition unit acquires face orientation data relating to each of a plurality of passers from the face recognition sensor over time,
The distance data acquisition unit temporally acquires distance data between the information output device and each of the plurality of passersby from the distance measurement sensor,
The gaze determination unit determines, for each of the plurality of passers, whether or not the time during which the face of the passer is facing a predetermined range including the information output device is the first time or more. Judgment,
The proximity determination unit determines, for each of the plurality of passersby, whether or not the time during which the passer is present within a predetermined distance from the information output device is the second time or more.
The output control unit
Corresponding to each of the plurality of passers-by, among the determination results in the gaze determination unit and the proximity determination unit, the image information, the audio information, and the drive control information associated with the determination result Determination in the gaze determination unit and the proximity determination unit for each of the plurality of passersby, based on the action value table storing the plurality of types of operations outputting at least one and the action value of each operation A selection unit that selects a combination of a passer and an action corresponding to an action having the highest value of the action from a combination of the plurality of passers and the plurality of kinds of actions associated with a result;
2. The control unit according to claim 1, further comprising: a control unit that causes the selected passer to output at least one of the image information, the audio information, and the drive control information according to the selected operation. The information output device described. The face orientation data acquisition unit acquires face orientation data relating to each of a plurality of passers from the face recognition sensor over time,
The distance data acquisition unit temporally acquires distance data between the information output device and each of the plurality of passersby from the distance measurement sensor,
The gaze determination unit determines, for each of the plurality of passers, whether or not the time during which the face of the passer is facing a predetermined range including the information output device is the first time or more. Judgment,
The proximity determination unit determines, for each of the plurality of passersby, whether or not the time during which the passer is present within a predetermined distance from the information output device is the second time or more.
The output control unit
Among the image information, the voice information, and the drive control information among the determination result in the gaze determination unit and the proximity determination unit, the image information associated with the determination result, corresponding to each of the plurality of passers-by Determination in the gaze determination unit and the proximity determination unit for each of the plurality of passersby, based on the action value table storing the plurality of types of operations outputting at least one and the action value of each operation An operation is randomly selected from a combination of the plurality of passers-by and the plurality of types of actions associated with a result, and an action value of the selected action among the plurality of passers-by is selected. A first selection unit to select the highest passer,
From the combination of the plurality of passersby and the plurality of types of actions associated with the determination result in the gaze determination unit and the proximity determination unit for each of the plurality of passersby based on the action value table, A second selection unit that randomly selects a passer, and further selects, from among the plurality of types of actions, the action having the highest action value value associated with the selected passer,
From the combination of the plurality of passersby and the plurality of types of actions associated with the determination result in the gaze determination unit and the proximity determination unit for each of the plurality of passersby based on the action value table, And at least one of a third selection unit that randomly selects an action and a passerby,
The output control unit is configured to transmit the image information and the image information corresponding to the operation selected by the selection unit to the passerby selected by the first selection unit, the second selection unit, or the third selection unit. The information output device according to claim 1, further comprising: a control unit that outputs at least one of audio information and the drive control information.   When it is determined that the calling is successful, the value of the action value of the selected action is increased in the action value table, and it is determined that the calling to the passer fails. The information output device according to claim 2, further comprising an action value table update unit that decreases an action value of the selected action in the value table. A hardware processor and a memory are provided, and at least one of image information, audio information, and drive control information for driving an object can be output to the passer in response to detection of the passer. Information output method executed by the device, and
A process of acquiring face direction data relating to the passer from a face recognition sensor over time;
Obtaining distance data over time from the device and the passerby from a distance measuring sensor;
A gaze determination step of determining whether a time during which a face of the passerby faces a predetermined range including the device is longer than or equal to a first time based on the acquired face direction data;
A proximity determination step of determining whether or not a time during which the passerby exists within a predetermined distance from the device is a second time or more based on the acquired distance data;
Based on the determination results in the gaze determination process and the proximity determination process, it is determined whether to output at least one of the image information, the audio information, and the drive control information, and the image information and the A step of outputting at least one of the image information, the audio information, and the drive control information when it is determined to output at least one of the audio information and the drive control information; And the audio information and the drive control information based on determination results obtained by the gaze determination process and the proximity determination process during or after the output period of at least one of the image information, the audio information, and the drive control information And a step of determining whether or not the call by at least one output of the drive control information is successful.   The program which functions a hardware processor as each part with which the information output device in any one of Claims 1 thru | or 6 is equipped.