JP2018186326A - Robot apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide information that can trigger an idea of people participating in a conference.SOLUTION: A detecting unit detects the situation of meeting and a providing unit provides deviation information deviating from a subject of the meeting.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a robot apparatus and a program.

  Patent Document 1 provides topics extracted based on a minutes database that stores meeting minutes information and an external database that is different from the minutes database for the purpose of developing meeting discussions. A conference support apparatus is disclosed.

Japanese Patent Laying-Open No. 2015-156099

  An object of the present invention is to provide information that can induce the idea of a person participating in a conference.

  The invention described in claim 1 is a robot apparatus having detection means for detecting a meeting situation and providing means for providing deviation information deviating from the theme of the meeting.

  The invention according to claim 2 is the robot apparatus according to claim 1, wherein, when the detection unit detects a stagnation of the conference, the providing unit provides the deviation information.

  The invention according to claim 3 is the robot apparatus according to claim 2, wherein the providing means changes and provides the deviation information according to the degree of stagnation of the conference.

  According to a fourth aspect of the present invention, the detection means detects the stagnation of the conference based on at least one of the conversation of the conference participant, the facial expression of the participant, and the length of the silence time. The robot apparatus according to claim 2, wherein the robot apparatus is a robot apparatus.

  According to a fifth aspect of the present invention, in any one of the first to fourth aspects, the providing means provides the deviation information while providing information according to the subject matter of the conference. It is a robot apparatus as described in.

  The invention according to claim 6 is characterized in that the providing means changes and provides the deviation information according to a deviation indicating the degree of deviation from the content of the meeting. It is a robot apparatus of description.

  The invention according to claim 7 is characterized in that the providing means switches between providing information related to the subject of the conference and providing the deviation information according to the degree of deviation. It is a robot apparatus of description.

  The invention according to claim 8 provides the information related to the subject of the meeting and not appearing in the meeting when the degree of departure is less than or equal to a threshold, and the degree of departure is The robot apparatus according to claim 7, wherein the robot apparatus is provided as the deviation information when larger than a threshold value.

  According to a ninth aspect of the present invention, in the robot according to any one of the first to fourth aspects, the providing means changes and provides the deviation information according to a participant in the conference. Device.

  The robot apparatus according to claim 9, wherein the providing unit changes and provides the deviation information according to a combination of a plurality of participants in the conference. is there.

  The invention according to claim 11 further includes history management means for managing a history of a participant's reaction to the deviation information, and the providing means changes and provides the deviation information according to the history. The robot apparatus according to any one of claims 1 to 4, wherein:

  The invention according to claim 12 further includes storage means for storing the character string and the deviation information in association with each character string, and the providing means adds the character string detected from the conversation of the meeting. 12. The robot apparatus according to claim 1, wherein the deviation information associated with the robot is provided.

  The invention according to claim 13 is the robot apparatus according to any one of claims 1 to 12, wherein the deviation information includes at least one of a character string, an image, and a sound. .

  The invention described in claim 14 is a program that causes a computer to function as a detecting unit that detects a conference state and a providing unit that provides deviation information deviating from the subject of the conference.

  According to the inventions described in claims 1, 12, 13, and 14, it is possible to provide information that can induce an idea of a person participating in the conference.

  According to the second and fourth aspects of the present invention, information that can induce an idea is provided when the conference is stagnant.

  According to the third aspect of the present invention, unexpected information corresponding to the degree of stagnation of the conference is provided.

  According to the fifth aspect of the present invention, unexpected information is provided between the time when the information along the subject of the conference is provided.

  According to the sixth, seventh, and eighth aspects of the invention, unexpected information corresponding to the degree of deviation is provided.

  According to the ninth and tenth aspects of the present invention, unexpected information suitable for the participant is provided.

  According to the eleventh aspect of the present invention, surprising information is provided using the reaction of the participant.

It is a figure which shows the external appearance of the robot apparatus which concerns on this embodiment. It is a block diagram which shows the robot apparatus which concerns on this embodiment. It is a figure which shows a recognition information management table. It is a figure which shows a theme related information management table. It is a figure which shows the deviation information management table. It is a figure which shows the example of provision of deviation information.

  A robot apparatus according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 shows an example of a robot apparatus according to this embodiment.

  The robot apparatus 10 has a function of detecting a situation around the robot apparatus 10 and providing information according to the detection result. The robot apparatus 10 may detect the surrounding situation while moving, or may detect the surrounding situation in a stopped state.

  For example, the robot apparatus 10 detects the situation of a conference, information related to the theme (theme) of the conference (hereinafter referred to as “subject related information”), information deviated from the subject (hereinafter referred to as “deviation information”). For example). The conference is, for example, a place or situation (for example, a scene where conversation or the like is performed) where one or more people participate. Of course, animals other than humans, robot devices, devices other than robot devices, etc. may participate in the conference. The place where the conference is held is not limited. For example, the conference may be performed in a building (for example, a room such as a conference room), or may be performed outdoors, in a car, in a train, or the like. Also, the content of the conference is not limited and may be any content.

  The robot apparatus 10 is a humanoid robot as an example. Of course, the robot apparatus 10 may be a robot of a type other than a humanoid. In the example shown in FIG. 1, the robot apparatus 10 includes a body part 12, a head 14 provided on the body part 12, leg parts 16 provided at the lower part of the body part 12, and both side parts of the body part 12. And the finger part 20 provided at the tip of each arm part 18.

  The robot apparatus 10 includes, for example, at least one sensor among a visual sensor, an auditory sensor, a tactile sensor, a taste sensor, and an olfactory sensor, and has a visual, auditory, tactile, taste, and olfactory sense equivalent to the human senses. Has at least one sensory ability. The robot apparatus 10 has, for example, superficial sensations (tactile sensation, pain sensation, temperature sensation, etc.), deep sensation (pressure sensation, positional sensation, vibration sensation, etc.), cortical sensation (two-point identification sensation, three-dimensional identification ability, etc.) You may have the ability to sort and understand. The robot apparatus 10 may have a sense of balance. For example, a sensor such as a camera 22 is provided on the head 14, and vision is realized by image recognition using an image obtained by the camera 22 or the like. In addition, the robot apparatus 10 is provided with a sound collecting device such as a microphone, and hearing is realized by voice recognition using sound obtained by the microphone or the like.

  Even if the robot apparatus 10 has a function of detecting a brain wave of a person (for example, a person is provided with a brain wave detection device and receives a brain wave information transmitted from the brain wave detection device and analyzes a brain wave). Good.

  The leg portion 16 corresponds to an example of a moving means, and is driven by a driving force from a driving source such as a motor, for example. The robot apparatus 10 can be moved by the legs 16. The leg 16 may have a shape like a human foot, a roller, a tire, or the like, or may have another shape. The leg portion 16 is merely an example of a moving unit, and the robot apparatus 10 may include, for example, a configuration for flying (such as a propeller, a wing, and a flying engine) as a moving unit other than the leg unit 16. Alternatively, a configuration for moving in water (for example, an engine for moving in water) may be provided. That is, the robot apparatus 10 only needs to include at least one of a means for moving on land, a means for flying, and a means for moving underwater as the moving means. Of course, the robot apparatus 10 may not include a moving unit.

  The robot apparatus 10 may have a capability of grasping and operating an object by the arm part 18 and the finger part 20. The robot apparatus 10 may have an ability to move while grasping or holding an object.

  The robot apparatus 10 may have a function of generating sound. The robot apparatus 10 may have a communication function and transmit / receive data to / from other apparatuses. For example, the robot apparatus 10 may connect to the Internet or the like to receive information, transmit information, or perform a search using the Internet or the like. The robot apparatus 10 may have an ability to communicate with a person, another apparatus, or another robot apparatus by making a sound or sending a communication message.

  The robot apparatus 10 may have an ability to make a judgment close to a person by machine learning using artificial intelligence (AI). Neural network type deep learning may be used, or reinforcement learning that partially enhances the learning field may be used.

  The robot apparatus 10 may communicate with other devices by a communication function and control operations of the other devices, may operate other devices using a remote controller or the like, or may use a remote controller or the like. Alternatively, other devices may be operated directly. In the direct operation, for example, an operation unit (for example, a button or a panel) provided in another device is operated by the robot apparatus 10. For example, when the operation of another device cannot be controlled by communication, the robot apparatus 10 may operate another device using a remote controller or the like, or may directly operate another device. For example, the robot apparatus 10 analyzes an image obtained by a visual sensor, identifies an operation unit or a remote control of another device, and operates the other device or the remote control.

  Further, the robot apparatus 10 may include a display unit 24. Various images and messages are displayed on the display unit 24. Further, the robot apparatus 10 may include a projection device and project various images, messages, and the like onto other objects. For example, an image or the like is projected and displayed on a table or a wall.

  Hereinafter, the function of the robot apparatus 10 will be described in detail with reference to FIG. FIG. 2 is a block diagram showing a configuration of the robot apparatus 10.

  The communication unit 26 is a communication interface, and has a function of transmitting data to another device and a function of receiving data from another device. The communication unit 26 may be a communication interface having a wireless communication function or a communication interface having a wired communication function. The communication unit 26 is compatible with, for example, one or more types of communication methods, and may communicate with the communication partner according to a communication method suitable for the communication partner (that is, a communication method supported by the communication partner). . The communication method is, for example, infrared communication, visible light communication, Wi-Fi (registered trademark) communication, short-range wireless communication (for example, Bluetooth (registered trademark), RFID (Radio Frequency Identifier), or the like). The communication unit 26 switches, for example, the communication method according to the communication partner, the surrounding environment (for example, the distance between the robot device 10 and the communication partner, and the presence or absence of an obstacle between the robot device 10 and the communication partner). Etc.), the communication method may be switched. Further, as a communication frequency band, for example, a short wavelength band such as 800 MHz to 920 MHz (LPWA (Low Power Wide Area) or the like), a long wavelength such as 2.4 GHz or 5 GHz (MuLTEfire), or the like may be used. For example, the communication unit 26 may switch the frequency band according to the communication partner, or may switch the communication method according to the surrounding environment.

  The situation collection unit 28 has a function of collecting information related to the situation around the robot apparatus 10 (hereinafter referred to as “situation information”) using various sensors. As the sensor, the above-described visual sensor, auditory sensor, tactile sensor, taste sensor, olfactory sensor, or the like is used. The situation collecting unit 28 collects data (voice data) of sounds around the robot apparatus 10 (for example, conversations and noises of conference participants) as situation information by using an auditory sensor, for example, and uses a visual sensor to collect the robot. Data (image data) of images around the device 10 (for example, moving images and still images) is collected as status information. For example, the facial expressions and attitudes of the participants in the conference are photographed by a visual sensor, and a moving image and a still image representing the facial expressions and attitudes of the participants are obtained. Of course, information on the surroundings of the robot apparatus 10 may be collected by sensors other than these.

  The status collection unit 28 may acquire status information from a device or a sensor other than the robot device 10. For example, the status collection unit 28 may collect voice data detected by a microphone, may collect information such as a character string input using an input device such as a keyboard, or may be captured by a camera. Image data such as a moving image and a still image obtained by the above may be collected.

  The recognition processing unit 30 generates recognition information by applying recognition processing to the situation information collected by the situation collection unit 28. For example, when voice data is collected by the situation collection unit 28, the recognition processing unit 30 applies a voice recognition process to the voice data to recognize a character string representing the sound indicated by the voice data. Generate as When image data is collected by the situation collection unit 28, the recognition processing unit 30 applies image recognition processing to the image data, thereby recognizing a character string representing an image represented by the image data. Generate as For example, when a character string is represented in the image data, the recognition processing unit 30 extracts the character string from the image data as recognition information. In addition, the recognition processing unit 30 detects the expression and attitude of the participant represented in the image data, and uses a character string representing the detection result (for example, a character string expressing the expression and attitude of the participant) as recognition information. It may be generated. The recognition processing unit 30 stores the recognition information in the storage unit 48. For example, the recognition processing unit 30 stores the character string as the recognition information, the appearance frequency (for example, the number of appearances) of the character string during the meeting, and the last recognition time of the character string in the storage unit in association with each other. . When a character string is input by an input device such as a keyboard, the recognition processing unit 30 uses the input character string as recognition information, the character string, the appearance frequency of the character string, and the character string The final recognition time is associated with each other and stored in the storage unit.

  The detection unit 32 detects a stagnation of the conference (meeting of the conference) based on the status information collected by the status collection unit 28 and the recognition information generated by the recognition processing unit 30. For example, the detection unit 32 detects the stagnation of the conference based on at least one information among the conversation of the conference participants, the facial expressions of the participants, and the length of the silence time.

  For example, in a case where a keyword related to stagnation (hereinafter referred to as “stagnation keyword”) is included in a character string group as recognition information extracted from a conversation by voice recognition, the detection unit 32 determines that the meeting is Determined to be stagnant. One or more stagnation keywords are predetermined and may be changed by the user. When the appearance frequency (for example, the number of appearances) of the stagnation keyword during the conversation is equal to or higher than a predetermined threshold, the detection unit 32 may determine that the meeting is stagnation.

  The detection unit 32 may determine the stagnation level (stagnation degree) of the conference based on the content of the conference. For example, the detection unit 32 determines the stagnation degree based on the appearance frequency of the stagnation keyword during the conversation. The detection unit 32 determines that the stagnation degree of the conference is higher as the appearance frequency of the stagnation keyword is higher. Moreover, the stagnation degree may be defined for each stagnation keyword. For example, when a stagnation keyword associated with a high stagnation degree is extracted from the conversation, the detection unit 32 determines that the stagnation degree of the meeting is high, and when a stagnation keyword associated with a low stagnation degree is extracted from the conversation It is determined that the degree of stagnation of the meeting is low. The detection unit 32 determines that the degree of stagnation of the conference is high when the number of occurrences of the stagnation keyword associated with the high degree of stagnation is equal to or greater than a predetermined threshold, and the stagnation keyword associated with the low degree of stagnation is detected. When the number of appearances is equal to or greater than a predetermined threshold, it may be determined that the degree of stagnation of the meeting is low. In addition, when both the stagnation keyword associated with the high stagnation degree and the stagnation keyword associated with the low stagnation degree are extracted from the conversation, the detection unit 32 preferentially adopts the stagnation keyword associated with the high stagnation degree. Then, it may be determined that the degree of stagnation of the conference is high, or the stagnation degree may be determined by adopting a stagnation keyword having a higher appearance frequency among both stagnation keywords. For example, if the number of appearances of stagnation keywords associated with a high stagnation degree is greater than the number of appearances of stagnation keywords associated with a low stagnation degree, the detection unit 32 determines that the stagnation degree of the meeting is high.

  In addition, when a stagnation keyword is extracted from a conversation and a keyword related to liveliness or activity (hereinafter referred to as “liveness keyword”) is extracted, the detection unit 32 uses both the stagnation keyword and the liveliness keyword. You may detect the stagnation of a meeting. One or more lively keywords are predetermined and may be changed by the user. For example, the number of appearances of a stagnation keyword in a conversation (when multiple types of stagnation keywords are extracted, the total number of appearances of these types of stagnation keywords) is the number of appearances of liveliness keywords (a plurality of types of liveness keywords are extracted). If the number is greater than the total number of appearances of these multiple types of lively keywords, the detection unit 32 determines that the meeting is stagnant. Further, the detection unit 32 applies a weighting process to the number of appearances of each stagnation keyword and the number of appearances of each lively keyword extracted from the conversation, and based on the value obtained by the weighting process, the presence / absence of stagnation of the meeting is determined. You may judge. Moreover, the detection part 32 may determine the stagnation degree of a meeting based on the difference between the appearance frequency of a stagnation keyword and the appearance frequency of a lively keyword. For example, when the number of appearances of the stagnation keyword is larger than the number of appearances of the lively keyword, it is determined that the degree of stagnation of the meeting is higher as the difference in the number of appearances is larger.

  The detection unit 32 may determine whether there is a stagnation of the conference based on the loudness of a human voice included in the conversation. For example, when the loudness of a human voice is equal to or less than a predetermined threshold, the detection unit 32 determines that the meeting is stagnant. The detection unit 32 may determine the degree of stagnation based on the loudness of a human voice. For example, it is determined that the stagnation degree is higher as the voice is lower.

  In addition, character strings representing facial expressions related to stagnation (hereinafter referred to as “stagnation facial expressions”) are included in human facial expressions (for example, character string groups representing facial expressions) extracted from image data by image recognition. If it is, the detection unit 32 may determine that the meeting is stagnant. One or more stagnation facial expressions are predetermined and may be changed by the user. When a plurality of people are participating in the conference, the detection unit 32 may detect the stagnation of the conference based on the facial expressions of each participant. For example, the detection unit 32 may determine that the conference is stagnant when the facial expressions of a number of people equal to or greater than a predetermined threshold correspond to stagnant facial expressions, or a plurality of people participating in the conference If the facial expressions of a number greater than or equal to a predetermined ratio in the case correspond to stagnant facial expressions, it may be determined that the conference is stagnant.

  The detection unit 32 may determine the degree of stagnation based on a human facial expression. For example, a stagnation level is defined for each stagnation expression, and when a stagnation expression associated with a high stagnation level is detected, the detection unit 32 determines that the stagnation level of the meeting is high and is associated with a low stagnation level. If a stagnation expression is detected, it is determined that the degree of stagnation of the meeting is low. The detection unit 32 determines that the stagnation level of the conference is high when the number of detections of the stagnation facial expression associated with the high stagnation degree is equal to or greater than a predetermined threshold, and the stagnation facial expression associated with the low stagnation degree. When the number of detections is equal to or greater than a predetermined threshold, it may be determined that the degree of stagnation of the meeting is low. When both a stagnation expression associated with a high stagnation degree and a stagnation expression associated with a low stagnation degree are detected, the detection unit 32 preferentially adopts the stagnation expression associated with a high stagnation degree. The degree of stagnation of the conference may be determined to be high, or the degree of stagnation may be determined by adopting a stagnation expression with a higher number of detections among both stagnation expressions. For example, when the number of detections of stagnant facial expressions associated with a high degree of stagnation is greater than the number of detections of stagnant facial expressions associated with a low degree of stagnation, the detection unit 32 determines that the degree of stagnation of the meeting is high.

  When a stagnation facial expression is detected, and when a lively or actively related facial expression (hereinafter referred to as a “lively facial expression”) is detected, the detection unit 32 uses both the stagnation facial expression and the lively facial expression. The stagnation of the conference may be detected. One or a plurality of lively expressions (for example, a character string representing a lively expression) is determined in advance and may be changed by the user. For example, the number of detections of stagnation facial expressions (when multiple types of stagnation facial expressions are detected, the total number of detections of these multiple types of stagnation facial expressions) is the number of detections of lively facial expressions (when multiple types of stagnation facial expressions are detected, If the number is greater than the total number of detections of these multiple types of lively expressions, the detection unit 32 determines that the meeting is stagnant. In addition, the detection unit 32 may apply weighting processing to the number of detections of each stagnation facial expression and the number of detections of each lively facial expression, and determine whether there is a stagnation of the meeting based on the value obtained by the weighting processing. . Moreover, the detection part 32 may determine the stagnation degree of a meeting based on the difference between the detection count of a stagnation facial expression and the detection count of a lively facial expression. For example, when the number of times of detection of a stagnation facial expression is greater than the number of times of detection of a lively facial expression, it is determined that the degree of stagnation of the meeting is higher as the difference in the number of detection times is larger.

  Moreover, the detection part 32 may determine the presence or absence of the stagnation of a meeting based on the length of the silence time in a meeting. For example, when the length of the silence time (for example, the length of time during which no sound having a loudness equal to or greater than a predetermined threshold is detected) is equal to or greater than a predetermined threshold, the detection unit 32 causes the meeting to stagnate. It is determined that

  Based on one information selected from the stagnation keyword, the magnitude of the voice, the stagnation expression, and the length of the silence time, or the combination of two or more information, The degree of stagnation may be determined.

  The providing unit 34 has a function of providing subject-related information related to the subject of the conference and deviation information deviating from the subject of the conference. For example, when the detection unit 32 detects a stagnation of the conference, the providing unit 34 provides deviation information. The providing unit 34 may provide information (for example, subject-related information) in accordance with the theme of the conference, and may provide deviation information between the provisions. For example, deviation information may be provided when a stagnation of a meeting is detected while providing subject-related information. The subject-related information and the deviation information are predetermined information such as character strings, sounds, images, and the like. For example, information indicating a subject (for example, a character string indicating the subject) and subject-related information related to the subject (for example, a synonym for a character string indicating the subject, an image or sound having the subject as a meaning, the subject Character strings, images, sounds, etc.) that give the same or similar impression to the person are associated with each other. Also, information indicating the subject (for example, a character string), deviation information deviating from the subject (for example, an antonym of the character string indicating the subject (an antonym, an antonym, etc.), an image or a sound having the antonym as a meaning, Character strings, images, sounds, etc.) that give an impression different from the impression that a character string gives to people or an opposite impression are associated with each other.

  The theme of the meeting is determined by the recognition processing unit 30 as described later. When the subject is determined, the providing unit 34 refers to the correspondence between the information indicating the subject and the subject-related information, or the correspondence between the information indicating the subject and the deviation information, to thereby extract the subject extracted from the conversation. Provide relevant subject-related information and deviation information. In addition, the subject-related information related to the meaning of the character string and the character string (for example, the same meaning as the character string, the image or sound having the character string as the meaning, the same impression as the character string gives to the person or Character strings, images, sounds, etc.) that give a similar impression may be associated with each other. In this case, the providing unit 34 may provide the subject-related information associated with the character string extracted from the conference conversation by referring to the association. Also, a character string, deviation information deviating from the meaning of the character string (for example, an antonym (antonym, antonym, etc.) of the character string, an image or sound having the antonym as a meaning, and the character string are given to a person. A character string, an image, a sound, or the like that gives an impression different from an impression or an opposite impression may be associated with each other. In this case, the providing unit 34 may provide the deviation information associated with the character string extracted from the conference conversation by referring to the association. Since the impression given to a person differs depending on the shape, color, taste, smell, size, etc. of the object, the subject related information and deviation information may be determined in consideration of these factors. Information indicating the correspondence between the information indicating the subject and the character string extracted from the conversation and the subject related information is stored as the subject related information management information 52 in the storage unit 48. Further, information indicating the correspondence between the information indicating the subject and the character string extracted from the conversation and the deviation information is stored as the deviation information management information 54 in the storage unit 48. Of course, the providing unit 34 may acquire and provide subject-related information and deviation information from a device other than the robot device 10 using the Internet or the like.

  The providing unit 34 may change and provide the deviation information according to the appearance frequency of the character string extracted from the conference conversation. For example, the providing unit 34 may provide deviation information associated with a character string having a low appearance frequency (for example, a character string having an appearance frequency equal to or less than a threshold). Such a character string may have been extracted, for example, due to misrecognition of speech, and the deviation information associated with such a character string may be information that is surprising to the content of the meeting. There is. Therefore, by providing such deviation information, surprising information can be provided to the conference participants and ideas can be induced.

  The providing unit 34 may change and provide the deviation information according to the degree of stagnation indicating the degree of stagnation of the conference. For example, the stagnation degree is associated with each departure information in advance, and the providing unit 34 may acquire and provide the departure information associated with the stagnation degree determined by the detection unit 32. The deviation information associated with a higher degree of stagnation is information that is less relevant to the character string associated with the deviation information. Therefore, the higher the degree of stagnation of a meeting, the more deviation information is provided with a higher degree of deviation from the contents of the meeting. This provides more surprising information to the participants and can trigger ideas.

  The providing unit 34 may display an image or a character string as deviation information on a display device, or may emit sound as deviation information from a speaker or the like. The providing unit 34 may project an image or a character string as deviation information on a wall, a table, or the like by a projector, or may display the deviation information on a display unit included in the robot apparatus 10. Of course, an image, a character string, and a sound as deviation information may be provided simultaneously.

  The setting unit 36 has a function of setting a deviation degree indicating the degree of deviation of information from the content of the conference. The departure degree may be specified by the user or may be automatically set. When the departure degree is set, the providing unit 34 may change and provide the departure information according to the departure degree. For example, the departure degree is predetermined for each departure information, and the providing unit 34 acquires and provides departure information associated with the departure degree set by the setting unit 36. The deviation information associated with a higher degree of deviation is information that is less related to the character string associated with the deviation information. Therefore, the higher the degree of deviation is, the more deviation information is provided from the content of the conference, and the idea can be induced thereby.

  The setting unit 36 may set the departure degree based on the stagnation degree determined by the detection unit 32. For example, the higher the degree of stagnation, the higher the degree of departure.

  The providing unit 34 may switch between providing subject-related information and providing deviation information according to the degree of deviation set by the setting unit 36.

  The theme of the meeting is determined by the recognition processing unit 30, for example. Based on the situation information (conversation content, images, etc.) collected by the situation collection unit 28, the recognition processing unit 30 identifies the content being discussed in the conference and identifies the subject of the conference from the content. For example, the recognition processing unit 30 may specify a character string having the maximum number of appearances in the conversation, and may specify a theme of the conference based on the character string, or may identify a conference based on a plurality of character strings. The subject matter may be specified. For example, a character string and information indicating the subject (for example, a character string) are associated with each other, and the providing unit 34 refers to the association to thereby extract a character string (for example, the maximum number of appearances) from the conversation. Information indicating a subject associated with a character string) may be specified. Thereby, the subject is determined from the content of the conversation. The providing unit 34 uses the character string extracted from the conversation as the subject related information, and corresponds to the extracted character string (theme related information) by referring to the correspondence between the information indicating the subject and the subject related information. Information indicating the attached subject matter may be specified. In this way, the subject matter may be determined from the content of the conversation.

  The providing unit 34 provides subject-related information related to the subject of the conference and not appearing in the conference when a low departure level is set (for example, when a deviation level equal to or less than a threshold is set), When a high deviation degree is set (for example, when a deviation degree larger than a threshold is set), deviation information may be provided.

  For example, when the content of the meeting is related to a meal, that is, when a character string or an image related to the meal is mainly detected, the theme of the meeting is “meal”. When a low deviation degree is set and the character string or image “cake” is not detected from the content of the meeting, the providing unit 34 provides the character string or image related to “cake” as subject-related information. . When a high deviation degree is set, the providing unit 34 provides deviation information belonging to a field not related to the subject “meal”. For example, the providing unit 34 provides a character string and an image related to “rocket” as deviation information.

  The history management unit 38 has a function of managing a history of participant responses to deviation information. The detection unit 32 detects the reaction of the conference participants based on the situation information collected by the situation collection unit 28 and the recognition information generated by the recognition processing unit 30. Specifically, the detection unit 32 selects one piece of information selected from the character string extracted from the voice data, the magnitude of the voice, the facial expression of the participant, the length of the silence, and the length of the conversation, Alternatively, a participant's reaction is detected based on a combination of two or more pieces of information. After the deviation information is provided by the providing unit 34, when the detection unit 32 detects the participant's reaction, the history management unit 38 associates the deviation information with information indicating the participant's reaction, The history information 56 is stored in the storage unit 48. For example, when the conversation becomes active or the facial expressions of the participants become active after the departure information is provided, the history management unit 38 associates the information indicating that it has become active with the departure information. These are stored in the storage unit 48 as history information. By referring to the history information, what kind of deviation information identifies what reaction the participant shows.

  Further, the history management unit 38 may manage the history of reaction for each participant and create history information for each participant. In this case, the history management unit 38 associates the user identification information for identifying the participant whose reaction has been detected by the detection unit 32, the provided deviation information, and the information indicating the response of the participant. These are stored in the storage unit 48 as history information. The user identification information is, for example, the name of the detected participant, the user ID, an image representing the detected participant's face, voice data representing the detected participant's voice, and the like. For example, when a participant's face is recognized by the image recognition process, an image representing the face may be used as the user identification information. Further, when a participant's voice is recognized by the voice recognition process, voice data representing the voice may be used as user identification information. The name and user ID may be input by an input device such as a keyboard, for example, and when a participant's face is recognized by the image recognition process, the name and user ID associated with the image representing the face, It may be a name or a user ID acquired from a device that manages them. Thereby, what kind of reaction is shown by what deviation information for each user is specified.

  The history management unit 38 may manage the history of the participant's reaction to the subject related information. In this case, the history management unit 38 may manage the history of the reaction for each participant. Information related to the management may be stored in the storage unit 48 in the same manner as the history information.

  The providing unit 34 may select and provide deviation information based on the history information. For example, when the conference is stagnant, the detection unit 32 identifies the participant of the conference based on the image data and audio data collected by the situation collection unit 28, and the providing unit 34 In the history information including the user identification information, the deviation information in which conversation or facial expression becomes active is specified, and the deviation information is provided. This provides information that can encourage discussion among individual participants.

  In addition, when a deviation degree is set when providing deviation information, the history management unit 38 may also include information indicating the deviation degree in the history information. In this case, the setting unit 36 may set the deviation degree based on the history information. For example, the providing unit 34 specifies the degree of deviation in which conversation or facial expression has become active in the history information including the user identification information of the person participating in the conference, and the setting unit 36 sets the degree of deviation. The providing unit 34 provides deviation information corresponding to the degree of deviation.

  The moving unit 40 has a function of moving the robot apparatus 10 by at least one of a means for moving on land, a means for flying, and a means for moving underwater. In the example illustrated in FIG. 1, the moving unit 40 includes the leg unit 16. Of course, the robot apparatus 10 may not include the moving unit 40.

  The working unit 42 has functions for operating devices other than the robot apparatus 10, lifting an object, and moving an object. In the example illustrated in FIG. 1, the working unit 42 includes the arm unit 18 and the finger unit 20. Of course, the robot apparatus 10 may not include the working unit 42.

  The speaker 44 has a function of generating sound. For example, a sound related to the deviation information is emitted from the speaker 44.

  The UI unit 46 is a user interface unit, and includes a display unit (for example, the display unit 24 shown in FIG. 1) and an operation unit. The display unit is a display device such as a liquid crystal display. The operation unit is an input device such as a touch panel or a keyboard. Of course, it may be a user interface that serves both as a display unit and an operation unit (for example, including a touch-type display or a device that electronically displays a keyboard or the like on the display). The robot apparatus 10 may not include the UI unit 46, or may include a hardware key (for example, various buttons) without including the display unit. The buttons as hardware keys are, for example, buttons specialized for numeric input (for example, numeric keys), buttons specialized for direction instructions (for example, direction instruction keys), and the like.

  The storage unit 48 is a storage device such as a hard disk or a memory (for example, SSD). The storage unit 48 stores, for example, recognition information management information 50, subject related information management information 52, deviation information management information 54, history information 56, other information (for example, various data, various programs, etc.), and the like. Has been. These pieces of information may be stored in separate storage devices, or may be stored in the same storage device.

  The recognition information management information 50 is information for managing the recognition information generated by the recognition processing unit 30 and the recognition information input by the user. In the recognition information management information 50, for example, recognition information (for example, a character string), the appearance frequency of the recognition information, and the last recognition time of the recognition information are associated with each other. Each time the recognition information is generated or acquired, the recognition information management information 50 is updated.

  The subject related information management information 52 is information indicating the subject related information. In the subject-related information management information 52, for example, information indicating a subject (for example, a character string) and one or more subject-related information (for example, a character string, an image, a sound, etc.) related to the subject are mutually in advance. It is associated.

  The deviation information management information 54 is information indicating deviation information. In the deviation information management information 54, for example, information indicating a subject (for example, a character string) and one or more pieces of deviation information (for example, a character string, an image, sound, etc.) deviating from the subject are associated with each other in advance. It has been. In addition, a degree of stagnation indicating the degree of stagnation of the meeting may be associated with each deviation information in advance. Further, the departure degree may be associated with each departure information in advance.

  The history information 56 is information indicating a history of human reaction to the provided deviation information. In the history information 56, for example, provided deviation information and information indicating human reaction are associated with each other. Further, history information 56 for each participant may be created. Each time deviation information is provided and a human reaction is detected, the history information 56 is updated.

  The control unit 58 controls the operation of each unit of the robot apparatus 10. The control unit 58 has, for example, artificial intelligence (AI) as an intelligent unit, and the function of each unit of the robot apparatus 10 may be controlled by the artificial intelligence.

  Note that at least one of the status collection unit 28, the recognition processing unit 30, the detection unit 32, the providing unit 34, the setting unit 36, and the history management unit 38 is a device other than the robot device 10 (for example, a server or user not shown). Terminal device etc.). In this case, at least one part may not be provided in the robot apparatus 10. Further, at least one of the recognition information management information 50, the subject related information management information 52, the deviation information management information 54, and the history information 56 may be stored in a device other than the robot device 10. In this case, the at least one information may not be stored in the robot apparatus 10. In the above example, the robot apparatus 10 provides subject-related information and deviation information. However, a terminal device such as a smartphone or a PC (personal computer) may detect the stagnation of the conference, Related information and deviation information may be provided.

  Hereinafter, the robot apparatus 10 will be described in more detail.

  FIG. 3 shows an example of a recognition information management table as the recognition information management information 50. In the recognition information management table, for example, a character string as recognition information, the number of appearances of the character string, and a final recognition time are associated with each other. The character string is a character string generated by applying voice recognition processing to voice data, a character string generated by applying image recognition processing to image data, or a character string input by an input device such as a keyboard. , Etc. For example, the detection unit 32 determines the presence or absence of the meeting and the degree of the stagnation using the character string managed by the recognition information management table.

  FIG. 4 shows an example of the subject related information management table as the subject related information management information 52. In the subject-related information management table, for example, a character string indicating a subject or a character string extracted from a conversation, one or more subject-related information (for example, a character string, an image, a sound, etc.), a last use time, The priorities are associated with each other. The character string indicating the subject is determined by the recognition processing unit 30 as described above. The priority order is the order of provision of subject-related information, and is determined in advance. For example, the priority of the subject related information that is estimated to have a high degree of association with the subject is set higher. Of course, the priority order may be changed by a user, an administrator, or the like, or the priority order may not be used. For example, the subject-related information may be provided randomly without being based on the priority order.

  FIG. 5 shows an example of the deviation information management table as the deviation information management information 54. In the deviation information management table, for example, a character string indicating a subject or a character string extracted from a conversation, one or a plurality of deviation information (for example, a character string, an image, a sound, etc.), a priority, and a last use time Are associated with each other. The character string is, for example, a character string generated by applying voice recognition processing to voice data during conversation. Each deviation information is associated with a stagnation degree (stagnation level) and a deviation degree (deviation level).

  The priority order is the order of provision of deviation information and is determined in advance. For example, in a meeting in which a character string associated with departure information is extracted, the priority of departure information that is assumed to have an effect of inducing a human idea is set higher. For example, the priority is provided in order from the deviation information of higher rank. For example, the providing unit 34 provides the deviation information with the first priority for a predetermined period, and provides the deviation information with the second priority for the period after the period ends. The same applies to deviation information whose priority is No. 3 and thereafter. Thereby, each deviation information is provided in the order according to the priority. The providing unit 34 may form a combination of a plurality of deviation information according to the priority order and provide the combination. For example, when a combination is formed by two pieces of deviation information, the providing unit 34 provides a combination of the pieces of deviation information having the first priority and the second priority over the above period, and then the second and third pieces of deviation information. A combination (or a combination of No. 3 and No. 4 deviation information) is provided over the above-described period, and combinations are similarly provided for the subsequent priorities. As another example, the providing unit 34 may provide a combination of deviation information with higher priority and lower deviation information. For example, the providing unit 34 provides a combination of the deviation information with the first priority and the deviation information with the lowest priority over the period, and then the deviation information with the second priority and the second deviation from the lowest. A combination with information is provided over the above period, and a combination is similarly provided for subsequent priorities. Thus, by providing various combinations, unexpected information can be provided to the participants, and the ideas of the participants can be induced.

  The stagnation degree is set to a value between 1 and 10, for example. The larger the value, the higher the degree of stagnation. A higher degree of stagnation is associated with deviation information that is assumed to be less relevant to a character string. As a result, as the degree of stagnation is detected, the more stagnant the conference is, the more unexpected information is provided, and the participants' ideas are induced. In addition, the stagnation degree of each deviation information may be predetermined and may be changed by the user. For example, when the stagnation degree of the meeting is determined by the detection unit 32, the providing unit 34 provides deviation information associated with the stagnation degree.

  The deviation degree is set to a value between 1 and 10, for example. The larger the value, the higher the degree of deviation. A higher degree of deviation is associated with deviation information that is assumed to be less relevant to the character string. As a result, when a higher degree of departure is set, more unexpected information is provided and the participants are encouraged to induce ideas. For example, when the departure degree is set, the providing unit 34 provides departure information associated with the departure degree.

  Hereinafter, an example of providing deviation information will be described with reference to FIG. FIG. 6 shows an example of the screen of the display unit. The screen 60 is a screen displayed on the display unit of the UI unit 46 of the robot apparatus 10, for example. On the screen 60, images 62 and 64 are displayed. The images 62 and 64 are deviation information provided by the providing unit 34. The image 62 is an image representing “rocket”, and the image 64 is an image representing “cake”. For example, when the character string “car” is extracted from the conversation of the meeting, an image 62 representing “rocket” and an image 64 representing “cake” having a meaning and impression deviating from the “car” are displayed. . The images 62 and 64 may be displayed according to the above priority, and the images 62 and 64 may be displayed according to the stagnation degree and the deviation degree.

  In the example shown in FIG. 6, two images are displayed as the deviation information. However, one image may be displayed, or three or more images may be displayed. The providing unit 34 may display a plurality of images as deviation information on the display unit at random.

  The providing unit 34 may display an image as deviation information on a device other than the robot device 10 (for example, a user terminal device or a display device), or the image may be displayed on an object (for example, a wall or a table) by a projector. , Screen, etc.). The providing unit 34 may display each image while changing the display location. For example, the providing unit 34 may display each image while changing the display location of each image at a predetermined time interval or at a random time interval.

  The providing unit 34 may change the display mode of the image as the deviation information. For example, the providing unit 34 may display an image in a pop-up format, may display an enlarged or reduced image, may change the display position of each image, and may change the color of each image. You may change and display. The providing unit 34 may display each image while changing the size, display position, and color of the image even in the same image. The providing unit 34 may preferentially display an image with a higher priority than a lower image, or may display an image with a high degree of deviation preferentially over an image with a low degree of priority. Displaying preferentially includes, for example, displaying an enlarged image, displaying near the center of the screen, increasing the display time, displaying first, and the like.

  The providing unit 34 may search for an image related to the character string using the character string as the deviation information as a search key, and provide the image as the deviation information. The search may be performed on an image database, for example, or may be performed using the Internet.

  In the example shown in FIG. 6, only an image is used as the deviation information. However, the providing unit 34 may display a character string as the deviation information, or may emit a voice representing the character string from the speaker. Good. At this time, the providing unit 34 may change the display mode of the character string or may change the volume. Of course, the providing unit 34 may provide a combination of an image and a character string as deviation information.

  When the deviation information is provided as described above, the reaction of the participant is detected by the detection unit 32, and information indicating the reaction is recorded as the history information 56.

  According to the present embodiment, for example, deviation information is provided when the conference is stagnant. Ideas can be triggered by meeting participants referring to deviation information. For example, the user can be influenced by the shape and impression of the object represented by the image as the deviation information, the meaning of the character string as the deviation information, the meaning of the voice, etc., and the idea can be induced.

  Further, when the conference participants themselves prepare the deviation information, the characteristics of the prepared deviation information change depending on the hobbies and preferences of the participants. For example, if a plurality of participants having the same or similar hobbies and preferences prepare deviation information, deviations may occur in the characteristics of the prepared deviation information (for example, deviation information having similar characteristics can be prepared. Even if such deviation information is provided, new ideas are less likely to be triggered. In this embodiment, since the robot apparatus 10 selects and provides the deviation information, it is not necessary for the participant of the conference to prepare the deviation information, and the participant's hobbies and preferences are not affected. Can provide information that is easy to induce ideas.

  Hereinafter, modified examples will be described.

  The providing unit 34 may provide information related to the content of the conference (for example, subject-related information), and may provide deviation information after the conference proceeds. In the initial stage of the conference (for example, within a predetermined time from the start of the conference), the providing unit 34 provides subject-related information (for example, a character string or the like) associated with the conference subject extracted from the content of the conference. Image, sound, etc.) and subject-related information associated with a character string extracted from the conversation of the conference, and then, for example, after a predetermined time has elapsed since the start of the conference, or Deviation information is provided when a stagnation is detected.

  When the subject related information is provided, the providing unit 34 increases the priority order of the character string having a higher appearance frequency in the meeting, and the subject related information associated with the character string having the higher priority order has a lower priority order. You may provide before the theme relevant information matched with the character string. The providing unit 34 may provide the subject-related information by changing it step by step. For example, the providing unit 34 provides subject-related information associated with a character string having the highest priority (a character string having the highest appearance frequency) over a predetermined period, and after the period, The subject-related information associated with the character string with the second highest priority is provided over that period, and the same processing is performed for the character strings with the third highest priority. As a result, the subject-related information that is more relevant to the content of the conference is provided first, and the subject-related information that is less relevant is provided step by step.

  The providing unit 34 may search the subject related information related to the character string using the character string extracted from the conference conversation as a search key and provide the subject related information. In this case, the providing unit 34 may search for the subject related information using only the character string having the high appearance frequency, or may search the subject related information using the character string having the high appearance frequency and the character string having the low appearance frequency. Alternatively, the subject-related information may be searched using only a character string having a low appearance frequency. When only a character string having a high appearance frequency is used, the providing unit 34 searches for subject-related information using, for example, one or a plurality of character strings whose appearance frequency is equal to or higher than a predetermined threshold. When using a plurality of character strings whose appearance frequency is equal to or higher than the threshold, the providing unit 34 may perform an AND search using a combination of the plurality of character strings, or each character string may be used as a single search key. The subject related information may be searched (OR search). When only a character string with a low appearance frequency is used, the providing unit 34 searches for subject-related information using, for example, one or a plurality of character strings whose appearance frequency is less than a predetermined threshold. When using a plurality of character strings whose appearance frequency is less than the threshold, the providing unit 34 may perform an AND search using a combination of the plurality of character strings, or using each character string as a single search key. The subject related information may be searched (OR search). When searching for subject-related information using a character string having a high appearance frequency and a character string having a low appearance frequency, the providing unit 34 may perform an AND search using a combination of the plurality of character strings. The subject related information may be searched using as a single search key (OR search).

  Further, the providing unit 34 may provide the deviation information having a low deviation degree before the deviation information having a high deviation degree. The providing unit 34 may provide the deviation information in stages. For example, the providing unit 34 provides the deviation information having the lowest deviation degree (for example, 1) over a predetermined period, and after the period, the deviation information having the second deviation degree (for example, 2) from the lowest degree is provided. The same processing is performed for deviation information that is provided over the period and the degree of deviation is the third (for example, 3) from the lowest. Thereby, deviation information with a lower degree of deviation from the content of the conference is provided first, and deviation information with a larger degree of deviation is provided step by step. In this way, by providing deviation information in which the degree of deviation becomes larger as the conference progresses, it is possible to cope with a situation where the conference is gradually stagnated.

  The providing unit 34 may change and provide the subject-related information in stages, and may provide the deviation information in stages when the stagnation of the meeting is detected. For example, the providing unit 34 provides the subject-related information in stages according to the priority order according to the appearance frequency, and when the stagnation of the meeting is detected, the provision information 34 is changed in stages according to the order according to the degree of deviation. May be provided. Providing information in this way can provide participants with information relevant to the subject of the meeting to encourage active discussion while the meeting is not stagnant. Provides unexpected information such as deviation information, and can induce new ideas.

  The providing unit 34 may change and provide the deviation information according to the conference participants. For example, in the deviation information management information 54, each deviation information is associated with user identification information for identifying a participant (user). When the participant is identified, the providing unit 34 provides deviation information associated with the user identification information of the identified participant. Thereby, deviation information is changed and provided according to a participant. Moreover, the departure degree may be associated with the user identification information of each participant, and the departure degree may be set for each participant. Information indicating this association is stored in the storage unit 48, for example. When the participant is identified, the providing unit 34 provides deviation information associated with the degree of deviation associated with the user identification information of the identified participant. As described above, the providing unit 34 may select and provide deviation information using the participant history information. Participants are identified by, for example, image recognition processing, voice recognition processing, and the like.

  The providing unit 34 may change and provide deviation information according to a combination of a plurality of participants in the conference. For example, in the deviation information management information 54, each piece of deviation information is associated with a plurality of pieces of user identification information, and when a plurality of participants are identified, the providing unit 34 identifies the users of the plurality of identified participants. Deviation information associated with identification information is provided. Thereby, deviation information is changed and provided according to a plurality of participants.

  The robot apparatus 10 is realized by cooperation of hardware and software as an example. Specifically, the robot apparatus 10 includes one or more processors such as a CPU (not shown). The function of each unit of the robot apparatus 10 is realized by the one or more processors reading and executing a program stored in a storage device (not shown). The program is stored in the storage device via a recording medium such as a CD or DVD, or via a communication path such as a network. As another example, each unit of the robot apparatus 10 may be realized by hardware resources such as a processor, an electronic circuit, and an ASIC (Application Specific Integrated Circuit). In the realization, a device such as a memory may be used. As yet another example, each unit of the robot apparatus 10 may be realized by a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or the like.

DESCRIPTION OF SYMBOLS 10 Robot apparatus, 28 Status collection part, 30 Recognition processing part, 32 Detection part, 34 Provision part, 36 Setting part, 38 History management part.

Claims (14)

Detection means for detecting the status of the meeting;
Providing means for providing deviation information deviating from the subject of the meeting;
A robot apparatus having: If the detecting means detects a stagnation of the meeting, the providing means provides the deviation information;
The robot apparatus according to claim 1. The providing means changes and provides the deviation information according to the degree of stagnation of the conference.
The robot apparatus according to claim 2. The detection means detects the stagnation of the conference based on at least one of the conversation of the participant of the conference, the facial expression of the participant, and the length of the silence time.
The robot apparatus according to claim 2 or claim 3, wherein The providing means provides the deviation information in between while providing information in line with the subject of the meeting;
The robot apparatus according to any one of claims 1 to 4, wherein the robot apparatus is characterized. The providing means changes and provides the deviation information according to the degree of deviation indicating the degree of deviation from the content of the meeting.
The robot apparatus according to claim 5. The providing means switches between providing information related to the subject of the meeting and providing the deviation information according to the degree of deviation.
The robot apparatus according to claim 5. The providing means provides information related to the subject of the meeting and not appearing in the meeting when the degree of departure is less than or equal to a threshold, and when the degree of departure is larger than the threshold, the deviation information As offered,
The robot apparatus according to claim 7. The providing means changes and provides the deviation information according to participants of the conference.
The robot apparatus according to any one of claims 1 to 4, wherein the robot apparatus is characterized. The providing means changes and provides the deviation information according to a combination of a plurality of participants in the conference.
The robot apparatus according to claim 9. History management means for managing the history of the participant's response to the deviation information;
The providing means changes and provides the deviation information according to the history;
The robot apparatus according to any one of claims 1 to 4, wherein the robot apparatus is characterized. For each character string, it further comprises storage means for storing the character string and the deviation information in association with each other,
The providing means provides the deviation information associated with a character string detected from a conversation of the meeting.
The robot apparatus according to any one of claims 1 to 11, wherein the robot apparatus is characterized. The deviation information includes at least one of a character string, an image, and a sound.
The robot apparatus according to any one of claims 1 to 12, wherein the robot apparatus is characterized. Computer
Detection means for detecting the status of the meeting,
Providing means for providing deviation information deviating from the subject of the meeting;
Program to function as.