JP2022050924A - Information processing device, information processing method, and information processing program - Google Patents

Abstract

To provide an information processing device, an information processing method, and an information processing program for analyzing communication without requiring a condition that two or more users are close to each other for a certain period of time or longer.SOLUTION: There provided is an information processing device, which includes: a recognition unit that recognizes whether or not at least two or more users have communicated based on voice data of a plurality of users; a calculation unit that compares a first feature amount which is the feature amount of each of the two or more users calculated from the data during communication of the two or more users that are recognized by the recognition unit as having communicated with a second feature amount which is the feature amount of each of the two or more users calculated from the data in the past communication of the two or more users, and calculates a degree of synchronization indicating the degree of synchronization of the two or more users based on a result of the comparison obtained for each of the two or more users.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information processing apparatus, an information processing method and an information processing program.

In recent years, techniques for evaluating communication using sensor devices have been developed. For example, in Patent Document 1, a technique of identifying a community in an organization in which the user participates based on the user's location information and associating the identification information of the community with the identification information of all users included in the community. Etc. are disclosed. Further, in Patent Document 2, a technique for determining whether or not communication has been established between users on the condition that two or more users have been continuous for a predetermined time or more within a predetermined distance based on the user's position information. Etc. are disclosed.

Japanese Unexamined Patent Publication No. 2018-37073 Japanese Patent No. 6533868

However, the above-mentioned technique requires that two or more users are in close contact with each other for a certain period of time depending on whether or not communication has been achieved. Therefore, for example, when the approach time such as greeting is short, the above-mentioned technique does not determine that communication has been achieved between users. That is, there was room for improvement in order to analyze short-term communication such as greetings.

Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to analyze communication without requiring a condition that two or more users are close to each other for a certain period of time or longer. It is to provide possible new and improved information processing devices, information processing methods and information processing programs.

In order to solve the above problems, according to a certain viewpoint of the present invention, a recognition unit that recognizes whether or not at least two or more users have communicated based on voice data of a plurality of users, and a recognition unit that communicates with the recognition unit. The first feature amount, which is the feature amount of each of the two or more users calculated from the data during the communication of the two or more users recognized as having attempted, and the past communication of the two or more users. The second feature amount, which is the feature amount of each of the two or more users calculated from the data, is compared with the second feature amount, and based on the result of the comparison obtained for each of the two or more users, the two or more users. Provided is an information processing apparatus including a calculation unit for calculating a degree of synchronization indicating the degree of synchronization.

The calculation unit may calculate the degree of entrainment indicating a higher degree as the difference between the comparison results obtained for each of the two or more users is smaller.

The second feature amount may indicate a feature amount calculated only from data in the past communication generated between the two or more users.

In the information processing device, a storage unit that stores two character strings in association with each other, and at least one user speaks a word indicated by one of the two character strings, and the other of the two character strings speaks. Further, a determination unit for determining whether or not a user different from the one or more users has spoken the indicated word is further provided, and when it is determined that the determination unit has spoken, at least two or more users communicate with the recognition unit. It may be recognized that the attempt was made.

The second feature amount may indicate a feature amount calculated only from data in the past communication when one of the words indicated by the two character strings is spoken by the two or more users.

The two strings may indicate a greeting or a call that occurs at the beginning of a conversation.

The first feature amount and the second feature amount may include the sound pressure level, fundamental frequency or utterance speed of words spoken by the two or more users.

The information processing device further includes a detection unit that detects whether or not at least two or more users have approached based on the position information of a plurality of users, and the recognition unit is detected to have approached by the detection unit. It may be recognized whether or not at least two or more users have a conversation based on the voice data of the user.

The position information of the plurality of users may include information indicated by positioning data for positioning the positions of the plurality of users.

The information processing device may further include an estimation unit that estimates that the relationship between the two or more users has changed when the degree of synchronization meets a predetermined criterion.

The predetermined criterion may include the case where the difference between the tuning degree and the past tuning degree or the average value of the past tuning degree exceeds the predetermined value.

Further, in order to solve the above-mentioned problems, according to another viewpoint of the present invention, it is recognized whether or not at least two or more users have communicated with each other based on the voice data of a plurality of users, and communication is attempted. The first feature amount of each of the two or more users calculated from the data in the communication of the two or more users recognized as having been recognized, and the data calculated from the data in the past communication of the two or more users. A degree of synchronization indicating the degree to which the two or more users are synchronized based on the result of the comparison obtained for each of the two or more users by comparing with the second feature amount of each of the two or more users. Is provided, and information processing methods performed by the computer, including.

Further, in order to solve the above problems, according to another viewpoint of the present invention, the computer is a recognition unit that recognizes whether or not at least two or more users have communicated with each other based on the voice data of a plurality of users. , The first feature amount of each of the two or more users calculated from the data during the communication of the two or more users recognized as having communicated by the recognition unit, and the past communication of the two or more users. The second or more users are compared with the second feature amount of each of the two or more users calculated from the data in the data, and the two or more users are synchronized based on the result of the comparison obtained for each of the two or more users. An information processing program is provided to function as a calculation unit for calculating the degree of synchronization indicating the degree of synchronization.

As described above, according to the present invention, it is possible to analyze communication without requiring the condition that two or more users are close to each other for a certain period of time or longer.

It is explanatory drawing which shows an example of the schematic structure of the information processing system which concerns on this embodiment. It is a block diagram which shows the functional structure of the server 10 which concerns on this embodiment. It is explanatory drawing for demonstrating an example of the calculation method of the degree of tuning. It is explanatory drawing for demonstrating an example of the method of estimating whether or not there was a change in a relationship between users. It is explanatory drawing for demonstrating an example of the operation flow of the server 10 which concerns on this embodiment. It is a block diagram which showed the hardware configuration of the server 10 which concerns on this embodiment.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, and duplicate description will be omitted.

Further, in the description of the present specification and the drawings, the information terminal owned by the first user U1 is the information terminal 5A, and the information terminal owned by the second user U2 is the information terminal 5B, as necessary. Distinguish information terminals. However, when it is not necessary to distinguish each information terminal, each information terminal is simply referred to as an information terminal 5.

<1. Information processing system overview>
An embodiment of the present invention relates to an information processing system capable of analyzing communication without requiring a condition that two or more users are in close proximity for a certain period of time or longer. Hereinafter, an outline of the information processing system will be described with reference to FIG.

FIG. 1 is an explanatory diagram showing an example of a schematic configuration of an information processing system according to the present embodiment. As shown in FIG. 1, the information processing system according to the embodiment of the present invention includes a positioning unit 3, an information terminal 5, and a server 10. Further, the user U1 and the user U2 indicate a user who is trying to communicate.

(network)
The network 1 connects the positioning unit 3, the information terminal 5, and the server 10. For example, the network 1 may include a public line network such as a telephone line network, the Internet, and a communication satellite network, a LAN (Local Area Network), a WAN (Wide Area Network), and the like. Further, the network 1 may include a dedicated line network such as IP-VPN (Internet Protocol-Virtual Private Network).

(Positioning department)
The positioning unit 3 has a function of positioning the position of the information terminal 5. The positioning unit 3 transmits positioning data indicating the positioning result to the server 10 via the network 1. For example, the positioning unit 3 receives a signal including identification information unique to each information terminal 5 emitted from the information terminal 5, and the position of the information terminal 5 is determined based on the reception strength of the identification signal. Positioning.

(Information terminal)
The information terminal 5 has a function of acquiring voice data of a user, a function of transmitting a signal including identification information of the information terminal 5 at regular intervals, and the like. The function of acquiring voice data may include, for example, a microphone. Further, the function of transmitting a signal including the identification information may include, for example, a function of transmitting BLE (Bluetooth Low Energy). The function of acquiring voice data and the function of transmitting a signal including identification information may be separately configured. Further, the information terminal 5 transmits the acquired voice data to the server 10 via the network 1. Further, the information terminal 5 is a terminal owned by each user, and may be, for example, a tablet terminal, a smartphone, or a wearable terminal.

(server)
The server 10 has a function of analyzing data, a function of transmitting and receiving data, a function of storing data, and the like. For example, in the present embodiment, the server 10 has a function of recognizing whether or not the user has communicated based on the data obtained from the positioning unit 3 and the data obtained from the information terminal 5, and the user synchronizes with the function. It has a function to calculate the degree of synchronization, which indicates the degree of presence.

In the present embodiment, among the above-mentioned information processing systems, the server 10 is particularly creatively devised. Hereinafter, the configuration and operation of the server 10 according to the present embodiment will be sequentially described in detail.

<2. Functional configuration of server 10 according to this embodiment>
FIG. 2 is a block diagram showing a functional configuration of the server 10 according to the present embodiment. As shown in FIG. 2, the server 10 includes a communication unit 11, a user detection unit 15, a voice recognition unit 19, a pair determination unit 23, an information analysis unit 27, a synchronization degree calculation unit 31, and a storage unit. It has 35 and a relationship estimation unit 39.

The communication unit 11 has a function of communicating with the positioning unit 3 or the information terminal 5 via the network 1. For example, the communication unit 11 receives positioning data from the positioning unit 3 and voice data from the information terminal 5.

The user detection unit 15 is an example of a detection unit, and has a function of detecting whether or not two or more users have approached each other. For example, the user detection unit 15 detects whether or not two or more users have approached each other based on the positioning data received from the positioning unit 3.

The voice recognition unit 19 is an example of a recognition unit, and has a function of recognizing character data indicating words spoken by a user based on voice data received from an information terminal 5 by a communication unit 11. Further, the voice recognition unit 19 may recognize the number of mora indicating the segment unit of the sound based on the words uttered by the user. For example, when the voice recognition unit 19 communicates with two or more users who are detected to be closer by the user detection unit 15, the voice recognition unit 19 is character data indicating words spoken by the two or more users. Is recognized, and whether or not the user has communicated is recognized.

Here, an example of recognizing whether or not two or more users have communicated will be described with reference to FIG. First, the user detection unit 15 detects that the user U1 and the user U2 have approached each other based on the positioning data received from the positioning unit 3. Next, when voice is input to the information terminal 5A owned by the user U1 and the information terminal 5B owned by the user U2, the information terminal 5A and the information terminal 5B are the communication unit 11 of the server 10. Send audio to. Here, when the sound pressure level of the voice received from the information terminal 5A and the information terminal 5B both exceeds a predetermined value, the voice recognition unit 19 recognizes that the user U1 and the user U2 have spoken to each other. It is recognized that communication has been attempted between the users.

The pair determination unit 23 is an example of the determination unit, and is a word indicated by one of two character strings associated with each other stored in the storage unit 35, which will be described later, based on the character data recognized by the voice recognition unit 19. Is issued by one or more users, and has a function of determining whether or not a user different from the one or more users has issued the word indicated by the other of the two character strings. Hereinafter, the two character strings stored in the storage unit 35 and associated with each other are referred to as a greeting pair.

For example, the storage unit 35 stores a character string indicating a plurality of greetings and a character string indicating a response to the greeting in association with each other. Here, when one or more users utter a word indicating one of the greetings, and a user different from the one or more users utters a word indicating a response associated with the greeting, the pair determination unit 23 Determines that two or more users have spoken the words indicated by the greeting pair. On the other hand, when one or more users do not speak either the word indicated by the greeting or the word indicated by the response to the greeting, the pair determination unit 23 emits the word indicated by the greeting pair by two or more users. It is determined that there was no such thing.

The information analysis unit 27 analyzes the feature amount based on the data received from the information terminal 5 by the communication unit 11. The information analysis unit 27 analyzes the feature quantities such as the sound pressure level, the fundamental frequency, and the utterance speed based on the voice data received from the information terminal 5, for example. The information analysis unit 27 may analyze the feature amount based on a known method. For example, the information analysis unit 27 may analyze the utterance speed based on the number of mora recognized from the words spoken by the user and the voice detection time of the words spoken by the user.

The synchronization degree calculation unit 31 is an example of the calculation unit, and calculates the synchronization degree indicating the degree of synchronization of two or more users based on the feature amount analyzed by the information analysis unit 27. For example, if a user U1 is not in good physical condition and the user U1 greets another user with a smaller voice volume than usual, the user U2 who cares about the user U1 and returns a greeting with a smaller voice volume than usual. May exist, and there may also be a user U3 who returns a greeting with the usual voice volume. Here, it is considered that the user U2 who returns the greeting with a voice volume smaller than usual is more in tune with the user U1 than the user U3 who returns the greeting with the usual voice volume. Therefore, the synchronization degree calculation unit 31 calculates the synchronization degree between the user U1 and the user U2 higher than the synchronization degree between the user U1 and the user U3. Hereinafter, the method of calculating the degree of synchronization will be described more specifically with reference to FIG.

FIG. 3 is an explanatory diagram for explaining an example of a method for calculating the degree of synchronization. Hereinafter, a method of calculating the degree of entrainment when two or more users greet each other will be described, but the entrainment degree calculation unit 31 may calculate the degree of entrainment from conversation without being limited to greetings.

First, the talking user 51A shows the user who greeted the user, and the responding user 51B shows the user who returned the greeting to the talking user 51A. The greeting-speaking 55A indicates the words spoken by the speaking user 51A, and the greeting response 55B indicates the words spoken by the responding user 51B. The greeting pair determined by the pair determination unit 23 indicates the greeting talk 55A and the greeting response 55B associated therewith. Next, the sound pressure level 59 indicates the value of the sound pressure level of the words spoken by the user. The average sound pressure level 63 indicates an average value of the sound pressure levels of words spoken by the user in the past. Next, the sound pressure level variation 67 shows the result of comparison between the sound pressure level 59 and the average sound pressure level 63. The average sound pressure level 63 indicates a comparison target with the sound pressure level 59, but the comparison target is the sound pressure level of the words spoken by the user in the latest past instead of the average sound pressure level 63. It may be a value.

The tuning degree calculation unit 31 calculates the tuning degree from the difference between the sound pressure level fluctuation 67A of the speaking user 51A and the sound pressure level fluctuation 67B of the responding user 51B. For example, the tuning degree calculation unit 31 calculates a higher tuning degree as the difference between the sound pressure level fluctuations of both users is smaller.

First, the degree of synchronization calculated based on the No. 1 row will be described. The sound pressure level variation 67A of the talking user 51A indicates "-5 dB", and the sound pressure level variation 67B of the user 51B also indicates "-5 dB". Therefore, the difference between the sound pressure level fluctuation 67A of the speaking user 51A and the sound pressure level fluctuation 67B of the responding user 51B is "0", and the degree of synchronization is calculated to be the highest.

Next, the degree of synchronization calculated based on the row of No. 3 will be described. The sound pressure level variation 67A of the user 51A indicates "-8 dB", and the sound pressure level variation 67B of the user 51B indicates "+ 2 dB". Therefore, the difference between the sound pressure level fluctuation 67A of the speaking user 51A and the sound pressure level fluctuation 67B of the responding user 51B is "10 dB", and when compared with the tuning degree calculated based on the row No. 1, the row No. 3 The degree of entrainment calculated based on this is low.

The tuning degree calculation unit 31 may quantitatively calculate the tuning degree according to the magnitude of the difference between the sound pressure level fluctuation 67A and the sound pressure level fluctuation 67B. For example, when the difference is "0 to 3 dB", the tuning degree indicates "3", when the difference is "4 to 6 dB", the tuning degree indicates "2", and when the difference is "7 dB to", the tuning degree is synchronized. The degree may indicate "1". Further, although the example of calculating the tuning degree using the sound pressure level as the feature amount has been mainly described, the tuning degree calculation unit 31 may calculate the tuning degree using another feature amount, or may have a plurality of tuning degrees. The degree of synchronization may be calculated using the feature amount. For example, the tuning degree calculation unit 31 calculates the difference in the feature amount between the users for each of the fundamental frequency and the utterance speed in addition to the sound pressure level, and the tuning degree is based on the calculated difference between the plurality of feature amounts. May be calculated.

Further, the average sound pressure level 63 may indicate an average value of sound pressure levels analyzed only from voice data during past communication generated between the same users. Further, the average sound pressure level 63 may indicate the average value of the sound pressure levels analyzed only from the voice data during the past communication when the user utters the words indicated by the same greeting pair. Further, the average sound pressure level 63 may indicate the average value of the sound pressure levels analyzed only from the voice data during the past communication in which the words indicated by the same greeting pair are spoken among the same users.

Here, the description of the server 10 according to the present embodiment will be returned with reference to FIG. The storage unit 35 has a function of holding software and various data. For example, the storage unit 35 holds two associated character strings and holds the feature amount analyzed by the information analysis unit 27.

The relationship estimation unit 39 is an example of the estimation unit, and has a function of estimating whether or not there is a change in the relationship between users based on the synchronization degree calculated by the synchronization degree calculation unit 31. Hereinafter, a method of estimating whether or not there is a change in the relationship between users will be described with reference to FIG.

FIG. 4 is an explanatory diagram for explaining an example of a method of estimating whether or not there is a change in the relationship between users. The horizontal axis of the graph of FIG. 4 shows the date and time when the user U1 and the user U2 communicate with each other, and the vertical axis shows the high degree of synchronization between the user U1 and the user U2. That is, the graph of FIG. 4 shows the time-series change of the degree of entrainment between the user U1 and the user U2.

For example, if there is no change in the relationship between the user U1 and the user U2, it is considered that communication can be achieved in the same manner as usual, so that the synchronization degree calculation unit 31 calculates the synchronization degree between the users to be high. obtain. Further, when there is a change in the relationship between the user U1 and the user U2, it is considered that either the user U1 or the user U2 can communicate in a different tone from usual, so that the synchronization degree calculation unit 31 is used. , The degree of synchronization between the users can be calculated low. In this case, when the difference between the synchronization degree of the user U1 and the user U2 and the past synchronization degree of the user U1 and the user U2 or the average value of the past synchronization degree exceeds a predetermined value, the user U1 and the user U2 When there is a change in the relationship, it is possible that either the user U1 or the user U2 communicates in a different tone than usual, so that the relationship estimation unit 39 determines the relationship between the users. It may be presumed that there has been a change.

Hereinafter, the difference A between the tuning degree calculated from the greeting performed at t1 and the tuning degree calculated from the greeting performed at t2 will be described as a value equal to or less than a predetermined value. Further, the difference B between the tuning degree calculated from the greeting performed at t2 and the tuning degree calculated from the greeting performed at t3 will be described as a value exceeding a predetermined value.

First, since the difference A between the synchronization degree calculated from the greeting performed at t1 and the synchronization degree calculated from the greeting performed at t2 is equal to or less than a predetermined value, the relationship estimation unit 39 may use the user U1 and the user. It is estimated that there was no change in the relationship with U2. Further, since the difference B between the tuning degree calculated from the greeting performed at t2 and the tuning degree calculated from the greeting performed at t3 exceeds a predetermined value, the relationship estimation unit 39 uses the user U1. It is presumed that there was a change in the relationship between the user and the user U2. Here, the past synchronization degree indicates the latest past synchronization degree, but may indicate the average value of a plurality of past synchronization degrees. For example, the difference C indicates the difference between the tuning degree calculated from the greeting performed at t7 and the average value P1 of each tuning degree calculated from the greetings performed at t4 to t6. In this case, the relationship estimation unit 39 estimates whether or not there is a change in the relationship between the users based on the difference C between the average value P1 and the degree of synchronization calculated from the greeting performed at t7. May be good.

The functional configuration of the server 10 according to the present embodiment has been described above. Subsequently, the operation of the information processing system according to the present embodiment will be described with reference to FIG.

<3. An example of the operation flow of the server 10 according to this embodiment>
FIG. 5 is an explanatory diagram for explaining an example of the operation flow of the server 10 according to the present embodiment. First, based on the positioning data received from the positioning unit 3, the user detection unit 15 detects that two or more users have approached (S01). Then, the communication unit 11 receives the voice data from the information terminal 5 (S05), and the voice recognition unit 19 recognizes the voice data received by the communication unit 11 as character data (S09).

Next, the pair determination unit 23 determines whether or not two or more users have issued any greeting pair held in the storage unit 35 based on the plurality of character data recognized by the voice recognition unit 19. (S13). When the pair determination unit 23 determines that any greeting pair is issued by two or more users, the process proceeds to S17 (S13 / Yes), and the pair determination unit 23 has two or more users in each of the greeting pairs. If it is determined that no emission has occurred, the process proceeds to S01 (S13 / No).

When the pair determination unit 23 determines that any greeting pair is issued by two or more users (S13 / Yes), the voice recognition unit 19 outputs the voice data to the information analysis unit 27 (S17). Then, the information analysis unit 27 analyzes the sound pressure level included in the voice data (S21).

Subsequently, the tuning degree calculation unit 31 compares the value indicated by the sound pressure level analyzed in S21 with the average value of the past sound pressure levels of the greeting pairs determined by S13 to be issued by two or more users. Then, the degree of synchronization is calculated based on the difference as a result of comparison of each user (S25).

Then, the storage unit 35 stores the synchronization degree calculated by the synchronization degree calculation unit 31 (S29), and the server 10 ends the operation of the information processing system.

The operation of the server 10 according to the present embodiment has been described above. Subsequently, the action and effect of this embodiment will be described.

<4. Action and effect of this embodiment>
According to the present embodiment described above, various effects can be obtained. For example, according to the present embodiment, even when the communication between two or more users is a short-time communication such as a greeting, the server 10 can effectively analyze the communication. Is. More specifically, in the present embodiment, the synchronization degree calculation unit 31 is analyzed from the respective feature quantities analyzed from the greeting data of two or more users and the past greeting data of the two or more users. Each feature amount is compared, and the degree of synchronization is calculated based on the result of the comparison. As another method, it is possible to analyze the relationship between two or more users by focusing only on the data of one greeting, but since the amount of information that can be obtained only from the data of one greeting is limited, the other method. It is difficult to obtain useful analysis results using the above method.

On the other hand, in the present embodiment, the feature amount analyzed from the new greeting data and the feature amount analyzed from the past greeting data as described above are used in contrast. Therefore, it is not necessary for multiple users to be close to each other for a certain period of time, and two or more users greet in the same tone as usual, or two or more users greet in a different tone than usual. It is possible to obtain a degree of entrainment that indicates whether there was a tendency for entrainment among users in a way different from the usual greeting.

Further, the synchronization degree calculation unit 31 analyzes each feature amount analyzed from the data of two or more users who have attempted communication, and the past data in the communication generated between the same user as the two or more users. The degree of synchronization is calculated based on the result of the comparison by comparing each feature amount. Here, the normal communication tone of a certain user may differ depending on the other user. Therefore, by using the feature amount analyzed from the past data during the communication generated between the same users as the comparison target as described above, as a result of the comparison, the communication generated between the same users is usually different from the usual one. It is possible to get the difference more accurately. As a result, the tuning degree calculation unit 31 can improve the tuning degree calculation accuracy.

Further, the entrainment degree calculation unit 31 is based on each feature amount analyzed from the voice data of two or more users who have communicated, and the past voice data of the same greeting pair as the words uttered by the two or more users. Each of the analyzed features is compared, and the degree of synchronization is calculated based on the result of the comparison. Here, the normal communication tone of a user may differ depending on the content of the greeting. Therefore, by using the features analyzed from the past voice data of the same greeting pair as the comparison target as described above, as a result of the comparison, the difference from the usual communication in which the words indicated by the same greeting pair are spoken. Can be obtained more accurately. As a result, the tuning degree calculation unit 31 can improve the tuning degree calculation accuracy.

Further, the user detection unit 15 detects whether or not the user is approaching, so that the voice recognition unit 19 can improve the accuracy of recognizing whether or not communication has been achieved.

Further, since the relationship estimation unit 39 estimates whether or not there has been a change in the relationship between the users, for example, the server 10 can grasp the problem that has occurred between the users.

The operation and effect of the server 10 according to the present embodiment have been described above. The above-mentioned information processing such as analysis of feature amount and calculation of synchronization degree is realized by cooperation between software and hardware of server 10. The hardware configuration of the server 10 according to the present embodiment will be described with reference to FIG.

<5. Hardware configuration of server 10 according to this embodiment>
FIG. 6 is a block diagram showing a hardware configuration of the server 10 according to the present embodiment. The server 10 includes a CPU (Central Processing Unit) 102, a ROM (Read Only Memory) 104, a RAM (Randome Access Memory) 106, an internal bus 108, an input / output interface 110, a display unit 112, and an input unit 113. The audio output unit 114, the storage unit 115, the drive 116, the network interface 117, and the external interface 118 can be provided.

The CPU 102 functions as an arithmetic processing unit and a control device, and controls the overall operation in the server 10 according to various programs. By the CPU 102 collaborating with the ROM 104, the RAM 106, and the software described later, functions such as a voice recognition unit 19, a synchronization degree calculation unit 31, and a relationship estimation unit 39 can be realized.

The ROM 104 stores programs, calculation parameters, and the like used by the CPU 102. The RAM 106 temporarily stores a program used in the execution of the CPU 102, parameters that are appropriately changed in the execution, and the like.

The CPU 102, ROM 104, and RAM 106 are connected to each other by an internal bus 108, and further, a display unit 112, an input unit 113, an audio output unit 114, a storage unit 115, a drive 116, and a network interface 117, which will be described later, are connected to each other via an input / output interface 110. Connected to the external interface 118.

The display unit 112 is, for example, a display device such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD), or an OLED (Organic Light Emitting Diode) device, and converts video data into video and outputs the video data. Further, the input unit 113 generates an input signal based on the input means for the member to input information such as a mouse, keyboard, touch panel, button, microphone, sensor, switch and lever, and the input by the member, and outputs the input signal to the CPU 102. It may be composed of an input control circuit or the like. Further, the audio output unit 114 is an audio output device such as a speaker and headphones, and converts audio data and the like into audio and outputs the audio data.

The storage unit 115 is a data storage device configured as an example of the storage unit 35 according to the present embodiment. The storage unit 115 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, a deleting device for deleting data recorded on the storage medium, and the like. The storage unit 115 is composed of, for example, an HDD (Hard Disk Drive), an SSD (Solid Storage Drive), or a memory having an equivalent function. The storage unit 115 drives the storage and stores programs and various data executed by the CPU 102.

The drive 116 is a reader / writer for a storage medium, and is built in or externally attached to the server 10. The drive 116 reads information stored in a removable storage medium such as a mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 106. The drive 116 can also write information to the removable storage medium.

The network interface 117 is a communication interface composed of devices for connecting to a communication network such as the Internet, for example. Further, the network interface 117 may be a wired LAN (Local Area Network) or a wireless LAN compatible communication device, or may be a wire communication device that performs wired communication.

The external interface 118 is composed of a connection port for connecting an external connection device such as a USB (Universal Serial Bus) port, an IEEE1394 port, a SCSI (Small Computer System Interface) port, an RS-232C port, or an optical audio terminal. It is a connected connection interface.

<6. Supplement>
Although the preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, the present invention is not limited to these examples. It is clear that a person having ordinary knowledge in the field of technology to which the present invention belongs can come up with various modifications or modifications within the scope of the technical ideas described in the claims. , These are also naturally understood to belong to the technical scope of the present invention.

For example, each user who uses the information processing system according to the present embodiment may be classified into one of a plurality of groups. For example, when the user U1 communicates with the user U2 belonging to the group B, the synchronization degree calculation unit 31 is the average value of the feature quantities analyzed from the data in the past communication between the user U1 and the user belonging to the group B. May be used as a comparison target to calculate the degree of synchronization. For example, when the group B to which the user U2 who is the boss of the user U1 belongs is the section chief group, the synchronization degree calculation unit 31 uses the average value of the feature quantities of the user U1 and all the users belonging to the section chief group as a comparison target. It is used to calculate the degree of synchronization between the user U1 and the user U2. In this case, the server 10 can estimate whether or not the relationship between the user U1 and the user U2 is average as compared with the relationship between the user U1 and the entire section chief group.

Further, the steps in the processing of the operation of the server 10 according to the present embodiment do not necessarily have to be processed in chronological order in the order described as the explanatory diagram. For example, each step in the processing of the operation of the server 10 may be processed in an order different from the order described in the explanatory diagram, or may be processed in parallel.

Further, the user detection unit 15 may detect whether or not two or more users are approaching from the video data obtained by shooting the user. According to such a configuration, the user detection unit 15 can detect whether or not two or more users are approaching even when the positioning unit 3 is not provided.

Further, the information analysis unit 27 may analyze the feature amount of the facial expression or the movement based on the video data obtained by photographing the user. The information analysis unit 27 may analyze the height of the corner of the mouth or the angle of the bow, for example, as a facial expression analysis. As a result, the information analysis unit 27 can analyze the communication only from the gesture. In addition, the information analysis unit 27 can analyze changes in the user's emotions. Further, the tuning degree calculation unit 31 can calculate the tuning degree with high accuracy by combining the feature amount analyzed from the audio data and the feature amount analyzed from the video data.

Further, when the relationship estimation unit 39 estimates that the relationship between users has changed, the communication unit 11 may send an alert to the user. In this case, the user who received the alert can grasp that it is estimated that the relationship with the other user has changed.

Further, the server 10 may further include, for example, an estimator that learns the relationship between the change in the degree of synchronization calculated in the past and the change in the relationship between users at the same time point when the degree of synchronization is calculated. good. Then, the relationship estimation unit 39 may estimate whether or not the relationship between users changes by using the estimator trained in the relationship. As a result, the server 10 can predictively grasp whether or not the relationship between users deteriorates.

Further, it is possible to create a computer program for causing the hardware such as the CPU, ROM, and RAM built in the server 10 to exhibit the same functions as the above-mentioned configurations of the server 10.

1 Network 3 Positioning unit 5 Information terminal 10 Server 11 Communication unit 15 User detection unit 19 Voice recognition unit 23 Pair judgment unit 27 Information analysis unit 31 Synchronization degree calculation unit 35 Storage unit 39 Relationship estimation unit

Claims (13)

A recognition unit that recognizes whether or not at least two or more users have communicated based on the voice data of multiple users.
The first feature amount, which is the feature amount of each of the two or more users calculated from the data during the communication of the two or more users recognized as having communicated by the recognition unit, and the two or more users. Compared with the second feature amount, which is the feature amount of each of the two or more users calculated from the data in the past communication, and based on the result of the comparison obtained for each of the two or more users. A calculation unit that calculates the degree of synchronization, which indicates the degree of synchronization of the two or more users.
An information processing device equipped with. The information processing apparatus according to claim 1, wherein the calculation unit calculates a degree of synchronization indicating a higher degree as the difference between the comparison results obtained for each of the two or more users is smaller. The information processing apparatus according to claim 1 or 2, wherein the second feature amount indicates a feature amount calculated only from data during past communication generated between the two or more users. The information processing device is
A storage unit that stores two character strings in association with each other,
Determination of whether or not at least one user has spoken a word indicated by one of the two character strings, and a user different from the one or more users has spoken the word indicated by the other of the two character strings. Department and
Further prepare
The information processing apparatus according to any one of claims 1 to 3, wherein when it is determined that the determination unit has spoken, the recognition unit recognizes that at least two or more users have communicated with each other. The second feature amount indicates the feature amount calculated only from the data in the past communication when one of the words indicated by the two character strings is spoken by the two or more users. The information processing device described in. The information processing apparatus according to claim 4 or 5, wherein the two character strings indicate a greeting word or a calling word generated at the beginning of a conversation. The first feature amount and the second feature amount according to any one of claims 1 to 6, which include the sound pressure level, fundamental frequency or utterance speed of words spoken by the two or more users. Information processing device. The information processing device is
A detector that detects whether or not at least two or more users have approached based on the location information of multiple users.
Further prepare
The one according to any one of claims 1 to 7, wherein the recognition unit recognizes whether or not at least two or more users have a conversation based on the voice data of the user detected to be closer by the detection unit. Information processing equipment. The information processing apparatus according to claim 8, wherein the position information of the plurality of users includes information indicated by positioning data for positioning the positions of the plurality of users. The information processing device is
An estimation unit that estimates that the relationship between two or more users has changed when the degree of synchronization meets a predetermined criterion.
The information processing apparatus according to any one of claims 1 to 9, further comprising. The information processing apparatus according to claim 10, wherein the predetermined reference includes a case where the difference between the tuning degree and the past tuning degree or the average value of the past tuning degree exceeds a predetermined value. Recognizing whether at least two or more users have communicated based on the voice data of multiple users.
Calculated from the first feature amount of each of the two or more users calculated from the data in the communication of the two or more users recognized as having communicated, and the data in the past communication of the two or more users. The degree to which the two or more users are synchronized is determined based on the result of the comparison obtained for each of the two or more users by comparing with the second feature amount of each of the two or more users. To calculate the degree of entrainment shown and
Information processing methods performed by a computer, including. Computer,
A recognition unit that recognizes whether or not at least two or more users have communicated based on the voice data of multiple users.
The first feature amount of each of the two or more users calculated from the data during the communication of the two or more users recognized as having communicated by the recognition unit, and the past communication of the two or more users. The second or more users are compared with the second feature amount of each of the two or more users calculated from the data in the above, and the two or more users are synchronized based on the result of the comparison obtained for each of the two or more users. A calculation unit that calculates the degree of synchronization, which indicates the degree of synchronization.
An information processing program to function as.

Images