JP2023004833A - Computer system, method and program for improving manner how to relate to each other in one-to-one communication - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a computer system, method and program for supporting one-to-one communication, capable of improving quality of the one-to-one communication.

SOLUTION: A computer system for supporting one-to-one communication is configured to: receive information representing a response that a first person out of two persons expects a second person out of the two persons, in the communication; receive voices and/or images of the communication; derive advice in the communication based on the information, the voices and/or the images; and provide the derived advice during the communication.

SELECTED DRAWING: Figure 2

COPYRIGHT: (C)2023,JPO&INPIT

Description

TECHNICAL FIELD The present invention relates to a computer system, method, and program for supporting communication, and more specifically, a computer system for supporting communication between two parties (between a first person and a second person). , methods and programs.

Interviews between subordinates and superiors are often held in organizations. This is done, for example, for purposes such as setting goals for subordinates, managing progress, and reporting results. A system for supporting such an interview is known (Patent Document 1).

Among such interviews, there is an interview called "1on1 meeting" (or "1on1") conducted one-on-one between a superior and a subordinate. "1on1" is intended to promote the growth of subordinates by having subordinates take the initiative and have a dialogue with their superiors.

JP 2006-190162 A

The inventor of the present invention believes that in communication between two parties, including "1on1", if one of the two can be encouraged to have an appropriate relationship with the other, communication between two parties I thought the quality should improve.

An object of the present invention is to provide a computer system, method, and program for supporting communication between two parties in order to improve the quality of communication between the two parties.

The present invention provides, for example, the following items.
(Item 1)
A computer system for supporting communication between two parties,
a first receiving means for receiving information indicating a response that a first person of the two parties expects of a second person of the two parties in the communication;
a second receiving means for receiving audio and/or video during said communication;
derivation means for deriving advice in the communication based on the information and the voice and/or the video;
providing means for providing said derived advice during said communication.
(Item 2)
2. The method of claim 1, wherein the derivation means derives the advice associated with the at least one criterion when the audio and/or the video meets at least one criterion associated with the expected response. computer system.
(Item 3)
3. The computer system of item 2, further comprising modifying means for modifying said at least one criterion based on a result of said communication.
(Item 4)
The changing means is
(i) altering the at least one criterion to relax the at least one criterion when no advice is provided during the communication;
(ii) altering the at least one criterion to reinforce the at least one criterion when advice is provided during the communication and the communication is rated poorly;
A computer system according to item 3.
(Item 5)
The second receiving means receives at least the audio,
The derivation means extracts voice features of the first person and/or the second person from the voice,
deriving the advice based at least on the input and the audio features;
A computer system according to any one of items 1-4.
(Item 6)
The characteristics of the voice of the first person include the pitch of voice when the first person utters a predetermined phrase, the speed of voice when the first person utters a predetermined phrase, the 6. The computer system according to item 5, including at least one of the volume of sound when a first person utters a predetermined phrase, and the number of times the first person utters a predetermined phrase.
(Item 7)
7. The computer system of item 5 or item 6, wherein the voice features of the second person include phrases contained in the second person's utterances.
(Item 8)
8. Any one of items 5 to 7, wherein the voice features of the first person and the second person include a ratio of the speech time of the first person and the speech time of the second person. The described computer system.
(Item 9)
The second receiving means receives at least the video,
The derivation means is
extracting motion features of the first person and/or the second person from the video;
Computer system according to any one of the preceding items, wherein the computer system derives the advice based at least on the input and on the movement characteristics.
(Item 10)
10. The computer system of item 9, wherein the movements include at least one of facial movements, arm movements, and head movements.
(Item 11)
10. The computer system of item 9, wherein the motion characteristics include at least one of nodding magnitude, nodding speed, nodding frequency, glabellar wrinkles, and facial expressions.
(Item 12)
12. The computer according to any one of items 1 to 11, wherein said providing means displays said advice on a video displayed to said first person and/or said second person during said communication. system.
(Item 13)
the advice is provided to the second person;
13. The computer system according to any one of items 1 to 12, wherein said advice includes what said second person should heed in said communication.
(Item 14)
14. The computer system according to any one of items 1-13, wherein the first person is a member in an organization and the second person is a supervisor of the member.
(Item 15)
A method for supporting communication between two parties, comprising:
receiving input indicating a response that a first of said two parties expects from a second of said parties in said communication;
receiving audio and/or video during said communication;
deriving advice in said communication based on said input and said audio and/or said video;
providing said derived advice during said communication.
(Item 15A)
16. The method of item 15, having the features of one or more of the above items.
(Item 16)
A program for supporting communication between two parties, said program being executed in a computer system comprising a processor unit, said program comprising:
receiving input indicating a response that a first of said two parties expects from a second of said parties in said communication;
receiving audio and/or video during said communication;
deriving advice in said communication based on said input and said audio and/or said video;
and providing the derived advice during the communication.
(Item 16A)
17. Program according to item 16, having the features described in one or more of the above items.
(Item 17)
A computer-readable storage medium storing a program for supporting communication between two parties, said program being executed in a computer system comprising a processor unit, said program comprising:
receiving input indicating a response that a first of said two parties expects from a second of said two parties in said communication;
receiving audio and/or video during said communication;
deriving advice in the communication based on the input and the audio and/or video;
and providing the derived advice during the communication.
(Item 17A)
18. The computer-readable storage medium of item 17, having the features of one or more of the previous items.

According to the present invention, it is possible to provide a computer system, method, and program for supporting communication between two parties, thereby improving the quality of communication between the two parties.

A diagram showing an example of a screen displayed on the display unit of an information terminal used by a subordinate A diagram showing an example of a screen displayed on the display unit of an information terminal used by a subordinate A diagram showing an example of a screen displayed on the display unit of an information terminal used by a subordinate The figure which shows an example of the screen displayed on the display part of the information terminal which a boss uses. A diagram showing an example of a configuration of a computer system 100 for supporting communication between two parties. A diagram showing an example of a data structure of advice that can be provided to support communication between two parties, stored in the database unit 200. A diagram showing another example of a data structure of advice that can be provided to support communication between two parties, stored in the database unit 200. A diagram showing an example of the configuration of the terminal device 300 FIG. 4 is a diagram showing an example of the configuration of the processor unit 120; FIG. 12 is a diagram showing an example of a configuration of a processor section 120', which is an alternative embodiment of the processor section 120; 7 is a flowchart illustrating an example process 700 by computer system 100 to facilitate communication between two parties. Flowchart showing an example of processing by derivation means 123 in step S703 8 is a flowchart illustrating an example of another process 800 by computer system 100 to facilitate communication between two parties. Flowchart showing an example of processing by the changing unit 126 in step S802

1. Definitions As used herein, the term "two persons" refers to any two persons. In one embodiment, "two persons" refers to two persons having some hierarchical relationship. Here, some hierarchical relationship may be, for example, a positional hierarchical relationship (eg, boss-subordinate relationship, teacher-student relationship, etc.), or an age-based hierarchical relationship (eg, senior-year lower-level relationship, etc.), or a hierarchical relationship based on experience or knowledge (for example, an expert-amateur relationship, etc.). Note that the hierarchical relationship based on experience or knowledge can be changed according to experience or knowledge that serves as a reference. For example, two people, A and B, can be ranked higher than B if A is more knowledgeable than B in some field, and B is more knowledgeable than A in another field. In some cases, B may be higher than A.

As used herein, "the first of two persons" refers to one of any two persons. In one embodiment, the "first of two persons" refers to the lower person of two persons having some hierarchical relationship. For example, the ``first of the two'' is a subordinate in a boss-subordinate relationship, a student in a teacher-student relationship, a younger person in an older-younger relationship, and an expert-amateur relationship. Including amateurs.

As used herein, the term "second of two persons" refers to the other of any two persons. In one embodiment, the "second person of two" refers to the higher person of two persons having some hierarchical relationship. For example, the ``second person of the two'' is the boss in the boss-subordinate relationship, the teacher in the teacher-student relationship, the older person in the senior-junior relationship, and the expert-amateur relationship. Including geniuses.

As used herein, "communication between two parties" means communication between two parties on a one-to-one basis. "Communication between two parties" includes communication in any field, such as communication in work, communication in school life, communication in home life, communication in hobby activities, and the like.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.

2. Application for supporting communication between two parties The inventor of the present invention developed an application for supporting communication between two parties. Applications for assisting communication between two parties can improve the engagement of individuals in communication between the two parties. Each of the two communicating parties can improve the quality of their communication by using an application for supporting communication between the two parties.

In the following, the two persons are the boss and the subordinate, the first person is the subordinate (referred to as “AAAAAA” in FIGS. 1 and 2), a subordinate and a superior hold a "1on1 meeting", and an application for supporting communication between the two supports the 1on1 meeting. will be described as an example.

Subordinates and superiors can use information terminals (for example, smart phones, tablets, personal computers, etc.) to hold 1on1 meetings and use applications for supporting 1on1 meetings. The application for supporting the 1on1 meeting may be a web application that can be used via a web browser, or may be an application installed on a terminal device. A 1on1 meeting can be conducted via a video call using an application for supporting 1on1 meetings.

An application for supporting 1on1 meetings has a function of managing 1on1 meetings between superiors and subordinates. For example, by entering a schedule for a 1-on-1 meeting, supervisors and subordinates can receive 1-on-1 meeting reminders prior to the schedule. For example, by entering the results of the 1on1 meeting after the 1on1 meeting, the results of the 1on1 meeting are accumulated, so that the boss and subordinates can refer to the past results.

1A to 1C show examples of screens displayed on the display unit of information terminals used by subordinates.

FIG. 1A shows an example of a screen 10 on which an agenda setting request for a 1on1 meeting is notified.

On the screen 10 , an agenda setting request display section 12 for the 1on1 meeting is displayed above the 1on1 meeting details display section 11 .

The details of the next 1on1 meeting are displayed in the 1on1 meeting details display section 11 . For example, in the 1on1 meeting details display section 11, the date and time of the next 1on1 meeting, homework from the previous 1on1 meeting, topics to be discussed at the next 1on1 meeting and expected responses, memos for the next 1on1 meeting, etc. are displayed. A subordinate can look at the 1on1 meeting details display section 11 and prepare for the next 1on1 meeting.

The agenda setting request display unit 12 is notified of an agenda setting request prompting the setting of an agenda for the next and subsequent one-on-one meetings for which the agenda has not yet been set. The agenda setting request is notified, for example, an arbitrary time before the schedule of the next 1on1 meeting. For example, the agenda setting request may be notified 1 hour, 3 hours, 6 hours, 12 hours, 1 day, 3 days, or 1 week before the next 1-on-1 meeting. The agenda setting request display unit 12 may display all 1on1 meetings whose agendas have not yet been set, or some of the 1on1 meetings whose agendas have not yet been set. One, two, three, etc. may be displayed. In the example shown in FIG. 1A, the user is notified to set an agenda for one 1on1 meeting (1on1 meeting with Mr. Taro, the manager) from the next time onward.

The agenda setting request display unit 12 includes a “Set” button 13 . Selecting the "Set" button transitions to the screen 20 shown in FIG. 1B.

FIG. 1B shows an example of a screen 20 for setting an agenda for a 1on1 meeting. In the example shown in FIG. 1B, the screen 20 is a screen for setting an agenda for a one-on-one meeting with Mr. Kacho Taro, his boss.

The screen 20 includes a homework display portion 21 , a topic input portion 22 and a memo column 25 .

The homework display section 21 displays homework assigned to subordinates from the previous time. For example, homework assignments such as "Think of an improvement plan for the business process" and "Request an interview with relevant departments" are displayed.

In the topic input section 22, it is possible to input a topic (or theme) to be discussed in a one-on-one meeting with Mr. Taro Kachou, the boss. Note that the terms "topic" and "theme" are used synonymously herein. Topics may be entered in a multiple-choice format as shown in FIG. 1B, for example, or may be entered in a free-form format. When a topic is input, the corresponding input section 23 for the story on that topic is displayed. The topic may be, for example, business-related, human relations, or subordinates themselves. A topic can be anything that can be discussed in a communication between two parties. For example, in a 1-on-1 meeting in school life, the topics may be academic, career, friends, family, and the like.

In the response input section 23, it is possible to input the response (or actions and/or attitudes) expected from Mr. Taro, the boss, regarding the talk on each topic. The expected response may be entered, for example, in a multiple-choice format as shown in FIG. 1B, or may be entered in a free-form format. The expected response may be, for example, "I want specific advice", "I want you to think with me", "I want you to listen to me", "I want to hear your opinion", "I want to report", and the like. The expected response may be, for example, a specific action or a specific attitude.

A memo can be freely entered in the memo field 25 . For example, you can enter your own competition for a 1on1 meeting. The entered memo can be displayed during the 1on1 meeting, so you can work on the 1on1 meeting while looking at the competition.

Entering the topic and expected response completes the agenda setting.

FIG. 1C shows an example of a screen 30 displayed during a one-on-one meeting between a superior and a subordinate. One-on-one meetings between superiors and subordinates are conducted via video calls.

Screen 30 includes an image display portion 31 and an advice display portion 32 .

The video display unit 31 displays the video of the other party during the 1on1 meeting. Additionally, a video of yourself during the 1on1 meeting may also be displayed. In this example, the image of Mr. Taro Kachou, who is the manager of the one-on-one meeting, is displayed, and the image of himself is also displayed in a small size.

The advice display section 32 displays advice for subordinates in the one-on-one meeting. The advice displayed on the advice display section 32 includes an advice message indicating what to pay attention to in the one-on-one meeting currently in progress. For example, based on the information presented, subordinates can notice what they should pay attention to in the ongoing 1on1 meeting, and strive to improve the quality of the ongoing 1on1 meeting. can.

The advice display section 32 is not displayed when advice to subordinates is not required, and can be displayed when it is determined that advice to subordinates is required. The advice display section 32 may be displayed by being overlaid on the image display section 31 , that is, displayed on the image displayed on the image display section 31 , or may be displayed outside the image display section 31 . You may do so.

In this example, the advice message "Let's ask a more in-depth question." is displayed in the one-on-one meeting currently in progress. The advice message may be uniform advice for individual subordinates, or individual advice determined based on the characteristics of each individual subordinate. It should be noted that advice is not limited to advice messages or sentences, but may be words, phrases, clauses, figures, for example. For example, diagrams may be utilized to advise actions.

FIG. 2 shows an example of a screen displayed on the display unit of the information terminal used by the boss.

FIG. 2 shows an example of a screen 40 displayed during a one-on-one meeting between a superior and a subordinate. As described above, a one-on-one meeting between a superior and a subordinate is conducted via video call.

The screen 40 includes an image display portion 41 , an expected correspondence display portion 42 and an advice display portion 43 .

The video display unit 41 displays the video of the other party during the 1on1 meeting. Additionally, a video of yourself during the 1on1 meeting may also be displayed. In this example, the image of Mr. AAAAA, a subordinate, who is in a one-on-one meeting is displayed, and the image of himself is also displayed in a small size.

The expected response display section 42 displays the response that the subordinate expects in the one-on-one meeting with his boss, Section Manager Taro. The expected correspondence may be the one entered in the correspondence input portion 23 of the screen 20 of FIG. 1B. For example, the boss can grasp what his subordinates want from the information displayed in the expected response display section 42 . This allows superiors to communicate with subordinates in 1-on-1 meetings in line with subordinates' wishes and expectations, which in turn leads to high quality two-way communication.

The expected correspondence display part 42 may be always displayed during the 1on1 meeting, or temporarily during the 1on1 meeting (for example, during a predetermined period at the beginning of the 1on1 meeting, when the boss operates, or It may be displayed when an advice display section 43, which will be described later, is displayed. The expected correspondence display section 42 may be displayed by overlaying it on the image display section 41 , that is, displayed on the image displayed on the image display section 41 , or may be displayed outside the image display section 41 . may be displayed.

The advice display section 43 displays advice to the boss in the one-on-one meeting. The advice displayed on the advice display section 43 includes an advice message indicating what to pay attention to in the one-on-one meeting currently in progress. For example, based on the information presented, the boss can notice what to pay attention to in the ongoing 1on1 meeting, and strive to improve the quality of the ongoing 1on1 meeting. can.

The advice display part 43 is not displayed when the advice to the boss is unnecessary, and can be displayed when it is determined that the advice to the boss is necessary. The advice display section 43 may be displayed by being overlaid on the image display section 41 , that is, displayed on the image displayed on the image display section 41 , or may be displayed outside the image display section 41 . You may do so.

In this example, the advice message "Let's be a little more specific" is displayed in the one-on-one meeting currently in progress. The advice message may be uniform advice for each superior, or individual advice determined based on the characteristics of each individual superior. It should be noted that advice is not limited to advice messages or sentences, but may be words, phrases, clauses, figures, for example. For example, diagrams may be utilized to advise actions.

After the 1on1 meeting is over, the subordinate can evaluate the 1on1 meeting with his boss Kacho Taro. Evaluation can be performed, for example, after screen 30 transitions to an evaluation screen (not shown).

A subordinate can, for example, input an overall evaluation, which is an overall evaluation of the one-on-one meeting with his superior Kacho Taro. The overall evaluation may be, for example, a two-level evaluation indicating whether or not the 1on1 meeting was satisfactory, or a multi-level evaluation indicating the degree of satisfaction with the 1on1 meeting.

Subordinates can, for example, input an individual evaluation of their boss's response in the one-on-one meeting in addition to or instead of the overall evaluation. The individual evaluation may be, for example, a two-level evaluation, such as whether or not the expected response was made for each topic, or whether the response was satisfied or not. It may be a multi-level evaluation that indicates the degree of

After the 1on1 meeting ends, the boss can evaluate the 1on1 meeting with subordinate AAAAA. Evaluation can be performed, for example, after screen 40 transitions to an evaluation screen (not shown).

The boss can, for example, input an overall evaluation that is an evaluation of the entire 1on1 meeting with subordinate AAAAA. The overall evaluation may be, for example, a two-level evaluation, such as whether or not the 1on1 meeting was satisfactory for the subordinate, or a multi-level evaluation indicating the degree to which the 1on1 meeting was satisfactory for the subordinate. may be

For example, in addition to the overall evaluation, or instead of the overall evaluation, the superior can input an individual evaluation as to whether the expectations of subordinates in the one-on-one meeting have been met. The individual evaluation may be, for example, a two-level evaluation, such as whether or not the expected response could be performed for each topic, or a multi-level evaluation indicating the degree to which the expected response was performed. Graded evaluation may be used.

Information input to the screens 20, 30, and 40 is accumulated in the application for supporting the 1on1 meeting and used for the 1on1 meeting later. The later 1on1 meeting may be a 1on1 meeting between the subordinate "AAAAA" and the boss "Kancho Taro", or a 1on1 meeting between the subordinate "AAAAA" and another boss, or with another subordinate. It may be a one-on-one meeting with the boss "Kancho Taro" or a one-on-one meeting with another subordinate and another boss.

In the above example, the 1on1 meeting is performed via video call, but the communication supported by the present invention is not limited to 1on1 meeting via video call. The present invention can support any form of communication between two parties as long as it is possible to obtain audio and/or video of at least one of the two parties. Such communication may include, for example,

In the above example, the example of supporting a 1on1 meeting between a boss and a subordinate has been explained, but the target communication supported by the present invention is not limited to a 1on1 meeting between the boss and a subordinate. The present invention supports arbitrary one-to-one communication by any two parties, as long as the correspondence between the two parties is preset by one of the two parties to expect from the other of the two parties. be able to. This kind of communication is, for example, between Mr. A (boss) in a certain company and Mr. A (boss) in order to learn from Mr. B (boss) in another company how to effectively draw out power from his subordinates. Communication between Mr. C (subordinate), a member of a certain company, and Mr. C (subordinate) to learn how to smoothly proceed with meetings and how to develop his future career from Mr. D (boss), the manager of another company. communication between high school student E to learn effective ways to study for entrance exams from college student F This includes, but is not limited to, communication between Mr. G and Mr. H or Mr. I to learn from Mr. H, an expert, or Mr. I who has knowledge and experience.

The application for supporting communication between two parties described above may be implemented, for example, by a computer system for supporting communication between two parties described below. Computer systems for supporting communication between two parties can improve the engagement of individuals in communication between two parties.

3. Configuration of Computer System for Supporting Communication between Two Parties FIG. 3 shows an example of the configuration of a computer system 100 for supporting communication between two parties.

Computer system 100 is connected to database unit 200 . Computer system 100 is also connected to at least one terminal device 300 via network 400 .

Although three terminal devices 300 are shown in FIG. 3, the number of terminal devices 300 is not limited to this. Any number of terminals 300 may be connected to system 100 via network 400 .

Network 400 may be any type of network. Network 400 may be, for example, the Internet or a LAN. Network 400 may be a wired network or a wireless network.

Computer system 100 may be a computer (eg, a server device) installed at a service provider that provides applications to facilitate communication between two parties. The terminal device 300 may be a computer (eg, terminal device) used by two parties in communication between them. For example, in the example described above, one of the terminal devices 300 may be a computer used by a superior, and another one of the terminal devices 300 may be a computer used by a subordinate. Here, the computer (server device or terminal device) can be any type of computer. For example, the terminal device can be any type of terminal device, such as smart phones, tablets, personal computers, smart glasses, smart watches, and the like.

The computer system 100 comprises an interface section 110, a processor section 120, and a memory 130 section. Computer system 100 is connected to database unit 200 .

The interface unit 110 exchanges information with the outside of the computer system 100 . The processor unit 120 of the computer system 100 can receive information from the outside of the computer system 100 via the interface unit 110 and can send information to the outside of the computer system 100 . The interface unit 110 can exchange information in any format. The information terminal used by the first person and the information terminal used by the second person can communicate with the computer system 100 via the interface section 110 .

Interface unit 110 includes, for example, an input unit that allows information to be entered into computer system 100 . It does not matter in what manner the input unit allows information to be entered into the computer system 100 . For example, if the input unit is a receiver, the receiver may receive information from outside the computer system 100 via a network for input. In this case, the type of network does not matter. For example, the receiver may receive information via the Internet or may receive information via a LAN.

Interface unit 110 includes, for example, an output unit that allows information to be output from computer system 100 . It does not matter in what manner the output unit enables information to be output from the computer system 100 . For example, if the output unit is a transmitter, the transmitter may output information by transmitting it to the outside of computer system 100 via a network. In this case, the type of network does not matter. For example, a transmitter may transmit information over the Internet or transmit information over a LAN.

The processor unit 120 executes processing of the computer system 100 and controls the operation of the computer system 100 as a whole. The processor unit 120 reads a program stored in the memory unit 130 and executes the program. This allows the computer system 100 to function as a system that executes desired steps. The processor unit 120 may be implemented by a single processor or multiple processors.

The memory unit 130 stores programs required for executing processes of the computer system 100, data required for executing the programs, and the like. The memory unit 130 may store a program for causing the processor unit 120 to perform processing for supporting communication between two parties (for example, a program for realizing processing shown in FIGS. 7 and 8, which will be described later). good. Here, it does not matter how the program is stored in the memory unit 130 . For example, the program may be pre-installed in memory unit 130 . Alternatively, the program may be installed in memory unit 130 by being downloaded via a network. In this case, the type of network does not matter. Memory unit 130 may be implemented by any storage means. Alternatively, the program may be stored in a computer-readable storage medium and installed in the memory unit 130 by reading the computer-readable storage medium.

The database unit 200 stores various information that can be used to support communication between two parties.

The database portion 200 stores advice that can be provided to assist communication between two parties. Advice that can be provided is stored in association with, for example, an expected response in communication between two parties and features extracted from audio and/or features extracted from video that may be obtained in communication between two parties. obtain. In one embodiment, the advice that may be provided may be stored in association with criteria depending on the correspondence expected in communication between the two parties. The criteria may be criteria for features extracted from audio and/or criteria for features extracted from video.

FIG. 4A shows an example of a data structure of advice stored in the database unit 200 that can be provided to support communication between two parties. In the example shown in FIG. 4A, advice for a second person is shown.

Advice that can be provided to assist communication between two parties is stored in association with criteria and the response that the second person expects from the first person in the communication between the two. The criteria are criteria for features extracted from audio and/or features extracted from video captured in communication between two parties. Each criterion is associated with a respective correspondence, and the criterion is different for each correspondence.

The criteria for the features extracted from the speech are the criteria for the features extracted from the first person's speech, the criteria for the features extracted from the second person's speech, the first person's utterance time, and the second person's speech time. Includes criteria for speech duration of

The criteria for features extracted from the first person's speech relate, for example, to the excess or deficiency of specific phrases in the first person's speech. For example, this criterion may be satisfied if the first person's speech does not contain a particular phrase for a predetermined period of time during the communication between the two parties. This criterion may be met if the first person's speech contains a particular phrase less than a predetermined number of times during a predetermined period of time during the communication, e.g. This criterion may be met if the particular phrase appears more than a predetermined number of times in the first person's speech during a predetermined period of time. The predetermined period may be any period such as, for example, 5 minutes from the start of communication between two parties, 10 minutes from the start, 30 minutes from the start, or 10 minutes after 5 minutes from the start. The predetermined number of times may be any number of times, such as 2 times, 3 times, 5 times, or 10 times.

Specific phrases are, for example, "Specifically", "First", "Next", "Okay", "That's a lot", "What should I do", "Do you have any ideas?", etc. including, but not limited to, the language of Any wording can be set as a specific wording.

Criteria relating to features extracted from the second person's speech relate, for example, to the pitch, speed, loudness and/or number of times a particular phrase is spoken in the second person's speech. For example, when the pitch of the voice of a specific phrase included in the voice of the second person during a predetermined period of time during communication between the two parties is above or below a predetermined height (Hz) For example, the speed of speech of a specific phrase included in the second person's speech during a predetermined period of time during the communication between the two parties may meet the predetermined speed (predetermined This criterion may be met if the number of seconds to speak the entire sentence of the phrase is greater or less than or equal to This criterion may be met if the loudness of the particular phrase contained is above or below a given loudness (dB), e.g. This criterion may be met if the specific phrase was included in the second person's speech during the interval less than or equal to a predetermined number of times. The predetermined period may be any period such as, for example, 5 minutes from the start of communication between two parties, 10 minutes from the start, 30 minutes from the start, or 10 minutes after 5 minutes from the start.

Preferably, the criteria for the features extracted from the second person's speech relate to the pitch of particular phrases in the second person's speech. The pitch of the voice tends to reveal the feelings of the speaker, and by using the pitch of the voice as a reference, it becomes possible to accurately estimate the second person's feelings, and eventually, to give advice. This is because it leads to an improvement in the accuracy of In another preferred example, the criteria for the features extracted from the second person's speech are, for example, the pitch and speed of a particular phrase in the second person's speech; The criteria for the features extracted from the speech of the two persons relate to the pitch, speed, and loudness of the speech of a particular phrase in the speech of the second person, and in another preferred example, the speech of the second person. Criteria for the features extracted from are related to the pitch, speed, loudness and number of times a particular phrase is spoken in the second person's voice. The speaker's feelings are most likely to be expressed by the pitch of the voice, followed by the speed, loudness, and number of times, in that order. This is because it becomes possible to make a good estimation, which in turn leads to a further improvement in the accuracy of advice.

Specific phrases include, but are not limited to, phrases such as "yes", "understood", "but", "but", "will try", "certainly", and the like. Any wording can be set as a specific wording.

The criteria for the first person's speech time and the second person's speech time relate, for example, to the ratio between the first person's speech time and the second person's speech time. For example, if the ratio of the speech time of the first person to the speech time of the second person during a predetermined period of time during communication between two parties is equal to or greater than or equal to or less than a predetermined ratio, this criterion is met. may be The predetermined period may be any period such as, for example, 5 minutes from the start of communication between two parties, 10 minutes from the start, 30 minutes from the start, or 10 minutes after 5 minutes from the start.

The criteria for features extracted from the video includes criteria for features extracted from the video of the second person. Note that the criteria for features extracted from the video include criteria for features extracted from the video of the first person in addition to or instead of the criteria for features extracted from the video of the second person. You can also try to

The criteria for features extracted from the video of the second person are, for example, the presence/absence, magnitude, speed, and frequency of movement of the second person. For example, if facial movements (e.g., facial expression changes, brow wrinkles) were or were not included in the video of the second person during a predetermined period of time during communication between the two parties, This criterion may also be satisfied, for example, if the magnitude of movement included in the image of the second person during a predetermined period of time during communication between the two is greater than or equal to a predetermined magnitude (movement distance). Alternatively, this criterion may be met if, for example, the speed of motion included in the video of the second person during a predetermined period of time during the communication between the two parties was a predetermined This criterion may be met if the velocity (movement velocity) was above or below, e.g. This criterion may be met if the frequency of movement is above or below a predetermined frequency (number of times per predetermined period). The predetermined period may be any period such as, for example, 5 minutes from the start of communication between two parties, 10 minutes from the start, 30 minutes from the start, or 10 minutes after 5 minutes from the start.

The movements of the second person include, but are not limited to, facial movements, arm movements, and head movements. The movement of the second person can be any movement that can be extracted from the video in the communication between the two. More specifically, the motion of the second person may be a nod.

For example, in the example shown in FIG. 4A, the expected response "I want specific advice" is a reference for features extracted from the first person's voice that "there is no word 'specifically'". , the criteria for the features extracted from the voice of the second person, such as "the speed of the speech of ``yes'' and ``understood'' is below the threshold and the volume is below the threshold"; Speech time: The second person's speech time is greater than 3:7", and the standard for the first person's speech time and the second person's speech time is "Let's be a little more specific!" Stored in association with advice. For example, an expected response of "I want you to think together" is combined with a criterion related to features extracted from the second person's voice that "the loudness of the voice of the second person is below a threshold" and "the first person's Utterance time: the second person's utterance time is greater than 5:5. are stored in association with the criteria for the person's video and the advice "You talk too much".

Thus, the correspondence is based on the criteria for the features extracted from the first person's speech, the criteria for the features extracted from the second person's speech, the first person's speech duration and the second person's speech duration At least one of the criteria regarding the person's image and the criteria regarding the two person images may be stored in association with the advice. This allows, given a response, to derive the associated advice if the criteria associated with that response are met.

FIG. 4B shows another example of a data structure of advice stored in the database unit 200 that can be provided to support communication between two parties. In the example shown in FIG. 4B, advice for the first person is shown.

The advice shown in FIG. 4B is stored in the database section 200 in the same manner as the advice shown in FIG. 4A. The advice for the first person shown in FIG. 4B may be a companion advice to the advice for the second person shown in FIG. 4A.

The advice stored in the database unit 200 may be stored in association with the attribute of the organization, for example. For example, the advice for companies stored in the database unit 200 is divided into advice for organizations in the primary industry, advice for organizations in the secondary industry, and advice for organizations in the tertiary industry. obtain. For example, the advice for students stored in the database unit 200 may be divided into advice for elementary school students, advice for middle school students, advice for high school students, and advice for university students. This makes it possible to provide different advice to organizations with different attributes.

In the present invention, the timing of storing the advice in the database unit 200 does not matter. Advice may be pre-stored in the database unit 200, for example, by a service provider that provides services for facilitating communication between two parties. Alternatively, the advice may be entered and stored by a user corresponding to the second person, for example, while the computer system 100 is used to operate the service.

Also, in accordance with the present invention, the criteria associated with the advice may be fixed or may vary. Preferably, the criteria associated with the advice can be changed based on the evaluation from one or both of the two parties of the communication between the two parties to which the advice was provided. Criteria associated with advice may be changed, for example, by the processes described below with reference to FIGS. 8A and 8B.

In the example shown in FIG. 3, the database section 200 is provided outside the computer system 100, but the present invention is not limited to this. It is also possible to provide the database unit 200 inside the computer system 100 . At this time, the database section 200 may be implemented by the same storage means as the storage means in which the memory section 130 is implemented, or may be implemented by a storage means different from the storage means in which the memory section 130 is implemented. In any event, database unit 200 is configured as a storage unit for computer system 100 . The configuration of database unit 200 is not limited to a specific hardware configuration. For example, the database unit 200 may be configured with a single hardware component, or may be configured with a plurality of hardware components. For example, the database unit 200 may be configured as an external hard disk device of the computer system 100, or configured as a cloud storage connected via a network.

FIG. 5 shows an example of the configuration of the terminal device 300. As shown in FIG.

The terminal device 300 includes an interface section 310 , an input section 320 , a display section 330 , a memory section 340 , a processor section 350 , an imaging section 360 and an audio output section 370 .

Interface unit 310 controls communication via network 400 . The processor unit 350 of the terminal device 300 can receive information from outside the terminal device 300 via the interface unit 310 and can transmit information to the outside of the terminal device 300 . Interface unit 310 may control communications in any manner.

The input unit 320 allows the user to input information to the terminal device 300 . It does not matter in what manner the input unit 320 enables the user to input information to the terminal device 300 . For example, if the input unit 320 is a touch panel, the user may input information by touching the touch panel. Alternatively, if the input unit 320 is a mouse, the user may input information by operating the mouse. Alternatively, if the input unit 320 is a keyboard, the user may input information by pressing keys on the keyboard. Alternatively, if the input unit is a microphone, the user may input information by inputting voice into the microphone. Alternatively, if the input unit is a data reader, information may be input by reading information from a storage medium connected to computer system 100 .

Display 330 may be any display for displaying information. For example, the display unit 330 displays the screens shown in FIGS.

The memory unit 340 stores programs for executing processes in the terminal device 300, data required for executing the programs, and the like. The memory unit 340 stores, for example, an application for supporting communication between two parties. The memory unit 340 may store applications that implement arbitrary functions. Here, it does not matter how the program is stored in the memory unit 340 . For example, the program may be pre-installed in memory unit 340 . Alternatively, the program may be installed in memory unit 340 by being downloaded via network 400 . Memory unit 340 may be implemented by any storage means.

The processor unit 350 controls the operation of the terminal device 300 as a whole. The processor unit 350 reads a program stored in the memory unit 340 and executes the program. This allows the terminal device 300 to function as a device that executes desired steps. The processor unit 350 may be implemented by a single processor or may be implemented by multiple processors.

The imaging unit 360 is any means capable of capturing still images or moving images. Audio input unit 360 is, for example, a camera. The camera may be a built-in camera in terminal device 300 or an external camera attached to terminal device 300 .

Audio output unit 370 is any means for outputting audio. Audio output unit 370 is, for example, a speaker.

Although each component of the terminal device 300 is provided in the terminal device 300 in the example shown in FIG. 4A, the present invention is not limited to this. Any one of the components of terminal device 300 may be provided outside terminal device 300 . For example, when the input unit 320, the display unit 330, the memory unit 340, the processor unit 350, the imaging unit 360, and the audio output unit 370 are configured by separate hardware components, each hardware component can It may be connected via a network. At this time, the type of network does not matter. Each hardware component may be connected via a LAN, wirelessly, or wired, for example. Terminal device 300 is not limited to a specific hardware configuration. For example, it is within the scope of the present invention to configure the processor section 350 with analog circuits instead of digital circuits. The configuration of the terminal device 300 is not limited to those described above as long as the functions can be realized.

FIG. 6A shows an example of the configuration of the processor unit 120. As shown in FIG.

The processor unit 120 includes first receiving means 121 , second receiving means 122 , deriving means 123 and providing means 124 .

The first receiving means 121 is configured to receive information indicating the response that the first person expects from the second person in the communication between the two parties. The first receiving means 121 can receive information from outside the computer system 100 via the interface section 110, for example.

The response that the first person expects of the second person in communication between two parties may be input to the terminal device 300 used by the first person via the input unit 320 (for example, a keyboard or a touch panel). The input information can be input from the terminal device 300 to the computer system 100 via the interface unit 110 . For example, when the first person inputs an expected response to the response input section 23 of the screen 20 shown in FIG. 1B before bilateral communication with the second person, the input expected response is The indicated information is input to the computer system 100 via the interface section 110 , and the first receiving means 121 receives the information from the interface section 110 .

The second receiving means 122 is arranged to receive audio and/or video during communication between two parties. The second receiving means 122 can receive audio and/or video during communication between two parties from outside the computer system 100 via the interface unit 110, for example.

Audio and/or video during communication between two parties may be audio and/or video of a first person, audio and/or video of a second person, or both. may be Preferably, the second receiving means 122 can receive the audio and/or video of the first person and the audio and/or video of the second person.

The voice of the first person can be input to the terminal device 300 used by the first person through the input unit 320 (for example, a microphone), and the input voice is transmitted from the terminal device 300 to the interface unit 110. can be input into computer system 100 via . An image of the first person can be input to the terminal device 300 used by the first person through the imaging unit 360 (e.g., a camera), and the input image is transmitted from the terminal device 300 to the interface unit 110. can be input into computer system 100 via .

The voice of the second person can be input to the terminal device 300 used by the second person through the input unit 320 (for example, a microphone), and the input voice is transmitted from the terminal device 300 to the interface unit 110. can be input into computer system 100 via . The image of the second person can be input to the terminal device 300 used by the second person through the imaging unit 360 (for example, a camera), and the input image is transmitted from the terminal device 300 to the interface unit 110. can be input into computer system 100 via .

The deriving means 123 is configured to derive advice in communication between two parties based on the information received by the first receiving means 121 and the audio and/or video received by the second receiving means 122. It is The deriving means 123 can derive, for example, advice associated with both the expected response indicated by the received information and the audio and/or video from among the advice stored in the database unit 200. More specifically, the deriving means 123 derives advice associated with the correspondence and criteria when the audio and/or video meets the criteria associated with the expected response indicated by the received information. can be done. For example, the derivation means 123 may avoid deriving advice when the audio and/or video do not meet the criteria. Criteria may be stored in database portion 200 in association with responses and advice.

The deriving means 123, for example, extracts the voice features of the first person and/or the voice features of the second person from the voice received by the second receiving means 122, and converts the extracted voice features into Can be compared with standards. The features of the voice of the first person are the pitch of voice when the first person utters a predetermined word, the speed of voice when the first person utters a predetermined word, It includes at least one of the volume of voice when the predetermined phrase is uttered, and the number of times the first person has uttered the predetermined phrase. The derivation means 123 uses means that are known or will become known in the field of speech recognition to identify the speech of the predetermined sentence, and measure the pitch, speed, loudness, and number of times of the identified speech. can be done.

The second person's voice feature may be a predetermined phrase included in the second person's utterance. The derivation means 123 may use means known or to be known in the field of speech recognition to identify the speech of the given phrase and determine whether the given phrase is present, or The number of times a given phrase appears can be measured.

The first person voice feature and the second person voice feature may be a first person speech time and a second person speech time. The derivation means 123 uses means known or to be known in the field of speech recognition to identify the first person's speech and the second person's speech, and It is possible to measure the utterance time of a person.

The deriving means 123, for example, extracts the motion features of the first person and/or the motion features of the second person from the video received by the second receiving means 122, and converts the extracted motion features into Can be compared with standards. The movement includes at least one of facial movement (eg, facial expression change, brow wrinkles), arm movement, head movement (eg, nodding). Motion characteristics include at least one of presence/absence, magnitude, speed, and frequency of these motions. Derivation means 123 may use means known or to be known in the field of motion recognition to identify predetermined motions, determine the presence or absence of identified motions, or determine the magnitude of identified motions. You can measure speed, speed and frequency.

For example, as shown in FIG. 4A, a case will be described where advice is stored in the database unit 200 in association with correspondence and criteria. If the information received by the first receiving means 121 indicates that "I want specific advice" is the expected response, the deriving means 123 receives the first person received by the second receiving means 122. Identify the phrases "specifically", "first", and "next" from the speech, and determine the presence or absence of the phrase "specifically" and the presence of the phrases "first" and "next" . Further, the deriving means 123 measures the pitch of the phrase "Yes" and the pitch and loudness of the phrase "I understand" from the speech received by the second receiving means 122; Whether the voice pitch of the "yes" phrase is below a threshold (e.g., 320 Hz) and whether the voice pitch of the "understood" phrase is below a threshold (e.g., 300 Hz) and loudness is less than or equal to a threshold value (eg, 60 dB). Further, the deriving means 123 measures the speech time of the first person and the speech time of the second person from the voice received by the second receiving means 122, and calculates the speech time of the first person and the speech time of the second person. is greater than 3:7. The deriving means 123 may derive the advice when at least some of the criteria associated with the advice are met, or derive the advice when all of the criteria associated with the advice are met. Advice may be derived.

For example, in the advice "Let's be a little more specific!", (1) there is no word "specifically" and (2) the voice pitch of the word "yes" is below the threshold. and the pitch of the phrase "understood" is below the threshold; (3) the ratio of the first person's speech time to the second person's speech time is greater than 3:7 , is associated with the criteria. For example, when at least one of (1) to (3) is satisfied, the derivation means 123 may derive the advice "Let's be more specific!" When all of 1) to (3) are satisfied, the derivation means 123 may derive an advice "Let's be more specific!".

For example, in the advice "The other party may not be convinced!", (1) the words "first" and "next" do not exist, (3) the ratio of the speech time of the first person to the speech time of the second person is 3; : greater than 7, for example, when at least one of (1) to (3) is satisfied, the derivation means 123 says, "The other party may not be convinced. or, when all of (1) to (3) are satisfied, the deriving means 123 provides the advice "The other party may not be convinced!" may be derived.

When the criteria associated with a plurality of pieces of advice are satisfied, the deriving means 123 may derive all of the plurality of pieces of advice, or derive a part of the plurality of pieces of advice. may

Thus, the derivation means 123 can determine whether the audio and/or video meet the criteria associated with the correspondence, and derive the corresponding advice if the criteria are met.

The providing means 124 are configured to provide the advice derived by the deriving means 123 during communication between the two parties. The providing means 124 may provide advice to the first person, may provide advice to the second person, or may provide advice to the first person and the second person. and both. For example, if the advice derived by the deriving means 123 is directed to the first person, the providing means 124 can provide the advice to the first person. For example, if the advice derived by the deriving means 123 is directed to the second person, the providing means 124 can provide the advice to the second person. The providing means 124 can provide advice via the interface section 110 . It doesn't matter how the advice is given here. For example, the advice can be transmitted to the terminal device 300 used by the first person via the interface unit 110 and displayed on the display unit 330 of the terminal device 300 . The advice is displayed on the image of screen 30, for example, as shown in FIG. 1C. For example, the advice can be transmitted to the terminal device 300 used by the second person via the interface unit 110 and displayed on the display unit 330 of the terminal device 300 . The advice is displayed on the image on the screen 40 as shown in FIG. 2, for example.

If the providing means 124 provides advice to both the first person and the second person, preferably the advice provided is paired advice.

For example, based on the provided advice, the first person asks how to get the second person to behave as expected, and how to get the second person to behave as expected. The second person can know what to pay attention to, etc., and can take actions or attitudes to get the second person to respond as expected. This may improve the quality of communication between the two parties.

For example, a second person may, from the advice provided, know what the first person expects to do in order to meet or satisfy the first person's expected response. In order to respond to the response or satisfy the first person, it is possible to know what to pay attention to, etc., and to respond to the response expected by the first person or to be satisfied with the first person. Able to take actions or attitudes to get. This may improve the quality of communication between the two parties.

In addition, since these advices are based on voice and/or video obtained during communication, that is, the most recent voice and/or video, they are highly accurate as advice and effectively improve the quality of communication between two parties. can be significantly increased.

FIG. 6B shows an example of the configuration of processor section 120 ′, which is an alternative embodiment of processor section 120 . The processor section 120 ′ has the same configuration as the processor section 120 except that it includes an acquisition means 125 and a change means 126 . The processor portion 120' can change the criteria for deriving the advice, and the processor portion 120' can derive the advice based on the changed criteria. In FIG. 6B, the same reference numerals are given to the same components as those shown in FIG. 6A, and detailed descriptions thereof are omitted here.

The processor unit 120 ′ includes first receiving means 121 , second receiving means 122 , deriving means 123 , providing means 124 , obtaining means 125 and changing means 126 .

The acquisition means 125 are arranged to acquire the result of communication between two parties that has already taken place. Acquisition means 125 may, for example, acquire results from outside the computer system 100 via the interface unit 110, or may acquire results from inside the computer system 100 (for example, storage means). good too. A result of a communication between two parties that has already taken place may be, for example, whether computer system 100 provided advice in that communication and/or an assessment of that communication.

An evaluation is an evaluation by a first person. The evaluation can be input to the terminal device 300 used by the first person via the input unit 320 (e.g., keyboard or touch panel), and the input information is sent from the terminal device 300 via the interface unit 110. can be input into computer system 100 . For example, when the first person inputs an evaluation of the communication on an evaluation input screen (not shown) after bilateral communication with the second person, the input evaluation is sent via the interface unit 110. is input to the computer system 100, and the acquisition means 125 receives the evaluation from the interface unit 110. FIG.

The evaluation may be, for example, a two-level evaluation indicating whether or not the communication with the second person was satisfied, or a multi-level evaluation indicating the degree of satisfaction with the communication with the second person. .

The evaluation may be an evaluation by a second person. The evaluation can be input to the terminal device 300 used by the second person via the input unit 320 (e.g., keyboard or touch panel), and the input information is sent from the terminal device 300 via the interface unit 110. can be input into computer system 100 . For example, when the second person inputs an evaluation of the communication on an evaluation input screen (not shown) after bilateral communication with the first person, the input evaluation is sent via the interface unit 110. is input to the computer system 100, and the acquisition means 125 receives the evaluation from the interface unit 110. FIG.

The changing means 126 is configured to change the criteria upon which the deriving means 123 derives advice, based on the past communication results between the two parties acquired by the acquiring means 125 . A result of a past two-party communication may be, for example, whether computer system 100 provided advice in that communication and/or an evaluation of that communication. For example, if computer system 100 provided advice in the past communication between two parties, but the evaluation for that communication was worse than the average, that is, if it can be said that the advice given by computer system 100 was not effective, the advice In order to change the derivation logic of , the modifying means 126 can modify the criteria. Alternatively, for example, if the computer system 100 did not provide advice in past bilateral communications, the modifying means 126 can modify the criteria to modify the advice derivation logic. This is based on the idea that the quality of communication is improved when computer system 100 provides advice rather than not providing advice.

Modifying means 126 may modify the criteria to tighten, for example, if computer system 100 provided advice but rated the communication worse than average. This makes it difficult for computer system 100 to provide advice. For example, modifier 126 may strengthen the criteria by raising at least one criterion and/or adding new criteria. The new criteria may be, for example, criteria associated with another correspondence, or preferably criteria used in another communication that was highly rated.

Modifying means 126 may modify, for example, to relax the criteria if computer system 100 did not provide advice. This facilitates the provision of advice by computer system 100 . For example, modifier 126 may relax the criteria by reducing at least one criterion and/or removing at least one criterion. For example, the modifying means 126 compares the reference used in the past communication between the two parties and the audio and/or video acquired in the communication and identifies the different parts. Then, the changing means 126 can change the reference for the different parts. For example, the modifying means 126 can remove the different parts from the reference. Alternatively, for example, the modifying means 126 can relax the different parts.

This allows the derivation means 123 to derive advice using the changed criteria.

By changing the criteria based on the evaluation of past communication in this way, the logic for deriving advice can be refined, and the quality of the advice derived by the deriving means 123 can be improved. This can improve the quality of communication between the two parties.

Each component of the computer system 100 described above may be configured with a single hardware component, or may be configured with a plurality of hardware components. When configured with a plurality of hardware components, it does not matter how the hardware components are connected. Each hardware component may be connected wirelessly or by wire. Computer system 100 of the present invention is not limited to any particular hardware configuration. It is within the scope of the present invention for the processor portion 120 to be configured with analog circuitry rather than digital circuitry. The configuration of the computer system 100 of the present invention is not limited to the above as long as the functions can be realized.

4. Process by Computer System for Supporting Communication between Two Parties FIG. 7A is a flow chart illustrating an example process 700 by computer system 100 for supporting communication between two parties. Process 700 is executed in processor portion 120 of computer system 100 .

In step S701, the first receiving means 121 of the processor unit 120 receives information indicating the response that the first person expects of the second person in communication between two parties. The first receiving means 121 can receive information from outside the computer system 100 via the interface section 110, for example.

In step S702, the second receiving means 122 of the processor unit 120 receives audio and/or video during communication between two parties. The second receiving means 122 can receive audio and/or video during communication between two parties from outside the computer system 100 via the interface unit 110, for example.

At step S703, the derivation means 123 of the processor unit 120 derives advice for communication between two parties based on the information received at step S701 and the audio and/or video received at step S702. The deriving means 123 can derive, for example, advice associated with both the expected response indicated by the received information and the audio and/or video from among the advice stored in the database unit 200. More specifically, the deriving means 123 derives advice associated with the correspondence and criteria when the audio and/or video meets the criteria associated with the expected response indicated by the received information. can be done. For example, the derivation means 123 may avoid deriving advice when the audio and/or video do not meet the criteria.

The derivation unit 123 can derive advice by, for example, processing described later with reference to FIG. 7B.

At step S704, the providing means 124 of the processor unit 120 provides the advice derived at step S703 during communication between the two parties. The providing means 124 may provide advice to the first person, may provide advice to the second person, or may provide advice to the first person and the second person. and both. For example, if the advice derived by the deriving means 123 is directed to the first person, the providing means 124 can provide the advice to the first person. For example, if the advice derived by the deriving means 123 is directed to the second person, the providing means 124 can provide the advice to the second person.

The providing means 124 can provide advice via the interface section 110 . For example, the advice can be transmitted to the terminal device 300 used by the first person via the interface unit 110 and displayed on the display unit 330 of the terminal device 300 . The advice is displayed on the image of screen 30, for example, as shown in FIG. 1C. For example, the advice can be transmitted to the terminal device 300 used by the second person via the interface unit 110 and displayed on the display unit 330 of the terminal device 300 . The advice is displayed on the image on the screen 40 as shown in FIG. 2, for example.

The process 700 allows the first person and/or the second person to receive advice during the communication between the two parties and seeks to best engage the other party in the communication between the two parties. be able to. This can improve the quality of communication.

FIG. 7B is a flowchart showing an example of processing by the derivation unit 123 in step S703.

At step S7031, the deriving means 123 refers to the criteria associated with the response that the first person expects from the second person indicated by the information received at step S701. The criteria are stored in the database unit 200 in association with, for example, correspondence and advice. For example, if the information received at step S701 indicates multiple correspondences, the derivation means 123 may refer to criteria associated with each of the multiple correspondences.

At step S7032, derivation means 123 determines whether the audio and/or video received at step S702 meets the criteria referred to at step S7031. If it is determined that the criteria are met, the process proceeds to step S7033, and if it is determined that the criteria are not met, the process proceeds to step S7034.

The deriving means 123, for example, extracts the voice features of the first person and/or the voice features of the second person from the voice received by the second receiving means 122, and converts the extracted voice features into Can be compared with standards. Additionally or alternatively, the deriving means 123 extracts the motion features of the first person and/or the motion features of the second person, for example from the video received by the second receiving means 122. , the extracted motion features can be compared with the reference.

For example, as shown in FIG. 4A, when multiple criteria are associated with one correspondence, derivation means 123 determines, for each of the multiple criteria, that the audio and/or video received in step S702 is Determine whether or not the criteria are met. For example, regarding the response "I want specific advice", (1) the word "specifically" does not exist, (2) the voice pitch of the word "yes" is below the threshold, and " (3) the ratio of the speech time of the first person to the speech time of the second person is greater than 3:7; Criteria and (1) the phrase "first" and "next" do not exist, (2) the pitch of the phrase "understood" is below the threshold, and the phrase "understood" Associated with a second criterion is that the loudness of the voice is less than or equal to a threshold, and (3) the ratio of the first person's speech time to the second person's speech time is greater than 3:7. In this case, it is determined whether each of the first criterion and the second criterion is satisfied.

For example, when the criteria are subdivided into a plurality of criteria, such as the above-described first criteria and second criteria, if at least a part of the subdivided parts is satisfied, the criterion is satisfied, and the process proceeds to step S7033. Alternatively, if all of the subdivided parts are satisfied, the criterion may be satisfied and the process may proceed to step S7033. That is, taking the above first criterion as an example, if at least some of (1) to (3) are satisfied, the criterion may be satisfied and the process may proceed to step S7033, or 1) to (3). is satisfied, the process may proceed to step S7033 assuming that the criterion is satisfied.

At step S7033, the deriving means 123 derives advice associated with the criterion determined to be satisfied. Specifically, the deriving means 123 derives advice stored in the database section 200 in association with the criteria. If it is determined that multiple criteria are met, the associated advice will be derived for each of the multiple criteria.

In step S7034, derivation means 123 determines not to derive advice. Processing may end here without proceeding to step S704.

Advice derived in step S7033 will be provided in step S704.

FIG. 8A is a flowchart illustrating an example of another process 800 by computer system 100 to facilitate communication between two parties. Process 800 is executed in processor portion 120 ′ of computer system 100 . Process 800 is a process for changing the criteria for deriving advice by deriving means 123 .

In step S801, the acquisition means 125 of the processor unit 120' acquires the result of communication between the two parties that has already taken place. Acquisition means 125 may, for example, receive results from outside the computer system 100 via the interface unit 110, or may acquire results from inside the computer system 100 (for example, storage means). good too. A result of a communication between two parties that has already taken place may be, for example, whether computer system 100 provided advice in that communication and/or an assessment of that communication. The evaluation may be an evaluation by a first person or may be an evaluation by a second person. Preferably, the evaluation may be an evaluation by a first person.

At step S802, the modifying means 126 of the processor unit 120' modifies at least one criterion for deriving advice based on the results obtained at step S801.

The changing means 126 can change the criteria for deriving advice, for example, by the process described later with reference to FIG. 8B.

Process 800 enables derivation means 123 to derive advice using the modified criteria. By changing the criteria based on the evaluation of past communication in this way, the logic for deriving advice can be refined and the quality of the advice derived by the deriving means 12 can be improved. This can improve the quality of communication between the two parties.

FIG. 8B is a flowchart showing an example of processing by the changing unit 126 in step S802.

In step S8021, the changing unit 126 determines whether or not the computer system 100 has provided advice in past communication between the two parties based on the result obtained in step S801. If it is determined that the computer system 100 provided advice in the past communication between the two parties, the process proceeds to step S8022; if it is determined that the computer system 100 did not provide the advice, the process proceeds to step S8023.

In step S8022, based on the result obtained in step S801, the changing means 126 determines whether the evaluation of the past two-party communication is high (for example, whether the evaluation is high among binary evaluations of high and low, or whether the average value, etc.). If the evaluation is determined to be low, the process proceeds to step S8024, and if the evaluation is determined to be high, the process proceeds to step S8025.

In step S8023, the changing means 126 changes the criteria so as to relax the criteria. This facilitates the provision of advice by computer system 100 . For example, modifier 126 may relax the criteria by reducing at least one criterion and/or removing at least one criterion. For example, modifying means 126 compares the audio and/or video captured in the past communication with the criteria used in the past communication and identifies differences. Then, the changing means 126 changes the reference for the different parts. For example, the modifying means 126 can remove the different parts from the reference. Alternatively, for example, modifier 126 may mitigate or strengthen the difference.

In step S8024, the modifying means 126 modifies the criterion to strengthen it. This makes it difficult for computer system 100 to provide advice. For example, modifier 126 may strengthen the criteria by raising at least one criterion and/or adding new criteria. For example, the change means 126 compares the audio and/or video acquired in the past communication with the reference used in the past communication, and the portion where the difference from the reference is relatively small (i.e. Identify the portion of the audio and/or video captured in the past communication that barely meets the criteria). Then, the changing means 126 changes the reference for the part where the difference from the reference is relatively small. For example, modifier 126 may raise the criteria such that the audio and/or video captured in the past communication no longer meets the criteria.

In step S8025, the changing means 126 decides not to change the criteria. This is because it can be determined that the logic used in the past communication is sufficient. This allows the processing to end.

In this way, the modifying means 126 can modify the criteria for deriving advice.

The process 800 will be described using a specific example.

Suppose that the derivation logic of the advice "Let's think together with the other party!" to be presented to Mr. X in a one-on-one meeting at a certain point in time is as follows.
If the following criteria are satisfied, the advice "Let's think together with the other party!" is given in the "start 30 minutes".
● Within 30 minutes of the start of the conversation, the words spoken by Mr. X must include the following.
"What are you having trouble with?"
"That's tough."
● Within 30 minutes of the start of the conversation, subordinate A's voice pitch, volume, speed, and number of times must meet the following: "Yes": 320 Hz or higher, 64 dB or higher, 0.21 seconds or less, 2.1 times "I see": 290 Hz or more, 61 dB or more, 0.35 seconds or less, 3.2 times or more Ratio of subordinate A's speaking time up to 30 minutes to boss X's speaking time Subordinate A 45%, Boss X 55%

On the other hand, it is assumed that the computer system 100 did not provide the advice "Let's think together with the other party!" because the above criteria were not met during the dialogue at "30 minutes to start".

In step S801, the acquisition unit 125 acquires the result of the 1on1 meeting.

In step S802, the changing means 126 changes the criteria.

Since no advice was provided in the 1on1 meeting (No in S8021), in step S8023, the changing means 126 changes so as to relax the criteria.

The changing means 126, for example, compares the voice obtained in the 1on1 meeting with the reference. As a result of comparing the voice obtained in the 1on1 meeting with the standard, there were the following differences.
○ Within 30 minutes of the start of the conversation, the words uttered by Mr. X's boss did not include "That must be hard."
○ Within 30 minutes of the start of the conversation, subordinate A's voice was 62 dB.

The changing means 126 can change the criteria as follows for the next and subsequent 1on1 meetings.
● Within 30 minutes of the start of the conversation, the words spoken by Mr. X must include the following.
"What are you having trouble with?"
● Within 30 minutes of the start of the conversation, subordinate A's voice pitch, loudness, speed, and number of times must satisfy the following: "Yes": 320 Hz or more, 62 dB or more, 0.21 seconds or less, 2.1 times "I see": 290 Hz or more, 61 dB or more, 0.35 seconds or less, 3.2 times or more Ratio of subordinate A's speaking time up to 30 minutes to boss X's speaking time Subordinate A 45%, Boss X 55%

This relaxation makes it easier for computer system 100 to provide advice. For example, in the next and subsequent 1on1 meetings, if the same communication as in the 1on1 meeting is made, the criteria will be met, and advice will be provided.

In this way, the logic for deriving advice is refined, and advice can be provided with higher accuracy in subsequent one-on-one meetings.

Process 800 will be described using another specific example.

Suppose that the derivation logic of the advice "Let's think together with the other party!" to be presented to Mr. X in a one-on-one meeting at a certain point in time is as follows.
If the following criteria are satisfied, the advice "Let's think together with the other party!" is given in the "start 30 minutes".
● Within 30 minutes of the start of the conversation, the words spoken by Mr. X must include the following.
"What are you having trouble with?"
"That's tough."
● Within 30 minutes of the start of the conversation, subordinate A's voice pitch, volume, speed, and number of times must meet the following: "Yes": 320 Hz or higher, 64 dB or higher, 0.21 seconds or less, 2.1 times "I see": 290 Hz or more, 61 dB or more, 0.35 seconds or less, 3.2 times or more Ratio of subordinate A's speaking time up to 30 minutes to boss X's speaking time Subordinate A 45%, Boss X 55%

On the other hand, it is assumed that the computer system 100 provides the advice "Let's think together with the other party!" because the above criteria are met during the "start 30 minutes".

At step S801, the acquisition unit 125 receives the result of the 1on1 meeting.

Satisfaction with the 1-on-1 meeting evaluated by subordinate A was 1 point out of 6, and was well below the average of the past 1-on-1 meetings between subordinate A and boss X.

In step S802, the changing means 126 changes the criteria.

Advice is provided in the 1on1 meeting (Yes in S8021), and the evaluation is low (in S8022), so in step S8024, changing means 126 changes the criteria so as to strengthen them.

The changing means 126, for example, compares the voice obtained in the 1on1 meeting with the reference. As a result of comparing the voice obtained in the 1on1 meeting with the standard, there was a part with the smallest difference from the standard as shown below.
○ Within 30 minutes of the start of the conversation, subordinate A's voice was 325 Hz.

The changing means 126 can change the criteria as follows for the next and subsequent 1on1 meetings.
● Within 30 minutes of the start of the conversation, the words spoken by Mr. X must include the following.
"What are you having trouble with?"
"That's tough."
● Within 30 minutes of the start of the conversation, subordinate A's voice pitch, loudness, speed, and number of times must meet the following: "Yes": 330 Hz or higher, 64 dB or higher, 0.21 seconds or less, 2.1 times "I see": 290 Hz or more, 61 dB or more, 0.35 seconds or less, 3.2 times or more Ratio of subordinate A's speaking time up to 30 minutes to boss X's speaking time Subordinate A 45%, Boss X 55%

Relaxing in this way makes computer system 100 less likely to provide advice. For example, in the next and subsequent 1on1 meetings, if the same communication as in the 1on1 meeting is made, the criteria will not be met, and no advice will be provided.

In this way, by repeatedly relaxing and strengthening the criteria for providing advice, it is possible to derive the optimal criteria (i.e., the criteria for which advice is provided and a high evaluation of communication can be achieved).

It should be noted that although the above example describes that each step is performed in a specific order, the order in which each step is performed is not limited to what is shown. Each step can be performed in any order that is logically possible.

In the examples described above with reference to FIGS. 7A, 7B, 8A, and 8B, the processing of each step shown in FIGS. Although it has been described that it is implemented by the program stored in the unit 130, the present invention is not limited to this. At least one of the processes of each step shown in FIGS. 7A, 7B, 8A, and 8B may be realized by a hardware configuration such as a control circuit.

In the above example, the computer system 100 is implemented as a server device, but the present invention is not limited to this. Computer system 100 may also be implemented by any information terminal device (eg, terminal device 300).

The invention is not limited to the embodiments described above. It is understood that the invention is to be construed in scope only by the claims. It is understood that a person skilled in the art can implement an equivalent range from the description of specific preferred embodiments of the present invention based on the description of the present invention and common technical knowledge.

INDUSTRIAL APPLICABILITY The present invention is useful as providing a computer system, method, and program for supporting communication between two parties.

100 computer system 110 interface section 120, 120' processor section 130 memory section

Claims (15)

A computer system for supporting communication between two parties, wherein a first person of said two parties uses a first terminal device, and a second person of said two parties uses a second using two terminal devices, voice and/or video during the communication is acquired by the first terminal device and/or the second terminal device;
a first receiving means for receiving information indicating a response that a first person of the two parties expects of a second person of the two parties in the communication;
a second receiving means for receiving the audio and/or video acquired by the first terminal device and/or the second terminal device;
derivation means for deriving advice in said communication based on said information and said audio and/or said video, said derivation means determining at least one criterion associated with said expected response to said audio and/or said video; and/or deriving means for deriving said advice associated with said at least one criterion when said video meets;
providing means for providing said derived advice during said communication to said first person via said first terminal and/or to said second person via said second terminal; When
A computer system comprising 2. The computer system of claim 1, further comprising modifying means for modifying said at least one criterion based on a result of said communication. The changing means is
(i) altering the at least one criterion to relax the at least one criterion when no advice is provided during the communication;
(ii) altering the at least one criterion to reinforce the at least one criterion when advice is provided during the communication and the communication is rated poorly;
3. The computer system of claim 2. The second receiving means receives at least the audio,
The derivation means extracts voice features of the first person and/or the second person from the voice,
deriving the advice based at least on the information and the audio features;
The computer system according to any one of claims 1-3. The characteristics of the voice of the first person include the pitch of voice when the first person utters a predetermined phrase, the speed of voice when the first person utters a predetermined phrase, the 5. The computer system according to claim 4, comprising at least one of the volume of sound when the first person utters a predetermined phrase, and the number of times the first person utters a predetermined phrase. 6. The computer system of claim 4 or claim 5, wherein the voice features of the second person include phrases contained in the second person's utterances. 7. Any one of claims 4 to 6, wherein the voice features of the first person and the second person include the ratio of the speech time of the first person to the speech time of the second person. the computer system described in . The second receiving means receives at least the video,
The derivation means is
extracting motion features of the first person and/or the second person from the video;
Computer system according to any one of claims 1 to 7, adapted to derive said advice based at least on said information and said movement characteristics. 9. The computer system of claim 8, wherein the movements include at least one of facial movements, arm movements, and head movements. 9. The computer system of claim 8, wherein the motion characteristics include at least one of nodding amplitude, nodding speed, nodding frequency, glabellar wrinkles, and facial expressions. 11. The providing means according to any one of claims 1 to 10, wherein said advice is displayed on a video displayed on said first terminal device and/or said second terminal device during said communication. computer system. the advice is provided to the second person via the second terminal device;
The computer system according to any one of claims 1 to 11, wherein said advice includes what said second person should heed in said communication. The computer system according to any one of claims 1 to 12, wherein said first person is a member in an organization and said second person is said member's superior. A method for supporting communication between two parties, wherein a first person of said two parties uses a first terminal device and a second person of said two parties uses a second wherein audio and/or video during said communication is acquired by said first terminal device and/or said second terminal device, said method being executed in a computer system comprising a processor unit; The method includes:
the processor unit receiving information indicating a response that a first person of the two parties expects from a second person of the two parties in the communication;
the processor unit receiving the audio and/or video acquired by the first terminal device and/or the second terminal device;
deriving advice in said communication based on said information and said audio and/or said video, said deriving comprising at least one criterion associated with said expected response; /or deriving the advice associated with the at least one criterion when the video meets;
The processor unit provides the derived advice to the first person via the first terminal device and/or to the second person via the second terminal device during the communication. to provide to
method including. A program for supporting communication between two parties, wherein a first person of the two uses a first terminal device, and a second person of the two uses a second terminal device. using the terminal device, voice and / or video during the communication is acquired by the first terminal device and / or the second terminal device, the program is executed in a computer system comprising a processor unit, Said program
Receiving information indicating a response expected from a second one of the two parties in the communication by the first one of the two parties;
receiving the audio and/or video obtained by the first terminal device and/or the second terminal device;
deriving advice in said communication based on said information and said audio and/or said video, said deriving comprising at least one criterion associated with said expected response; /or deriving the advice associated with the at least one criterion when the video meets;
providing said derived advice during said communication to said first person via said first terminal and/or to said second person via said second terminal;
A program that causes the processor unit to perform processing including

Images