JP4069207B2 - Communication device - Google Patents

Description

  The present invention relates to a communication device that supports voice communication by providing received voice data to client terminals of a plurality of users.

  Now that the Internet environment has become widespread, non-face-to-face text-based communications such as e-mail, electronic bulletin boards, and chat are routinely used. Above all, the text chat has a point that leads to a face-to-face conversation in that it communicates sequentially. On the other hand, it has the advantage over face-to-face conversation that “multi-threaded conversation” is easily possible. Here, the multi-thread conversation is a conversation in which conversations on a plurality of topics proceed simultaneously in a conversation space, and a certain participant is simultaneously participating in a plurality of topics. Even in face-to-face conversations, if a large number of people gather, multiple topics may be developed simultaneously in the same place, but this is merely a division of the group based on topics, so it is not considered a multi-threaded conversation.

  In daily face-to-face conversations, all participants in a conversation need to share a single topic for a certain period of time, and it is necessary to speak about that topic, and it is required that there is only one speaker (synchronous Constraints). Therefore, it can be said that face-to-face dialogue is inefficient. In addition, as a result, even if you come up with something different from the topic that is currently being exchanged, it is often impossible to utter immediately and forget the content of the idea.

  In addition, there is a big difference between text chat and face-to-face conversation in terms of the presence or absence of the influence of the positional relationship of the participants. In the case of face-to-face conversation, especially in situations where a large number of people share a place, such as meetings and dinners, the location of the seats that are first attached determines the categories of people who are likely to communicate and those who do not (the proximity Constraints). To talk to someone who is away from your seat, you need to move your seat or call out loud. In the former case, the conversation itself is interrupted, and in the latter case, the conversation of the calling party is interrupted. In either case, the conversation progresses greatly.

  On the other hand, since there is no such restriction in the text chat dialog, it is possible to speak the contents related to the thoughts and the past arbitrary utterances without worrying about the utterance order. That is, since it becomes possible to perform a multi-threaded dialogue, the dialogue is made efficient. In the text chat conversation, it can be said that the speech history is a conversation space, and the login name of each participant displayed in the speech history is a part of the participant. Here, only the presence information of who is participating is meaningful, and there is no concept of the positional relationship of the participants. Therefore, unlike face-to-face dialogue, it is possible to speak without worrying about the positional relationship of participants. Therefore, it is very effective when collecting ideas without fail at, for example, planning meetings.

  However, in text dialogue, it is difficult to make enough remarks because of the troublesome character input. For this reason, there is a need for a device that supports communication by voice dialogue, particularly a device that can realize multi-thread dialogue by voice dialogue.

  As an apparatus that supports communication using voice data, the following Non-Patent Document 1 discloses an automatic sound such that a voice of a speaker belonging to the same thread can be heard more clearly using voice recognition regardless of a spatial positional relationship. “The Mad Hatter's Cocktail Party”, a simultaneous conversation environment with added effects, is disclosed. In this system, since the same thread is automatically determined using speech recognition, a problem of recognition accuracy arises and the reliability is low.

PMAoki, M. Romaine, MH Szymanski, JD Thornton, D. Wilson, and A. Woodruff: The Mad Hatter's Cocktail Party: A Social Mobile Audio Space Supporting Multiple Conversations, Proc. ACM SIGCHI Conf. OnHuman Factors in Computing Systems, pp. 425-432,2003.

  However, in conversations using voice (face-to-face conversations, telephone calls, voice chats), multi-threaded conversations are currently impossible or very difficult. The main reason for this is that, in general, a person cannot hear and remember multiple statements at the same time and understand them.

  In order to conduct face-to-face multithreaded conversations, we need to hear, remember and understand multiple statements at once. However, this is difficult within the human cognitive ability. Therefore, multithreaded conversations are rarely performed. On the other hand, in the text chat, data related to the utterances of who and when have been uttered is stored and displayed as the utterance history, and the participants can freely browse the history. Since this plays a role of supplementing human short-term memory, the restriction of synchronism is eliminated, and the restriction of proximity is also eliminated because the speech history functions as a place for dialogue. As a result, it seems that multi-threaded conversation is possible in text chat. In other words, speech history is indispensable for realizing multi-threaded dialogue.

  In the voice chat system, individual utterances are recorded on the computer as voice data, and a log list indicating when and who uttered is also displayed. Therefore, the voice chat system can be listened to as many times as necessary. In other words, since the short-term memory of a person is assisted by the storage of voice data and logs, it is a voice communication medium that facilitates multi-threaded conversation rather than face-to-face conversation and telephone.

  However, the relevance of individual utterances (voice data) is not visible on the log, and it is not possible to know which utterance (voice data) is connected to which utterance (voice data) unless you listen to it after playing the sound. . For this reason, it is still difficult to implement multi-threaded conversations with existing voice chat systems.

  The present invention has been made in view of such a situation, and supports communication by voice. A communication device capable of realizing multi-thread conversation by making it possible to grasp a mutual relationship between voice data. The purpose is to provide.

  The inventors investigated the tendency of what measures are taken in order to maintain a multi-thread situation when a conventional text chat system is used. The results are shown below.

As a result of preliminary analysis of the data obtained in the chat dialogue recording, there are three types that are considered to contribute to follow the flow of the story while making it difficult to refer to the speech history, and to promote the dialogue as smoothly as possible. It was found that there is an expression of.
1) An expression (including proper nouns) that clearly indicates to whom the remark is directed (">People" in FIG. 6)
Example) This year is still coming. > Mr. B 2) An expression that clearly indicates which utterance is relevant (">word" in FIG. 6)
Example) It seems like you can get an opinion> Maron cream 3) An expression that clearly states which statement you are speaking (in this case, it is assumed that you are copying and pasting) (">Copy" in Fig. 6) ""
Ex) A: Don't you think chestnuts and fresh cream do not match? >ALL> I think it fits very well. Maron cream is really good!
("Don't think chestnuts and fresh cream don't fit?>ALL" was copied and pasted.)

  A graph of the ratio of these three expressions appearing during the dialogue is shown in FIG. Further, FIG. 7 shows a graph showing the result of appearance frequency obtained by classifying the three expressions for each interval (inter-utterance distance) after performing the semantic connection determination between the speeches. The data to be analyzed is the speech history data for a total of 870 utterances for 3 dialogues for 311 dialogues for 3 dialogues and 559 for 3 dialogues. is there.

  As can be seen from FIG. 6, when the three expressions are combined, one of the expressions is used to smoothly advance the story at a rate of about 1/4 of the analysis target speech. Moreover, when FIG. 7 is seen, when the adjacent utterances are different topics (when the distance between utterances is 2 or more), the frequency of appearance of any expression increases. For this reason, in order to make it easier to follow the flow of the story as the situation becomes more complicated, it is necessary to clearly indicate the relationship between the statements, such as who is speaking, which story, and what is the statement. I can guess that.

From the above results, the inventors developed the communication device of the present invention. The communication apparatus of the present invention includes means for receiving voice data from client terminals of a plurality of users, means for storing and storing the received voice data, and a list of the stored and stored voice data in the client terminals of the plurality of users. It is possible to perform a dialogue using a plurality of users using voice data, comprising means for providing and means for providing the requested voice data to the client terminal in response to a request from the client terminal that has received the list. A communication device that
Means for adding information indicating the relationship between the voice data in the dialogue to the list , the voice data is generated by the client terminal, the preceding voice data is selected from the list of the voice data, and the generated When the audio data and the identifier of the selected preceding audio data are uploaded,
The means for accumulating and storing the audio data accumulates and stores the generated audio data in association with an identifier that identifies the selected preceding audio data,
The means for adding information indicating the relation between the audio data to the list adds information related to the stored audio data to the audio data list and identifies the preceding audio data in the information related to the audio data. An identifier is assigned .

According to the present invention, by referring to the list, it is possible to grasp which voice data is the voice data.

Alternatively, in the present invention, when voice data is generated by a client terminal, an identifier of a speaking partner is selected from a list of users, and the generated voice data and the identifier of the selected speaking partner are uploaded, Means for accumulating and storing voice data is means for accumulating and storing the generated voice data and the identifier of the selected speech partner, and means for adding information indicating a relation between the voice data to the list of voice data. The information on the voice data stored and stored is added to the list, and the identifier of the speech partner is added to the information on the voice data.

According to the present invention, by referring to the list, it is possible to know the speaking partner for each voice data, and to grasp the mutual relationship of the voice data.

Alternatively, according to the present invention, voice data is generated by a client terminal, preceding voice data is selected from a list of voice data, and the generated voice data and an identifier of a speaker of the selected preceding voice data Is uploaded, the means for accumulating and storing audio data accumulates and stores the identifier of the speaker of the selected preceding audio data in association with the generated audio data as the identifier of the other party of speech, The means for adding information indicating the relationship to the list of voice data adds information related to the stored and stored voice data to the list and adds the identifier of the speech partner to the information related to the voice data. To do. According to the present invention, by referring to the list, it is possible to know the speaking partner for each voice data, and to grasp the mutual relationship of the voice data.

The communication apparatus of the present invention includes a plurality of user client terminals and a server, wherein the server receives voice data from the plurality of client terminals, and stores and stores the received voice data. And means for providing a list of the stored and stored audio data to the client terminals of a plurality of users, and means for providing the requested audio data to the client terminal in response to a request from the client terminal that has received the list And a communication device that enables conversation using voice data among a plurality of users,
The server includes means for adding information indicating an association between audio data in the dialogue to the list, and each client terminal includes a sending means for sending a signal, and a receiving means for receiving a signal from the sending means, The transmitting means and the receiving means are capable of transmitting and receiving signals such that the transmitting means provided in the client terminal of the speaker indicates the receiving means provided in the client terminal of the speaking partner,
The voice data is transmitted from the client terminal of the speaker to the server, and the identifier of the speaker partner is transmitted to the server from the client terminal provided with the receiving unit of the speaking partner who has received the signal from the transmitting unit of the speaker. When,
The server accumulates and stores the received voice data and the received identifier of the speaking partner, adds information about the voice data to the list of the voice data, and adds the information about the voice partner to the information about the voice data. An identifier is assigned .

According to the present invention, when a speaker transmits voice data from a client terminal to a communication apparatus and transmits a signal to the receiving unit of the speaking partner using his own transmitting unit, the speaking partner includes a receiving unit that receives the signal. The identifier of the speaking partner is transmitted from the client terminal to the communication device. The communication device accumulates and stores the received voice data and the received identifier of the speaking partner, adds information on the voice data to the list of the voice data, and adds the information on the voice partner to the information on the voice data. Give an identifier . The speaker can add the information of the speaking partner to the list only by sending a signal to the receiving means of the speaking partner using the transmitting means, such as pointing to the speaking partner.

The communication device of the present invention can input text data, and the input text data is added to the list . According to the present invention, communication can be achieved not only by voice data but also by text data. For example, it is possible to input specific and detailed contents by voice, and input only the titles and keywords of the contents as characters, and the relationship between individual voice data can be clearly indicated. It becomes. Also, “emoticons” that cannot be input by voice can be added to the voice and input.

According to the communication device of the present invention, users who participate in communication can grasp the mutual relationship of voice data by referring to the list , so that a multi-threaded conversation by voice is possible. In particular, when the information indicating the relationship between the voice data includes information that identifies the speaker and the other party, it is possible to specify who speaks and who speaks for each voice data. You can see if there is. In addition, when information indicating the preceding voice data is included as the information indicating the relationship between the voice data, it is possible to grasp which of the preceding voice data the individual voice data has received. it can. If both of these are included, the interrelationship between the audio data can be grasped in more detail.

The user can specify the speech partner of the speech data simply by selecting the speech partner, and by selecting the preceding speech data in the dialogue, the speech data received any speech data. Therefore, it is possible to easily associate the audio data, and the multi-threaded dialogue can be smoothly advanced without delay.

Further, by using a transmitting means for transmitting a signal and a receiving means for receiving the signal, information for specifying the speaking partner can be added to the list as if the speaker points to the speaking partner. If this system is introduced to face-to-face conversations, not only multi-threaded conversations are possible, but a completely new and efficient conversation style can be established. This can be expected to be very effective in meetings and the like. In addition, the conversation partner is seen by the operation of directing the transmission means to the speech partner side, and a situation in which the log data is continuously viewed is avoided, and the dialogue proceeds more naturally.

Furthermore, when the text data is added to the list, by referring to the list, it is possible to grasp the semantic connections between the audio data. As a result, multi-threaded conversation can be performed more efficiently. Further, in the conventional mixed text / voice chat system, participants who do not have voice playback means cannot understand the contents of the voice data at all. According to this system, it is possible to roughly understand the contents of audio data by using text data such as added titles and keywords.

(First embodiment)
Hereinafter, the communication apparatus S according to the present invention will be described with reference to the drawings. FIG. 1 is an explanatory diagram illustrating the communication device S according to the present embodiment. The communication device S is realized by a server that is a computer system, and can be connected to a plurality of client terminals C1,..., Cx that are also computer systems via a network such as the Internet.

  The communication device S is a system that supports voice communication by receiving voice data from the user's client terminals C1, Cx, and providing the received voice data, such as a voice chat system. .

  The client terminals C1,..., Cx can access the communication device S via a network for participating in voice communication. The client terminals C1,..., Cx have a recording function for recording voices, and the recorded voices can be uploaded as voice data to the communication device S via the network.

  The communication device S displays a user interface screen on each of the client terminals C1,... Cx in response to a request from the client terminals C1,. FIG. 2 is a diagram showing a user interface screen provided by the communication device S to the client terminals C1,.

  In the client terminals C1,..., Cx, when the handle name is input in the column 1 for inputting the handle name that is an identifier of the participant and the login button 2 is pressed, the communication device S receives the signal, In the user list column 3 of the user interface screen, an identifier (in this case, the handle name) of the logged-in user and the log data stored and stored in the history list column 4 are displayed.

  The communication device S causes the client terminals C1, Cx to realize the following functions by a program provided simultaneously with the user interface screen.

(Recording function)
The communication device S realizes a recording function for causing the client terminals C1,..., Cx to record sound and generate sound data and upload the sound data to the communication device S. There are three types of recording functions to be realized: (1) normal recording function, (2) preceding voice data designation recording function, and (3) speech partner designation recording function.

(Normal recording function)
When the client terminals C1,..., Cx detect that the “Record” button 6 is pressed in the recording-related button group 5 on the user interface screen, the “utterance complete” button as shown in FIG. Display on the screen in a cover-up form and start recording. When the client terminals C1,..., Cx detect that the “speak complete” button is pressed, the recording is terminated, and the generated voice data and the identifier of the speaker of the voice data (here, the handle name) are sent to the communication device S. Upload. Here, the speaker is a person who has made a speech that is the source of the voice data, and is a person who generates and uploads the voice data.

  FIG. 4 is a diagram showing log data extracted from the column 4 of the history list on the user interface screen. The communication device S uses an identifier a1 that specifies voice data, an identifier (handle name) a2 that specifies a speaker, and an upload time a3 that is a voice time as basic log data related to the received voice data. The log data is stored in association with the log data. Next, the communication device S provides the basic log data to each of the client terminals C1,..., Cx, and adds the generated basic log data to the history list column 4 of the redisplayed user interface screen. Display.

(Preceding voice data designation recording function 1)
Further, the client terminals C1,..., Cx have a function of uploading the generated voice data and the preceding voice data in association with each other. After the client terminal C1,..., Cx completes the recording of the voice by the normal recording function, the preceding voice data is selected from the history list column 4 on the user interface screen, and the “preceding speech designation” button 7 is selected. When is pressed, the identifier a1 specifying the selected preceding voice data is uploaded to the communication device S together with the voice data and the speaker identifier (handle name). The history list shows the history of audio data, and also plays the role of a list of preceding audio data.

(Preceding voice data designation recording function 2)
As another preceding voice data designation recording function, when the preceding voice data is selected from the history list column 4 and the “Reply recording to reply” button 8 is pressed, a “speak complete” button is displayed. And start recording the remarks. When the “speak complete” button is pressed, the recording is terminated, and the generated voice data, the identifier (handle name) of the speaker, and the identifier 1a that identifies the selected voice data are uploaded to the communication device S. To do.

  The communication device S receives the voice data generated by one of the two preceding voice data designation recording functions, the speaker identifier (handle name), and the identifier a1 that identifies the selected preceding voice data. Then, an identifier a1 for specifying the selected preceding audio data is added to the generated audio data and basic log data, and these are stored in association with each other. Thereafter, the basic log data and the identifier a1 of the selected preceding audio data are provided to each of the client terminals C1,. In the history list of the client terminals C1,..., Cx, basic log data is displayed, and an identifier a4 that identifies preceding audio data, for example, “>> [2]” at the end of the basic log data. Is granted.

(Speaking party specified recording function 1)
Further, the client terminals C1,..., Cx have a speech partner designation recording function for performing recording by designating a speech partner. There are three patterns for this function. As a first pattern, a speech is recorded by the normal recording function, a speech partner is selected from the column 3 in the user list, and when the “designate speech partner” button 9 is detected, the “speaker designation” is confirmed for confirmation. "Button 9 is changed to the identifier (handle name) of the selected speech partner. When it is detected that the changed button 9 is pressed, the generated voice data and an identifier (handle name) for specifying the speaking partner are uploaded to the communication device. For example, it is assumed that “Bob” is selected from the column 3 of the user list after the normal recording is completed. In this case, the “speak partner designation” button 9 is changed to a button with a display of “> Bob”, and when pressing of the button 9 is detected, the speech designating “Bob” is completed.

(Speaking party specified recording function 2)
Further, as a second pattern, when the client terminal C1,..., Cx detects that the user to be a speaking partner is selected from the column 3 of the user list, "" Button is displayed on the display screen and recording starts. When it detects that the “speak complete” button is pressed, the recording is terminated, and the generated voice data, the identifier of the speaker (handle name), and the identifier of the selected speaker (handle name) are uploaded to the communication device S. To do.

(Speaking party specified recording function 3)
In the third pattern, the speaker of the voice data selected from the history list is designated as the speaking partner. When the client terminal C1,..., Cx selects the voice data from the history list column 4 and detects that the “record reply to speaker” button 11 is pressed, the client terminal C1,. Start recording. When pressing of the “speak complete” button is detected, the recording is terminated, and the generated voice data, the speaker identifier (handle name), and the speaker identifier (handle name) of the selected voice data are communicated. Upload to device S. The identifier of the speaker of the selected voice data is handled as the identifier of the speaking partner.

  The communication device S receives the voice data received by the above function, the basic log data, and the identifier of the selected speech partner (“speaker identifier of the selected speech data” in the speech partner designation recording function 3). The information is stored and stored, and the information is provided to each of the client terminals C1,. In the history list column 4 of the client terminals C1,..., Cx, basic log data is displayed, and an identifier (handle name) a5 of the user selected as the speaking partner is added. For example, in the history list column 4 displayed on the client terminals C1,..., Cx, the identifier (handle name) of the message partner is given at the end of the basic log data, such as “> Bob”. In addition, when you receive a designation from other users as a speaking partner, ">>> You:" is displayed at the beginning of the line.

  The above functions can be operated even when combined. For example, the basic log data is also provided with a function of giving both the designation of the speaking partner and the designation of the preceding voice data, such as “> Bob >> [4]”. In addition, the speech partner designated recording function has a function of designating a plurality of speech partners, such as “> Susie> Andy”.

(Playback function)
The communication device S causes the client terminals C1, Cx to realize a reproduction function for reproducing the received audio data. The client terminals C1,..., Cx communicate log data corresponding to the selected log data when the log data is selected from the history list column 4 and the “listen to this” button 12 is pressed. Request to device S. When the communication device S receives a voice data transmission request from the client terminals C1,..., Cx, the communication device S provides the received voice data to the client terminals C1,. The client terminals C1,... Cx reproduce the received audio data.

  Further, when the client terminal C1,..., C3x detects a double click in the column 4 of the history list, it requests the communication device S for audio data corresponding to the double clicked log data. The communication device S provides the requested voice data to the client terminals C1,... Cx, and the client terminals C1,. According to this, since the audio data can be reproduced only by double clicking, the operability is good.

  Further, as an additional function of the reproduction function, when it is detected that the “listen to next” button 13 is pressed, the communication device S is requested for the audio data next to the audio data reproduced immediately before in the order of the log data arrangement. The communication device S provides the requested voice data to the client terminals C1,... Cx, and the client terminals C1,. Using this function, you can listen to audio data in the order of the log data in the history list (in chronological order).

  As an additional function of the playback function, when the pressing of the “listen to me” button 14 is detected, only the audio data corresponding to the log data to which the identifier (handle name) of the user who operated the button is given as the speaking partner is provided. Is requested to the communication device S. The communication device S provides only the audio data to the client terminals C1,... Cx, and the client terminals C1,. If there are multiple addresses for the user, the audio data is later than the audio data selected in the column 4 of the history list (the audio data uploaded later) and is closest in time to the selected audio data. Play audio data. You may make it reproduce | regenerate sequentially in time series. Using this function, you can listen to only the audio data addressed to you from the history list.

  Further, as an additional function of the playback function, when it is detected that the “listen to preceding speech” button 15 is pressed, the communication device S is requested to provide preceding audio data added to the log data specified from the history list column 4. To do. The communication device S provides only the audio data to the client terminals C1,... Cx, and the client terminals C1,. Using this function, you can hear the related previous audio data retroactively.

(Second Embodiment)
FIGS. 5A and 5B are explanatory diagrams for explaining the communication device S of the present embodiment. The communication apparatus S according to the present embodiment is mainly used in a face-to-face conversation such as a meeting, and realizes a multi-thread conversation while performing the face-to-face conversation. The communication device S is provided with a function capable of designating a speaking partner by performing an operation such that the speaker points to the speaking partner in the communication device S of the above embodiment.

  The communication device S includes a transmission unit b1 that transmits a signal and a reception unit b2 that receives a signal transmitted from the transmission unit. The transmitting means b1 has an infrared light emitting diode (LED) built in at the tip thereof, and an identifier is transmitted by infrared light when a switch is pressed. Each of the sending means b1 has one by one, and a unique identifier is set for each of the sending means b1 so that the identifier to be sent is different for each sending means b1. One receiving means b2 is connected to each client terminal C1,..., Cx via an RS-232C interface.

  Hereinafter, the client terminal C1 of the speaker and the client terminal C2 of the speaking partner will be described as an example (FIG. 5B). The speaker uploads the voice data and the identifier of the speaker from the client terminal C1 to the communication device S by using the voice recording function, and, as shown in FIG. Press the switch. When the sending means b1 sends the speaker's identifier X by infrared light, the receiving means b2 receives the identifier X at the infrared light receiving unit, and sends the identifier X to the client terminal C2 of the speaking partner to which the receiving means b2 is connected. Tell. The client terminal C2 notifies the communication device S of the received identifier X and the identifier Y of the speaking partner who uses the client terminal C2.

The communication device S receives the voice data and the speaker identifier X from the speaker client terminal C1, and receives the speaker identifier X and the speaker partner identifier Y from the client terminal C2 of the speaker. To do. When the identifier X of the speaker received from the client terminal C1 of the speaker matches the identifier X of the speaker received from the client terminal C2 of the speaker partner, the identifier Y of the speaker partner is added to the log data of the received voice data. Append. The speaker can add the information of the speaking partner to the log data of the voice data by a simple operation that points to the speaking partner only by sending a signal to the receiving unit b2 by the transmitting unit b1. The received speaker identifier X may be added to the log data of the audio data. Thereby, the information of the speaker can be added to the log data of the voice data by a simple operation in which the speaker indicates the partner of the speaker. It is more effective to add both the identifier Y of the speaking partner and the identifier X of the speaking party.

(Third embodiment)
The communication device S of the present embodiment is capable of inputting text data in addition to voice data. When the communication device S receives the identifier a1 specifying the text data and the voice data from the client terminals C1,..., Cx, the communication device S stores the text data in association with the log data of the voice data. The text data is provided to the client terminals C1,..., Cx together with the log data of the voice data, and the text data is displayed together with the log data in the history list column 4 of each client terminal C1,. As a result, the relationship between the individual audio data can be clearly specified. Also, “emoticons” that cannot be input by voice can be added to the voice and input.

(Effectiveness experiment)
An experiment was conducted to confirm the effectiveness of the present invention by taking the first embodiment as an example. The experiment was conducted under the following three conditions for 14 groups of 2 groups consisting of 7 graduate students. All subjects had some form of text chat experience but no voice chat experience. In addition, the order of use of individual groups is different in order to suppress the effects of system familiarity.

・ Base: A voice chat system with a general interface is used in non-face-to-face situations. Actually, among the functions provided by the communication device of the present invention (hereinafter referred to as “ChaTEL”), the one that can use only the speech history and the “listen to this” and “record” buttons was used.
・ Non-face-to-face ChaTEL: Use ChaTEL in a non-face-to-face situation.
・ Face-to-face ChaTEL: Use ChaTEL in face-to-face situations.

  In the experiment, the group consisting of 7 subjects was first divided into two subgroups of 3 and 4, and each subgroup was given a different topic. Taught to talk for 20 minutes. The topics I gave were relatively free themes such as “places I would like to go to” and “about the old play”. Regarding these topics, we requested that each subgroup continue to talk until it is completed, but we are not allowed to talk about other topics or participate in topics from other subgroups. Not done. In the non-face-to-face condition, the experiment was performed in a place where all members were completely separated from each other, and in the face-to-face condition, settings were made so that everyone was placed in a circle.

  It is expected that the dialogue partner itself will be affected by the speech partner designation and preceding speech designation functions provided in this system, since it will be easier to deal with multi-thread situations. Therefore, we first compared the dialog structure when using the Base system that does not have the speaking partner designation and preceding speech designation functions in non-face-to-face situations and when using ChaTEL. Here, in order to compare dialog structures, we identify which utterances of the individual utterances of the data obtained in the experiment are semantically connected, and use the concept of tree structure, Number of paths), number of paths (number of paths connecting individual utterances within one thread), and number of ends (number of threads at the end). Also, the difference between the ID of the start message of each thread and the ID of the last message is the thread length. This can be used as one of the materials for determining whether each thread is deployed widely or deeply. The results are shown in Table 1.

  From Table 1, A.I. B It can be seen that there is no significant difference in the items excluding the thread length for both the subject groups in Base and this system. About thread length, it turns out that the subject group of both A and B tends to become larger when this system is used rather than Base. This means that even if there is a branch of remarks, one thread will continue longer if this system is used. If the number of threads themselves is almost the same, the longer the thread continues, the more multi-thread situation will occur. It means that it is easy to occur. Therefore, when this system is used, multithreading can be continued longer.

  In addition, we calculated how much difference there was in the number of threads that each participating member participated simultaneously while using the system, between the Base system that does not have the speaking partner designation and preceding speech designation functions and this system. In addition, the number of simultaneous participation threads of each participant was obtained as follows. First, for each thread, the first and last utterances by individual participants were sought, and during both utterances, the participants were considered to be participating in the thread. After that, when each remark was made, it was calculated and calculated how many of the existing threads each participating member was participating in. The results are shown in Table 2.

  From Table 2, it can be seen that the average number of simultaneously participating threads is larger when both the A and B subject groups use this system than Base. Also, the fact that the average number of threads participating in Base is 1.0 or less means that you are devoted to listening until you have a chance to speak, and the average number of threads participating in this system. Is 1.0 or more, suggesting that the user always speaks in one thread and tries to speak in another thread.

  From the previous section, it was found that this system, which has the function to specify the other party and the previous one, is functioning effectively in the occurrence of multi-thread situations. However, this alone is not sufficient in terms of whether it is effective in a multi-thread situation in a face-to-face situation. Therefore, a comparison experiment was performed between the case where this system was used in a non-face-to-face situation and the case where it was used in a face-to-face situation. First, in this case as well, the dialog structure when using both systems was compared in the same way as in the previous section. The results are shown in Table 3.

表３より、どちらの被験者群も、非対面、対面状況での対話構造に大きな差は見られなかった。

From Table 3, there was no significant difference in the dialogue structure between the non-face-to-face and face-to-face situations in either group of subjects.

  Furthermore, in order to see the difference between the non-face-to-face situation and the face-to-face situation, the video recording data recorded during the experiment in the face-to-face situation was examined. Among them, it was confirmed from both audio data and video that in the non-face-to-face situation, the voice was spoken at almost the same volume as the normal face-to-face conversation, but in the face-to-face situation, the voice was spoken. Furthermore, it was observed that the majority of participants were gazing at the monitor on which the utterance history was displayed, even though the participant members were in front of them in the face-to-face situation. From these, it can be inferred that the use of this system in the face-to-face situation interacts while maintaining the same place physically but maintaining its own independent space. On the other hand, there were some scenes that responded to the laughter of other participating members and shared the atmosphere of the place.

  According to the communication device of the present invention, in order to enable face-to-face multi-thread conversation without restrictions on proximity and synchronization, it is possible to efficiently input speech by voice, speech history and partner designation / correspondence speech By providing a designated function, multi-threaded conversations can be performed by voice even in face-to-face situations. Furthermore, we analyzed that the developed system is effective for the progress of multi-threaded dialogue, focusing on dialogue data obtained through experiments.

  A comparison between a voice chat system with a normal interface and a system developed in this study under non-face-to-face conditions confirmed that the thread length per thread was longer in the latter than in the former. In addition, it was confirmed that using the system developed in this study increases the number of threads in which each participant is involved simultaneously. From these facts, it is shown that the same topic can be maintained for a long time by specifying the other party or preceding speech, and it is easy to create a multi-thread situation using this system. It was suggested.

Even if there are scenes sharing the atmosphere of the venue, such as sharing laughter, from the video recording during the meeting situation, it is easy to pay attention to the speech history, and the participants check the facial expression etc. It was suggested that the advantages of dialogue were not utilized. Psychologically, it is necessary to make it easier to share the face-to-face conversation space by grounding the conversation spaces that are independent of each other to the real space. Therefore, a function has been added that allows the speaker to specify the other party by “pointing” at the other party so that the movement of the line of sight can be confirmed with a natural action in a face-to-face situation.
This makes it possible to specify the person to speak by the natural act of physically pointing at the other party in the real world. By looking at the other party again, the atmosphere of the place itself becomes more natural. It became a thing.

  It goes without saying that the present invention can be modified as appropriate without departing from the spirit of the present invention.

It is explanatory drawing explaining the communication apparatus of this Embodiment. The figure which shows the user interface screen which a communication apparatus provides to a client terminal. The figure which shows the "comment completion" button. The figure which extracts and shows the log data of the column of the history list of a user interface screen. Explanatory drawing explaining the communication apparatus of this Embodiment. A graph of the rate at which three expressions appear during the dialogue. The graph which shows the result of the appearance frequency which classified three expression for every interval (distance between utterances) between utterances.

Explanation of symbols

S Communication device C1,... Cx Client terminal 1 Column for entering handle name 2 Login button 3 User list column 4 History list column 5 Recording related buttons 6 “Record” button 7 “Specify preceding speech” button 8 “ "Reply recording" button 9 "Specify speaking party" button 10 "Recording specified party" button 11 "Record reply to speaker" button 12 "Listen to this" button 13 "Listen to next" button 14 "Send to yourself" 15 button “listen to previous speech” button a1 identifier for identifying voice data a2 identifier for identifying speaker a3 upload time as speech time a4 identifier for identifying preceding voice data b1 transmission means b2 reception means

Claims (9)

Means for receiving audio data from client terminals of a plurality of users;
Means for accumulating and storing the received audio data;
Means for providing a list of the voice data stored and stored to the client terminals of a plurality of users;
In response to a request from the client terminal that has received the list, a means for providing the requested voice data to the client terminal.
It is a communication device that enables dialogue using voice data between multiple users,
Means for adding to the list information indicating the association between audio data in the dialogue;
When audio data is generated by the client terminal, preceding audio data is selected from the audio data list, and the generated audio data and an identifier of the selected preceding audio data are uploaded,
The means for accumulating and storing the audio data accumulates and stores the generated audio data in association with an identifier that identifies the selected preceding audio data,
The means for adding information indicating the relation between the audio data to the list adds information related to the stored audio data to the audio data list and identifies the preceding audio data in the information related to the audio data. A communication device characterized by assigning an identifier . Recording means for recording the voice on the user interface screen of the client terminal to generate the voice data, a list of voice data for selecting the preceding voice data, and the voice data generated by the recording means together with the voice 2. The communication apparatus according to claim 1, further comprising: a preceding speech designation unit that uploads an identifier of the preceding audio data selected from the data list. On the user interface screen of the client terminal, a list of voice data for selecting the preceding voice data, a reply recording unit for recording a reply to the selected voice data to generate voice data, and the recording And a speech completion means for uploading the voice data generated by the reply recording means for replying together with the identifier of the preceding voice data selected from the voice data list. 1. The communication device according to 1. Means for receiving audio data from client terminals of a plurality of users;
Means for accumulating and storing the received audio data;
Means for providing a list of the stored and stored audio data and the list of users to client terminals of a plurality of users;
Means for providing the requested audio data to the client terminal in response to a request from the client terminal that has received the list;
It is a communication device that enables dialogue using voice data between multiple users,
Means for adding information indicating a relationship between audio data in the dialog to the audio data list;
When voice data is generated by the client terminal, an identifier of a speaking partner is selected from the list of users, and the generated voice data and the identifier of the selected speaking partner are uploaded,
The means for accumulating and storing the audio data accumulates and stores the generated audio data in association with the identifier of the selected speech partner,
The means for adding information indicating the relation between the audio data to the list of the audio data adds information related to the stored and stored audio data to the list and adds the identifier of the speaking partner to the information related to the audio data. A communication device characterized by that. From the user list together with the recording means for recording the voice on the user interface screen of the client terminal, generating the voice data, the user list for selecting the identifier of the speaking partner, and the voice data generated by the recording means 5. The communication apparatus according to claim 4, further comprising: a speech partner designating unit that uploads an identifier of the selected speech partner. On the user interface screen of the client terminal, a user list for selecting an identifier of the speaking partner, a partner-designated recording unit for recording a speech for the selected speaking partner and generating voice data, and selecting from the user list 5. The communication apparatus according to claim 4, further comprising: a speech completion means for uploading voice data generated by the partner-designated recording means together with the identifier of the speech partner. Means for receiving audio data from client terminals of a plurality of users;
Means for accumulating and storing the received audio data;
Means for providing a list of the voice data stored and stored to the client terminals of a plurality of users;
Means for providing the requested audio data to the client terminal in response to a request from the client terminal that has received the list;
It is a communication device that enables dialogue using voice data between multiple users,
Means for adding information indicating a relationship between audio data in the dialog to the audio data list;
Audio data is generated by the client terminal, preceding audio data is selected from the audio data list, and the generated audio data and an identifier of a speaker of the selected preceding audio data are uploaded. When,
The means for accumulating and storing the audio data accumulates and stores an identifier of a speaker of the selected preceding audio data in association with the generated audio data as an identifier of a speech partner,
The means for adding information indicating the relation between the audio data to the list of the audio data adds information related to the stored and stored audio data to the list and adds the identifier of the speaking partner to the information related to the audio data. A communication device characterized by that. A list of voice data for selecting the preceding voice data on the user interface screen of the client terminal and a reply to the speaker of the selected preceding voice data are recorded to the speaker who generates voice data. A reply recording means, and a speech completion means for uploading the voice data generated by the reply recording means to the speaker together with the identifier of the speaker of the preceding voice data selected from the voice data list when the recording is completed; The communication device according to claim 7, wherein the communication device is displayed. It has client terminals and servers for multiple users,
The server includes means for receiving audio data from the plurality of client terminals;
Means for accumulating and storing the received audio data;
Means for providing a list of the voice data stored and stored to the client terminals of a plurality of users;
Means for providing the requested voice data to the client terminal in response to a request from the client terminal that has received the list;
It is a communication device that enables dialogue using voice data between multiple users,
The server comprises means for adding information indicating a relationship between audio data in the dialogue to the list;
Each of the client terminals includes a transmission unit that transmits a signal, and a reception unit that receives a signal from the transmission unit,
The transmitting means and the receiving means are capable of transmitting and receiving signals such that the transmitting means provided in the client terminal of the speaker indicates the receiving means provided in the client terminal of the speaking partner,
The voice data is transmitted from the client terminal of the speaker to the server, and the identifier of the speaker partner is transmitted to the server from the client terminal provided with the receiving unit of the speaking partner who has received the signal from the transmitting unit of the speaker. When,
The server accumulates and stores the received voice data and the received identifier of the speaking partner, adds information about the voice data to the list of the voice data, and adds the information about the voice partner to the information about the voice data. A communication device characterized by assigning an identifier.

Images