JP2007027918A - Real world communication management apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method of managing a user's behavior on a real world associated with call origination. <P>SOLUTION: A communication recording management section 2804 stores and manages communication recordings. A user name acquisition means plays a roll of integrally converting accounts recorded in the communication recording management section in a form easily understandable by a user. Means for handling various communications such as electronic mail, phone calls, and chats are connected to the communication recording management section, which integrally handles the communication recordings. The stored communication recordings are exhibited for the user via a user interface section 2805 and operated by the user. In the case of implementing operations such as a response to the existing communication recordings, a required communication utilizing means is called. An environment wherein the communication recording management means integrates the communication recordings into one is obtained. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique for monitoring and recording interpersonal communication in the real world.

  Research on human behavior in the real world has been conducted for a long time. For example, many systems have been devised that track user position information acquired using GPS (Global Positioning System) and record it on a map or the like. Such information is used when referring to the behavior of the day later. When date and time information occupies an important factor, it is convenient if time / position (time and space) information indicating when and where the user was, for example, can be used when writing a diary.

Feiner, Steven, Blair MacIntyre, Marcus Haupt, and Eliot: Solomon.Windows on the World: 2D Windows for 3D Augmented Reality.Proceedings of UIST '93 (Atlanta, GA, 3-5 November 1993), pp. 145-155. Mike Addlesee, Rupert Curwen, Steve Hodges, Joe Newman, Pete Steggles, Andy Ward, Andy Hopper.Implementing a Sentient Computing System.IEEE Computer Magazine, Vol. 34, No. 8, August 2001, pp. 50-56. Japanese Patent No. 3104204 JP 2002-7358

  In terms of recording human behavior in the real world, the so-called 5W1H, When, Where, Who, What, Why, How It is desirable to obtain the context. Among them, when and where, there are already acquired frameworks, but there are no frameworks available for the remaining elements. Context awareness methods aiming at making computers recognize these contexts with high accuracy have been actively studied in the field of computer science.

  Regarding Who in 5W1H, when the behavior of a certain user is tracked, the subject of the behavior is always the monitoring target user, so it may be considered that there is no need for acquisition. However, in reality, information regarding the communication partner of whom the user has met is also extremely important, like the subject, and if it can be obtained, it is very significant from the viewpoint of recording user actions.

  However, at present, there is generally no reliable framework for obtaining and recording information about who the user has met. Certainly, information on who the user was with can be acquired. For example, in the augmented reality (AR) field, each user hangs an identification tag with a wireless communication function from his neck, and when he meets a person, the identification tag that the other party hangs is recognized by the computer. Therefore, a system for presenting attribute information such as the person's name to the user has been devised (see Non-Patent Document 1 above). In the system of Non-Patent Document 1, when a user wears an HMD (Head Mounted Display) and observes an object, various information such as the name of the object is superimposed on the user's field of view.

  Also, a system has been devised in which each user carries a wireless transmission device and a large number of position sensors arranged in the environment constantly record the user position (see Non-Patent Document 2 above). The system of Non-Patent Document 2 is one of the classic studies in the ubiquitous field called ActiveBAT (with a former system called Active Badge). In Japan, for example, a plurality of general contractors have previously constructed a similar system.

  Even without using special equipment, the location information of each individual and the base station information of the mobile phone acquired by the GPS (Global Positioning System) installed in the existing mobile phone are integrated, and the same space-time can be obtained. By picking up the shared people, you can grasp the fact that they were in the same space and time.

  However, in the above system, it is not possible to distinguish between a person who just happens to pass by, for example, a person who has just been in the vicinity, and a person who has communicated such as a dialog. In many cases, information about a person who is simply present is not so important, and information about who communicated (for example, dialogue) is more important. If you know who, where and what you talked about today, it will be useful to refute your actions later. On the other hand, even if you know information about who you were with today, only a large number of people are listed, and in many cases it may be useless just by listing meaningless information Is expensive.

  As a communication partner recording method, for example, many patent applications have been filed regarding an electronic exchange method of business card data. For example, Patent Document 1 describes a technique related to an electronic notebook that can exchange business card data recorded in advance in the electronic notebook with a simple operation. Patent Document 2 describes a technique related to a portable electronic device that exchanges business cards without contact. A plurality of systems for exchanging business card data at the same time as shaking hands by passing a weak current through the human body and using the human body as a transmission cable have been devised. However, the above-described technique is only applicable to formal communication that requires business card exchange, and cannot be applied to daily conversation. In addition, these technologies require a special device operation and human contact for exchanging business card data, so that there is a problem that it is limited and too complicated to deal with daily communication. .

  An object of this invention is to provide the new system for confirming the user action in the real world.

  Who communicated: Real-world communication is based on utterances. According to the Mehrabian's communication model, non-verbal elements account for 93 percent of human communication. However, it is rare that communication without any utterance is established. For example, even if two people share the same time and the same space, the latter is considered to have been more importantly communicated when there is no conversation between the two and when there is an active conversation That makes sense. Therefore, it is possible to confirm user behavior in the real world by focusing on the user's utterance and automatically recording who communicated with whom.

  According to one aspect of the present invention, a communication record management unit that manages a utterance record that includes a speaker identifier that identifies a speaker, and a time when the utterance has occurred, and a time that is managed by the communication record management unit Utterance recording means for grouping utterance records close to the utterance record as dialogue records, and presenting a part or all of the conversation records generated by the utterance integration means to the user in time series based on the dialogue occurrence time Communication record presenting means is provided.

  It is preferable that the apparatus further comprises utterance record presenting means for presenting a part or all of the utterance records recorded in the utterance record management means to the user in time series. Utterance records are managed like emails in mailers. Generally, a PC is used, but the utterance notification terminal information terminal may also include the function.

  Since the utterance information arranged in time series may be filtered in some way (for example, see only the utterance record of the day, or see only the utterance record of Mr. XX) "All".

  The utterance record management means comprises user name acquisition means for acquiring a user name specified by a speaker identifier ID for identifying a speaker included in the utterance record recorded in the communication record management means, and the utterance record presentation means The communication record presenting means preferably presents the user name corresponding to the utterance. The utterance record is not displayed as it is, but is displayed after some processing (having means for replacing the user ID with the user name).

  The utterance record presenting means communication record presenting means presents the utterance records recorded in the utterance record managing means and communication record managing means as a group and presents utterances that are close in time. That is, each utterance can be handled as a grouped dialogue.

  It is preferable that the utterance record presenting means and the communication record presenting means present, as a main speaker in the dialogue group, a speaker who frequently speaks in the group among the speakers in the grouped speech dialogue. This is because a person who speaks frequently is considered to be heavily weighted as a participant. In addition, as an assignment of the user name to the group, the name of the self (own user) may be removed for the time being.

  On the other hand, it is possible to deal with a talker who does not accompany a speech by further acquiring receiver information indicating who has heard each utterance. In addition, by acquiring information on the location where the dialogue occurs, it is possible to handle a large feature of where the dialogue was performed. Since human memory is often tied to a location, information about the location of the dialogue is very important when accessing the dialogue record.

  Preferably, the dialogue record includes an utterance content in which the content of each utterance constituting the dialogue is recorded. This may be, for example, a recording of a dialogue or a summary of the dialogue content. In addition, emotions are often included in real human utterances, and emotions can be estimated to some extent by measuring the tone and height of the voice. Preferably, the utterance record further includes estimation information of the mental state of the speaker, and the dialogue record is presented together with the estimation information.

A dialog duration calculating means for calculating a duration of the grouped utterances,
The communication record presenting means utterance record presenting means preferably presents the duration calculated by the dialog duration calculating means in association with the speaker. By calculating the duration of the dialogue, it is possible to obtain information such as importance regarding the dialogue. However, since this is used for estimating the importance or the like, it is not necessarily accurate.

  The communication record presenting means utterance record presenting means preferably filters and presents only the grouped utterance dialog records whose duration calculated by the dialog duration calculating means is equal to or greater than a certain threshold. It is estimated that the longer the dialogue, the more important. Therefore, it is possible to know the characteristics of the dialog by filtering by the length of the dialog.

  Preferably, the communication record presenting means utterance record presenting means presents in ascending or descending order based on the duration calculated by the dialog duration calculating means. It is estimated that the longer the dialogue, the more important. Therefore, the importance can be seen at a glance by sorting and presenting in ascending or descending order.

  Preferably, user search means for searching only an utterance record or a dialog record including a specific speaker designated by the user from the utterance record recorded by the communication record management means is provided. Finding only a record of communication with a specific person is an extremely important function that is usually performed in email.

  According to another aspect of the present invention, the communication record management means manages an electronic mail record in addition to an utterance record, and the communication record presenting means includes an electronic mail record managed by the communication record management means, There is provided an information processing apparatus characterized in that a part or all of a dialogue record is presented to a user in chronological order based on an e-mail transmission time and a dialogue occurrence time. E-mail management means for managing e-mail, and e-mail linkage means for associating an e-mail account managed by the e-mail management means with an ID for identifying a speaker recorded in the utterance record management means The utterance record presenting means integrates the electronic mail managed by the electronic mail management means and the utterance record recorded in the utterance record management means and presents them to the user. An apparatus is provided. By using it integrated with e-mail, it can be easily handled as a communication tool such as telephone and chat.

  Preferably, the information for identifying the speaker included in the utterance record managed by the communication record management unit is associated with the e-mail address included in the e-mail record managed by the communication record management unit The account management means for managing the utterance record, and when replying to the utterance record by e-mail, the account management means obtains an e-mail address corresponding to the utterer of the utterance record, and the obtained e-mail address is Set the reply mail address. This makes it possible to send an e-mail as a reply to the utterance record, and to use different communication means called utterance e-mails in cooperation with each other.

  Similarly, when a reply to a dialog is sent by e-mail, the e-mail addresses of all participants or main participants in the dialog are acquired and set as destinations. This makes it possible to send an e-mail as a reply to the dialogue record.

  Preferably, an ID for identifying an appropriate utterance record or dialogue record is added to the In-Reply-To header indicating the reply message identifier of the reply e-mail or the References header indicating the quote message identifier. These IDs are very useful when managing communication records integrated with email.

  Extracting only communication records related to a certain person from the communication records is a very important function in using the communication records. It is preferable to have a user search means for searching only records in which the same person is involved in e-mail recording and utterance recording or dialogue recording.

  Preferably, it is desirable that not only electronic mail recording but also various communications performed via other networks, for example, chat recording, telephone recording, video telephone recording, and the like can be integrated and handled in the same manner. As such a response on the transmission side, it is preferable to perform control for notifying an utterance only to a terminal having a known (registered) ID. If it does in this way, only the utterance of the registered user will be recorded. Accordingly, privacy can be protected and noise in the utterance record (conversation record of an unrelated person) can be reduced. A framework in which different types of communication records can be used with a single interface is useful for supporting human communication.

  According to the present invention, it is possible to record who the user communicated with, so that communication in the real world can be handled in the same way as communication in the virtual world. Accordingly, it is possible to confirm user behavior in the real world by reviewing the communication record later.

  Hereinafter, real-world communication management technology according to each embodiment of the present invention will be described with reference to the drawings. In this specification, the information processing apparatus is a broad concept including a personal computer (PC), a mobile terminal (including a mobile phone), and the like.

  First, the real world communication management technique according to the first embodiment of the present invention will be described.

  An information processing apparatus that is an utterance notification apparatus according to the present embodiment includes an utterance detection unit that detects a user's utterance, and another terminal that closes the user's utterance when the utterance is detected by the utterance detection unit And utterance notification means for notifying the user. The information notified by the utterance notification means includes a speaker identifier for identifying a speaker. By including the speaker identifier, the device can recognize who was spoken to.

  Each user always carries the information processing apparatus. When the monitoring target user carrying this information processing apparatus utters, the utterance detecting means detects the user's utterance. Various methods are conceivable as an actual speech detection method. As a typical well-known method, there is a method of detecting a voice of a monitoring target user using a microphone. Here, separation of noises such as other people's utterances and environmental sounds may be a problem, but it is possible to detect the utterances of the monitored user with high accuracy by appropriately filtering by the volume level etc. . Even if the user to be monitored wears a pin microphone or the like, the speech can be detected with practical accuracy.

  Furthermore, it is possible to improve the accuracy of speech detection by combining the acquisition of an image around the mouth with a camera, detection of vocal cord vibrations, voiceprint analysis, and the like. When speaking, the user may notify the system of the utterance by pressing any button. Regarding the speech detection method, the above description is merely an example, and various known methods can be used.

  When the information processing apparatus detects an utterance, the information processing apparatus notifies an adjacent user (actually, a portable terminal carried by the user) of the occurrence of the utterance. Originally, it is desirable that notification of an utterance be performed only for a target user who is speaking. However, it is usually difficult for a computer to identify a conversation partner, and the content of the utterance can be intercepted by another person present at the place. Therefore, in the present embodiment, the utterance is notified to all users close to the uttering user. In this specification, “proximity” refers to a range in which the voice of the user's utterance can reach (recordable).

  Various methods can be considered for utterance notification. For example, by using short-range wireless communication, notification can be performed by transmitting a message notifying utterance to any terminal within the communication range. Conceivable. Since the notification of the utterance only needs to be delivered to the conversation partner, a communication radius of about 1 to 2 m, which is an average distance in general conversation, is appropriate. However, when the volume is increased by a loudspeaker or the like, the notification area should be widened, and in the case of a secret voice, the notification area should be narrowed. Therefore, the size of the notification area is variable in relation to the speech volume level. In practice, a mechanism may be provided that detects the utterance volume level and changes the size of the notification area accordingly.

  FIG. 2 is a conceptual diagram when an omnidirectional short-range wireless communication system is used in the utterance notification according to the present embodiment. As shown in FIG. 2, in the omnidirectional short-range wireless communication system according to the present embodiment, consider a case where there are six terminals A to F, for example. Terminal A201 and terminal D204, and terminal C203 and terminal F206 are interacting with each other. As indicated by the arrows, the terminals B202, C203, and F206 are moving upward in FIG. Here, it is assumed that the user of terminal A201 speaks. In that case, the terminal A 201 notifies the utterance to the terminals existing in its own utterance notification area. Here, the terminal B 202 and the terminal D 204 in the utterance notification area receive the notification and know that the terminal A 201 has uttered. On the other hand, since the terminals C203, E205, and F206 exist outside the utterance notification area of the terminal A201, the utterance of the terminal A201 cannot be recognized.

  FIG. 3 is a diagram showing a concept when a directional short-range wireless communication system is used for the utterance notification in the present embodiment. In this case, only the terminal D304 exists in the utterance notification area of the terminal A301, and the terminal B302 is a terminal outside the area. In general, in consideration of the fact that utterances are often directed in the direction of the conversation partner, it is desirable that the utterance notification be directed in the direction that the user is facing. In this case, the utterance notification area AR2 has a fan shape as shown in the figure.

  Furthermore, as described above, if the utterance notification radius can be adjusted according to the voice level, it is possible to notify the utterance only to the communication partner without performing the utterance notification to the terminal that is not the communication partner as much as possible. become.

  On the other hand, when the user is making a long-distance utterance via the communication path, an utterance notification including the speaker identifier needs to be transmitted to the receiver via the communication path. The utterance notification may be transmitted via a communication path used for transmitting the utterance (FIG. 25A) or may be transmitted via another communication path (FIG. 25B). For example, when a speaker and a receiver are having a conversation via a telephone line, the speaker's utterance reaches the receiver through the telephone line, but the utterance notification may be transmitted through the telephone line or the Internet network. It may be transmitted through other networks. If another communication path can be used, it is possible to transmit the utterance notification to the receiver even when the communication path for transmitting the utterance does not permit transmission of the utterance notification. In the case of long-distance utterance via a communication path, depending on the state of the communication path, a situation may occur in which the transmission of the utterance notification is delayed compared to the speech. Therefore, in order to associate the utterance with the utterance notification, it is desirable that the utterance notification includes information for specifying the utterance such as the utterance occurrence time.

  A speaker identifier (speaker ID) for identifying a speaker is added to the packet for notification of the utterance. As long as the speaker can be identified, a name or nickname may be used. However, considering privacy protection, a mechanical ID such as an alphanumeric list is preferred. In this case, each user can continue to use the speaker ID for a certain period, but can acquire a new speaker ID at an arbitrary timing. If the same speaker ID is used for a long time, there is a high possibility that a third party will be able to track the behavior. Therefore, it is preferable to refresh periodically like a cookie on the Internet. Of course, it is possible to continue using a unique speaker ID for a long time.

  In this way, in synchronization with the utterance of the speaker, the occurrence of the utterance and the speaker ID indicating the speaker can be notified to the receiver (terminal owned by the speaker) who is listening to the utterance. . Therefore, it becomes possible for the receiver (the terminal owned by the receiver) to recognize who is talking to whom.

  Next, a real world communication management technique according to the second embodiment of the present invention will be described. The information processing apparatus according to the present embodiment has an utterance identifier assigning means for assigning an utterance identifier (utterance ID) for identifying an utterance, and the information notified by the utterance notification means includes the utterance ID. To do. This utterance ID may include a speaker identifier (speaker ID) for identifying the above speaker (a specific example will be described later).

  The utterance ID is uniquely assigned to a continuous utterance. The continuous utterance described here refers to continuous utterances performed by the same speaker. As a method for detecting a continuous utterance, for example, if utterance detection is interrupted and a new utterance is not detected for a time longer than a threshold (for example, 0.5 seconds), it is determined that a series of utterances is completed. be able to. However, in this specification, “continuous utterance” is simply referred to as “utterance” unless there is a particular need for distinction.

  The definition of utterance that continues here is, for example, when an utterance "Good morning" is given, an utterance ID is assigned to each phoneme "O-HA-YO-U-GO-ZA-I-MA-SU". It is not intended to be shaken, but is intended to assign a single utterance ID to it. Of course, you may assign an ID individually for each phoneme (if the threshold is made extremely small in the above example, the utterance ID will be finely assigned), but in order to avoid complications, assign one utterance ID to the whole. Is preferred. That is, the utterance ID means that an ID is assigned to each utterance group, and the meaning of the group is arbitrary.

  When a speaker speaks and another speaker speaks, a dialogue is established. The length of the utterance is generally about “Good morning, Mr. XX”. On the other hand, the dialogue can be defined as a group of "Good morning, Mr. Xx", "Good morning. Good weather today" "Yes, I'm glad that the laundry dries well" ... . During the continuation of a certain utterance, the utterance of another person interrupts and the utterance is interrupted. On the other hand, the utterance may continue without interruption despite interruption of the utterance of the other person. In particular, when considering a dialogue with a large number of people, each person's utterance often overlaps in time. If the utterance constituting the dialogue is finished and the utterance is not resumed for a time longer than a certain threshold (for example, 3 seconds), it can be determined that the dialogue is completed.

  FIG. 8 is a diagram illustrating an example of a dialogue by the users A, B, and C. The horizontal axis is the time axis, and the rectangles marked as utterances on this time axis indicate that the user has made utterances for that time. An utterance ID is added to each utterance. In the example according to the present embodiment, the utterance ID includes the utterer ID, and by looking at the utterance ID, which user's utterance can be known. For example, ID: A01 indicates an utterance by user A.

  First, the user A starts an utterance identified by the utterance ID: A01, and after the utterance ID: A01 ends, after Δt1, the users B and C simultaneously start the utterance IDs: B01, C01, respectively. The user C who notices that the utterance collided interrupts the utterance, and only the user B continues the utterance. After the utterance ID: B01 ends, the user C starts the utterance ID: C02 again after Δt2. Here, regarding the interruption time Δt1, Δt2, and Δt3 of each utterance, the threshold value that is a criterion for determining the break of the dialogue is not exceeded, and only Δt4 exceeds the threshold value. Therefore, the utterance IDs A01, B01, C01, C02, and A02 can be treated as a group of utterances, that is, a dialogue. The utterance ID: C03 and the subsequent utterance (not shown) are classified into different dialogs. In the present embodiment, the conversation is simply determined based on the size of the time gap, but it is desirable that the conversation can be originally determined based on the utterance content. In addition, it is possible to determine the break of the dialogue from the change of the participant in the dialogue and the movement of the position.

  Here, since an utterance ID is added to a series of utterances, the ID can be used when referring to the utterance later. For example, suppose that a user says, “I promise to pay the bonus of this year for 10 months”, and the utterance ID: KJ3002-200503241923 is added to the series of utterances. An employee who was reluctant to pay only a bonus for two months later quoted the utterance ID: KJ3002-200503241923 Can you insist? Since the utterance ID is notified to a nearby user when the utterance occurs, the person who has heard the utterance can specify the utterance. If the utterance is recorded in association with the utterance ID, it is also possible to present the recorded content in association with the utterance ID.

  Next, real-world communication management technology according to the third embodiment of the present invention will be described. In the information processing apparatus according to the present embodiment, the utterance notification means intermittently notifies the utterance while the utterance continues. Since the utterance continues over time, the utterance notification usually needs to be performed continuously over time. FIG. 13 is a diagram illustrating the timing of continuation of utterance and utterance notification.

FIG. 13A shows an example in which an utterance notification is transmitted when the user starts utterance. Here the speaker ID: S125753 users with is, is doing a speech from time t _s over the t _e. An utterance ID: 200503241923 is added to the utterance. As described above, in the method in which the utterance notification is performed only once at the start of the utterance, the utterance notification is not transmitted to the person who enters the conversation from the middle while the utterance is continuing (the person who has moved into the utterance notification area). For this reason, the midway participant (its terminal) cannot detect the utterance. When each utterance ends in a short time, it does not become a big problem, but when uttering for a long time, it may become a problem. Although not shown, the utterance notification may be transmitted only once at the end of the utterance. In this case, there is an advantage that the duration of the utterance can be transmitted together with the method of transmitting at the start of the utterance.

  FIG. 13B is an example in which an utterance start notification and an utterance end notification are transmitted at the start and end of an utterance, respectively. In this example, the utterance end notification is transmitted simultaneously with the end of the utterance. However, when it is determined that the utterance is ended when the utterance is interrupted for a predetermined time or more, a time lag corresponding to at least the waiting time occurs. A person who has received both the utterance start notification and the utterance end notification can be regarded as having heard the utterance (in this example, the utterance with the utterance ID: 200503241923). On the other hand, a person who has received only one of the utterance notifications can be regarded as having heard a part of the utterance. However, as in the previous example, in the case of an utterance for a long time, there is a problem that a person who participates in the middle of the utterance and leaves before the utterance ends cannot receive the utterance notification.

FIG. 13C shows an example in which an utterance notification is transmitted at regular intervals while the utterance is continued. In this case, the person who participated from the middle of the utterance can also receive the utterance notification. If a serial number is added to each utterance notification transmitted in the utterance, it is possible to know what percentage of the utterance has been heard. In FIG. 13C, it is assumed that the utterance end is recognized when the utterance notification is interrupted. However, as shown in FIG. 13D, the utterance end notification may be transmitted when the utterance ends. If this is done, it is possible to prevent missed speech detection, but there is a problem that the amount of communication increases. It is necessary to appropriately adjust the transmission interval t ₀ of the utterance notification from the average utterance duration. If the user is not moving to increase the speech interval t _0, it is also possible to reduce the amount of communication. Further, the speaker identifier is not notified by all utterance notifications, but can be appropriately thinned out (for example, transmitted only once every two times).

13A shows the case where t ₀ is longer than the utterance duration in FIG. 13C, and FIG. 13B shows the case where t ₀ is longer than the utterance duration in FIG. 13D. You can also think that
As shown here, there are various patterns of the utterance notification method, but the duration of the utterance is long or short (when there are many long utterances, the method of FIG. 13C or FIG. 13D is desirable, but short. 13 (A) and 13 (B) are desirable when there are many utterances) and whether or not the user has moved (if the user does not move, problems such as FIGS. 13 (A) and 13 (B)) may occur. It is preferable to select adaptively in consideration of the above.

  Next, a real world communication management technique according to the fourth embodiment of the present invention will be described. The information processing apparatus according to the present embodiment further includes an utterance receiving unit that receives an utterance notification transmitted from another terminal, and further includes a communication record management unit that records the other utterance notification received by the utterance receiving unit. Is also possible. By recording the utterance notification, it is possible to record who (Who) spoke to.

  Further, by recording the own utterance notification notified by the utterance notification means in the communication record management means, it is possible to record who (Who) spoke to.

  Further, the information processing apparatus further includes reception notification means for returning the reception notification to the terminal that has issued the utterance notification, and the communication record management means notifies the reception notification returned for its own utterance notification. It is also possible to record in association with.

  When receiving the utterance notification, if the receiving terminal sends back a reception notification including the user ID (same as the speaker ID, which is also called the receiver ID because it indicates the receiver), who will hear the utterance? You can also know if it was done. However, in consideration of the fact that utterances are made by both participants in a normal conversation, it is possible to specify the conversation partner to a certain extent even if only the utterance notification is used without introducing the incoming notification.

  In the information processing apparatus, the utterance record managed by the communication record management means can be managed in association with the utterance time. By including time information, the machine can recognize when it was spoken (when it was spoken). Of course, the time at which the notification information is received may be confirmed even if the time information is not included in the notified information.

  In the information processing apparatus, the utterance record managed by the communication record management means can be managed in association with the utterance place. By including location information, the machine can recognize where it was spoken (where it was spoken). Of course, even if the location information is not included in the notified information, if the receiving terminal has means for acquiring the current position, the information can be used. For example, the receiving terminal can acquire the position using GPS or the like, or can acquire the position by receiving position information from a civil marker or the like placed in the environment.

  The information processing apparatus further includes an utterance content recording unit that records the utterance content, and the communication record management unit can also record the utterance content recorded by the utterance content recording unit in association with the utterance notification. If you have a microphone that records the conversation and a camera that captures the situation, you can accumulate what (What) and how (Talk) you talked to. These pieces of information can be referred to later using the above-mentioned utterance ID and utterance time as clues, and the contents of the conversation can be reconfirmed or quoted.

  However, there is no need to keep the recorded data or recorded data of the utterance contents as they are. If voice recognition and semantic analysis are available, the utterance contents can be converted into text, summarized, or added with keywords. You can also do it. For example, it is useful only when extracting a keyword that frequently appears in an utterance or when searching later.

On the other hand, there is a research result that the amount of data processed by one person in the lifetime is at the order of Pbit (P is 10 ¹⁵ ), and it is not impossible to compress all utterance records and store them in the computer. There is a possibility that it will become commonplace after 10 years.

  As of 2005, it seems that the maximum value of a personal computer's own HDD is about 1 TB (= 1,000 GB). Since 1PB is 1,000TB, if the storage capacity is increased 1,000 times, you can record all the events of your lifetime. Since the capacity of HDDs is growing at a rate of about twice a year every year, if the same pace can be maintained, the required capacity can be covered in over 10 years. With proper compression, all recordings are likely to be possible with smaller capacities.

  FIG. 15 is a diagram illustrating an example of a table for managing utterance records. Each utterance is recorded in association with an utterance ID 1501, an utterer 1502, a receiver 1503, an utterance time 1504, an utterance place 1505, an utterance duration 1506, and an audio file 1507 in which the utterance contents are recorded. The utterance ID 1501 is a unique ID mechanically assigned to the utterance. The speaker 1502 can know from the ID of the speaker included in the utterance notification. In FIG. 15, a user name corresponding to a known ID is acquired by a user name acquisition unit, which will be described later, and described instead of the ID. The utterance time 1504 and the utterance place 1505 may be described in the utterance notification as described above, or may be acquired by another means included in the terminal. The utterance duration time 1506 can be obtained from the utterance start time and end time. The audio file in which the utterance content is recorded includes the audio recorded by the microphone that operates during the continuation of the utterance. Details will be described in Examples described later.

  Here, the listener is able to acquire because the above-described reception notification is introduced. If the reception notification is not used, the receiver cannot be acquired. However, it is possible to infer the receiver by knowing the speaker while the conversation is ongoing by the notification of the utterance.

  Next, a real-world communication management technique according to the fifth embodiment of the present invention will be described. The information processing apparatus according to this embodiment has electronic signature means for applying an electronic signature to the utterance record. That is, in order to give evidence ability to the citation of an utterance, an electronic signature like a consensus sign by an utterance participant can be added.

  FIG. 14 is a diagram illustrating an example of an electronic signature added to a dialogue record exchanged between user A 1411 and user B 1415. The dialogue record 1403 records the dialogue time, dialogue location, dialogue participants (dialogues), utterance IDs included in the dialogue, and the like. The dialogue record is formed by integrating the utterance records constituting the dialogue. The dialogue start time is the utterance start time of the first utterance constituting the dialogue, and the dialogue end time is the utterance end time of the last utterance constituting the dialogue. The dialogue place is obtained by integrating the utterance places of the utterances constituting the dialogue, and is shown as a movement trajectory when the dialogue participant is talking while moving. Participants in the dialogue are utterers of the utterances that make up the dialogue, but users who just listen without uttering may be treated as participants. Need to introduce incoming call notifications).

  In addition to the information described above, a binary file in which the utterance content is recorded may be added to the dialog record. If the recorded file of the utterance content is included in the dialogue record, it is possible to specify who made the utterance with extremely high accuracy. It is desirable to add it especially when evidence capacity is required.

  Participants in the dialogue confirm that the dialogue has been exchanged with each other at that time and place, and then give an electronic signature. A simple proof can be obtained by holding the dialogue records to which the digital signatures of both the user A 1411 and the user B 1415 are added.

  For more strictness, it is desirable to give a time stamp by a third party Time Stamping Authority (TSA). The dialogue record to which the electronic signatures of the participants in the dialogue (user A 1411 and user B 1415 in the example of FIG. 14) are added is transmitted to the TSA 1417. The TSA 1417 attaches a time stamp (1421) to the hash value of the dialog record (1422), and issues a digital signature (1423), thereby issuing an electronic receipt (1424). For example, a time stamp that conforms to RFC3161 is preferably used. Since the dialogue record to which the electronic receipt has been given is returned to the user, by holding it alternately (the TSA 1417 holds the original receipt (1425)). It becomes possible to prove when the dialogue was performed when, where, where, who, what, and how.

  In the information processing apparatus, by providing utterance record output means for outputting the utterance record recorded in the communication record management means to the outside, even if there is not enough storage area inside the information processing apparatus, the output means can be used for external A large amount of information can be accumulated by outputting to. The external output may be performed through a network such as the Internet or a telephone network, or may be performed through a storage medium such as a USB memory or a card type memory using a nonvolatile memory.

  In the above, the utterance notification device that basically detects the utterance of a human speaker and notifies the utterance has been described. However, according to the fifth embodiment of the present invention, the content that reproduces the content including the utterance The playback device can make an utterance notification similar to the above-described configuration.

  For example, when a performer is talking on a content playback device such as a television or radio, the television or radio can make an utterance notification corresponding to the utterance. The content playback device may perform speaker identification necessary for performing the speech notification, but information for identifying the speaker is embedded in the content in advance, and the content playback device may perform the speech notification based on the information. desirable. The utterance notification itself configured in a predetermined format may be embedded in the content or distributed in the same manner as the content. Even if an utterance is a real person or a content playback device such as a television, the same utterance notification is transmitted, so that these records can be handled without separation.

  Furthermore, if the content playback device receives the above-mentioned reception notification corresponding to the utterance notification and forwards it to the utterer or the content creator, it is possible to determine who the utterer heard from the remote site. It is possible to know even from. The forwarding address is described in the incoming call notification, but it is desirable that this is specified in the original outgoing call notification.

  It simply forwards the incoming call notification and does nothing special. Since the reception notification includes the address of the transmission destination, it can be normally transmitted through another means (such as the Internet) without going through the content reproduction apparatus. However, it is preferable to provide a communicable content reproduction apparatus connected to a nearby network with a function of transferring an incoming call notification. Since the characteristics such as the utterance notification and the reception notification that are transmitted and received are the same as the above-described configuration, the description thereof is omitted.

  Next, real-world communication management technology according to the sixth embodiment of the present invention will be described. In the information processing apparatus according to the present embodiment, communication record management means for managing an utterance record including a speaker identifier for identifying a speaker and an utterance occurrence time, and temporal information managed by the communication record management means Utterance recording means for grouping utterance records close to the utterance record as dialogue records, and presenting a part or all of the conversation records generated by the utterance integration means to the user in time series based on the dialogue occurrence time It is characterized by that.

  In the information processing apparatus according to the present embodiment, the communication record presenting means groups the utterances that are close in time in the utterance record recorded in the communication record management means as a dialog. When considering interpersonal communication through conversation, a dialogue consists of several utterances. There are one or more topics in the dialogue, and utterances by participants in the dialogue are alternately performed on the topic. In managing the utterance record, it is desirable not to make individual utterance units, but also to make dialogue units that are semantically and temporally organized in order to avoid complications.

FIG. 19 is a section of Natsume Soseki's “Kokoro”, and FIG. 22 is a corresponding dialogue record (information not included in the original such as speech time was added as an example for explanation). . In the quoted section, there are a total of nine utterances of “I” and “Teacher”. When a dialogue is formed by a combination of multiple utterances, but when the time interval between utterances exceeds a threshold (for example, 5 seconds), the dialogue is separated and recognized.
1) Dialogue 1 (Dialogue ID: TALK01): Private utterance 1 + Teacher's utterance 1
2) Dialogue 2 (Dialogue ID: TALK02): Private speech 2 + Teacher speech 2 + Private speech 3 + Teacher speech 3
3) Dialog 3 (dialog ID: TALK03): Teacher utterance 4 + Private utterance 4 + Teacher utterance 5
The following three dialogs can be recognized.

  FIG. 23A shows an example in which the utterance record shown in FIG. 22 is expressed in dialog units. In FIG. 23A, a dialogue ID 2301, a main dialogue person 2302, a dialogue time 2303, a dialogue place 2304, a dialogue duration 2305, and a voice 2306 are included.

  The dialog duration 2305 is calculated as a time obtained by subtracting the start time of the first utterance included in the dialog specified by one dialog ID 2301 from the end time of the last utterance. Also good.

  Even if only a single utterance in FIG. 22 (for example, the utterance “No” of the teacher of ID: T002) is referred later, the meaning is not understood, but Dialog 2 including ID: T002 (ID: TALK02 FIG. 23 (A)) ) To understand the content of the story. Further, when referring to the utterance record, it is easier to understand the contents when the individual utterances are grouped and displayed as shown in FIG. 23 than when a large number of individual utterances are arranged as shown in FIG. There are various ways to divide the dialogue. In FIG. 23B, reference numerals 2311 to 2316 indicate that conversations once classified as shown in FIG. 23A are the same in conversation and continuous in time (interaction interval is less than 5 minutes, and This is an example in which a large dialogue is formed by further gathering dialogues with no other dialogue in between.

  Furthermore, by arranging the dialogue records in time series, the user can intuitively browse the dialogue records. It is also effective to divide and display by date, week, and month. In this information processing apparatus, the information processing apparatus includes a user name acquisition unit that acquires a user name corresponding to a speaker ID that identifies a speaker included in the utterance record recorded in the communication record management unit, and the communication record presentation unit includes: A user name corresponding to each utterance can also be presented.

  As described above, since the ID is not necessarily described so that a person can easily identify an individual, it is desirable to display a known ID with a name or nickname that can be understood by the user. In an environment where there is no concern about privacy (such as an office environment where all members are acquainted with each other), the name can be used as an ID if there is no fear of duplication, but when using it in a public environment, It's a big problem to send your name to others around you. Therefore, it is desirable for the ID to be a mechanical enumeration of alphanumeric characters that cannot be identified by others.

  Therefore, if they are acquainted with each other and both agree, it is necessary to teach the other person the correspondence between their ID and their name. The information processing apparatus holds an ID conversion table composed of an ID 1801 and a name 1802 as shown in FIG. 18, and a known (registered) ID is converted into a name and presented to the user. If the ID is periodically changed from the viewpoint of privacy protection, an expiration date 1803 is set for the ID. It is desirable that IDs that have expired be deleted and reacquired. However, IDs such as public officials are publicly known even if they are not acquainted with each other, and may not be deleted. It is desirable that ID information is exchanged when the electronic business card is exchanged. Alternatively, ID information of all members of a department can be collected and distributed to all members.

FIG. 7 shows an example of electronic business card data described in the vCard method that is most often used as an electronic business card format. In addition to general business card data such as name and affiliation, the fifth line describes the ID (AC2658703552) currently used and the expiration date (200550312). Here, only one ID that is currently in use is notified, but if a list of IDs that have been used in the past is also given to the other party, it may be found that the user has encountered an unexpected place in the past. . For example, the following conversation may be exchanged. “Ah, when you traveled to Australia 10 years ago, did you hear the directions?”
`` Oh, was that so, the world is narrow (lol) ''

  When reliability is required for ID information, authentication by a third party is required. Each user distributes by attaching the certificate of the certification authority to the ID that he uses. Since the certificate ensures that the ID is used by the user, it is ensured with high reliability that the utterance record having the ID is an utterance by the user.

  On the other hand, for the purpose of privacy protection, only the utterance notification with a known (registered) ID is recorded and the utterance notification with an unknown ID is not received, or the receiving side response or a known (registered) ID It is also possible to deal with the transmission side that notifies the utterance only to the terminal having. If it does in this way, only the utterance of the registered user will be recorded. Accordingly, it is possible to reduce privacy (irrelevant third-party conversation recording) in speech recording while protecting privacy.

Next, a real-world communication management technique according to the seventh embodiment of the present invention will be described. The information processing apparatus according to the present embodiment may cause the communication record presenting means to present, as a representative speaker of the dialogue, a speaker who has spoken frequently in the group among the participants of the dialogue. it can. When managing utterance records in dialog units, it is desirable to have a label characterizing the dialog for access. In a dialogue, the name of the person who spoke actively can be an index that expresses the characteristics of the dialogue. Each dialog is added with spatio-temporal information and expressed, for example, as follows.
“March 24th, 2005 12: 23-34 @ Restaurants Kamata, Shudo, Ueda (and 4 others)”

  A typical speaker can be identified by calculating the total utterance time of each utterer in the conversation and acquiring the necessary number of user names in the order of longer utterance time. If the total number of utterances (or the percentage of the utterance time of the speaker in the duration of the conversation) is less than the specified value even if the number is less than the required number, the presentation is canceled or the number of utterances is considered as a parameter, not just the utterance time It is also possible to do.

  Next, a real-world communication management technique according to the eighth embodiment of the present invention will be described. In the information processing apparatus according to the present embodiment, the utterance record managed by the communication record management unit further includes information indicating the utterance recipient, and the dialogue record generated by the utterance integration unit is the utterance recipient. It is characterized by being presented together.

  In communication records, it is often important to obtain who was listening to the utterance. If only the speaker is recorded, the user who is present in the dialogue, but does not speak, will be omitted from the recording, but this problem can also be avoided by recording the user.

  Next, real-world communication management technology according to the ninth embodiment of the present invention will be described. In the information processing apparatus according to the present embodiment, the utterance record managed by the communication record management unit further includes information indicating an utterance occurrence location, and the dialogue record generated by the utterance integration unit is an occurrence location of the dialogue. It is characterized by being presented together.

  In communication records, it is often important to obtain where the utterance (dialogue) was made. By acquiring and recording the utterance location for each utterance, it is possible to present the location of the conversation. In the case where the conversation participants are performing the conversation while moving, the places where the conversation occurs may be distributed over a wide range. In that case, a transition of a typical place can be shown as place A → place B → place C.

  Next, real-world communication management technology according to the tenth embodiment of the present invention will be described. In the information processing apparatus according to the present embodiment, the utterance record managed by the communication record management unit further includes utterance content, and the dialogue record generated by the utterance integration unit includes dialogue content composed of utterance content, and It is also characterized by being presented together. The utterance content can be recorded by recording the utterance content with a microphone, and the utterance content can be recalled later by storing it in association with the dialogue record.

  Next, real-world communication management technology according to the eleventh embodiment of the present invention will be described. In the information processing apparatus according to the present embodiment, the utterance record managed by the communication record management unit further includes estimation information of the mental state of the speaker, and the dialogue record generated by the utterance integration unit is the estimation information. It is characterized by being presented together.

  Utterance includes various human emotions. Human emotion recognition rate in normal natural utterances is said to be 60% to 70%, but even machines can recognize emotions at almost the same level by quantitatively analyzing conversation speed, voice volume, talk tone, etc. It has become. For example, the average value of the fundamental frequency of speech is high for happiness and surprise, and low for disgust. It is also known that there is a certain relationship between the standard deviation of the fundamental frequency of speech and the type of emotion. Therefore, for example, if the apparatus of FIG. 4 is a mobile phone, it is possible to quantitatively analyze the conversation speed, the volume of voice, the tone of the talk, etc., using a microphone for voice recording. This analysis result can also be recorded in association with the utterance content.

  Emotion recognition is also performed when recording the utterance content, and when the utterance record is presented to the user, it is used as a clue when the user browses the information by presenting the emotion information for each utterance to the user. it can. For example, utterances with emotions of anger are described in red, and utterances with emotions of sadness are described in blue. This provides a great clue for the user to find the necessary utterance record. Also, when viewing the utterance record at a later date, it may be found that a person had an unexpected emotion (to the listener) during the utterance. In particular, this emotion information is a feature that is difficult to use in text messages such as e-mails, and is therefore a major feature in utterance recording.

  Similarly, the recording can be simplified by automatically extracting keywords related to the utterance. Alternatively, this keyword can be used as a search key when searching at a later date.

  Next, real-world communication management technology according to the twelfth embodiment of the present invention will be described. The information processing apparatus according to the present embodiment further includes a dialog duration calculating unit that estimates the duration of grouped utterances, and the communication record presenting unit combines the duration estimated by the dialog duration calculating unit. Present.

  Information on how long each dialogue lasted is an important indicator in managing the dialogue. This is because the longer the conversation, the more likely it is to be important. A dialogue that ends in one or two words often makes little sense, such as greetings, but a dialogue that lasts for an hour, for example, is often meaningful in any way. it is conceivable that.

  The conversation duration time can be calculated from the start time of the first utterance included during the conversation to the end time of the last utterance included during the conversation. Although it is considered that each start time and end time include some errors, there is no problem because accuracy is not particularly required. It is also effective from the viewpoint of easy recognition by the user that the display colors and font sizes are presented in three levels of less than 1 minute, less than 1 minute, less than 10 minutes, and more than 10 minutes. Further, the communication record presenting means may present only grouped utterances whose duration calculated by the dialog duration calculating means is equal to or greater than a prescribed threshold. People conduct a large amount of detailed dialogue a day. When all of them are presented, the amount of information is too large to find the desired dialogue. For example, if conversations with a duration of 1 minute or less are filtered, the amount of remaining conversations is expected to be significantly reduced. It is desirable that the filtering threshold can be changed by the user.

  Further, the communication record presenting means may present the utterances grouped in ascending order or descending order according to the duration calculated by the dialog duration calculating means. For example, if the utterances can be presented in the order of long utterance duration, it is expected that the conversation records are arranged in order of relatively high importance.

  Next, real-world communication management technology according to the thirteenth embodiment of the present invention is described. In the information processing apparatus according to the present embodiment, the communication record presenting means includes user search means for searching only the utterance record including the specific utterer designated by the user from the utterance record recorded by the communication record management means. Have. Dialogue is something between people. Therefore, there may be a request to extract only a dialogue with a certain person. Therefore, by providing the user search means, it is possible to easily extract and call out a dialogue with a specific person or a dialogue between groups.

  Next, a real world communication management technique according to the fourteenth embodiment of the present invention will be described. In the information processing terminal according to the present embodiment, the communication record management unit manages the electronic mail record in addition to the utterance record, and the communication record presenting unit includes the electronic mail record managed by the communication record management unit, A part or all of the dialogue record is presented to the user in a time series based on the e-mail transmission time and the dialogue occurrence time.

  A user manages mail transmitted and received daily using mail management software, that is, a mailer. A general mailer can present e-mails in time series or search by an attribute such as a sender. If mail and utterance records managed by these mailers can be integrated and handled within the same screen, the user can separate the two types of communication, email and real-world dialogue, by separate means. No need to handle. Of course, since it may be more convenient to handle separately, it is desirable to have a mode for separating display and a mode for integrating for each means of communication.

  As for the operation, it is desirable to be able to handle both communication records without any distinction, such as creating a new e-mail as a reply to a certain utterance record and quoting the utterance record. For this purpose, it is necessary to have at least account management means for managing the e-mail account information in the mailer address book and the utterance ID of the utterance record (received record) in association with each other. For example, when replying to a speaker by e-mail, it is necessary to obtain an e-mail address corresponding to the speaker and create a mail with that e-mail address as the destination. This can be solved simply by having a table that associates speaker identifiers with email addresses.

  On the other hand, when replying to a dialogue by e-mail, the e-mail address is acquired based on the user ID (speaker ID and receiver ID) of the participant of the dialogue, and the e-mail address is set as the destination You need to create an email. Since the conversation participants usually include many people, the user may need to perform processing such as deleting unnecessary addresses. At that time, the e-mail addresses are sorted in order of utterance time, etc. If it is done, it becomes easy to make a selection (the same process as the above-described dialog representative extraction process may be performed).

  Also, when quoting or replying, it is very useful to describe the ID associated with communication records such as utterances and e-mails when processing with a computer. E-mail has a framework called Message-ID and is defined in RFC2822. When replying to a certain mail, the Message-ID of the original mail is described in the In-Reply-To header so that the mailer that receives the mail responds to the mail with the corresponding Message-ID. Knowing that, you can display the thread. Also, when quoting a part of an email, by describing the Message-ID of the original email in the References header, the mailer that received the email quotes the email with the corresponding Message-ID. To know what you are doing, you can associate two emails.

  The above processing standardized in the electronic mail can be performed by using the utterance ID for the utterance described in each embodiment of the present invention. Since the utterance ID is known to the person who heard the utterance, the utterance ID can be determined at a later date by indicating the utterance ID. In the dialogue, another utterance that was made after one utterance is often a response to the previous utterance, so information corresponding to the In-Reply-To header (the utterance ID of the original utterance) Can be included in the utterance notification, because it is possible to know the connection between utterances later when referring to the utterance record.

  Also, when calling an utterance made in the past and commenting on it, it is useful if information corresponding to the Reference header (the utterance ID of the citation utterance) can be included in the utterance notification. When the utterance is continuously performed after viewing the utterance content included in the utterance record recorded in the information processing apparatus described in each embodiment of the present invention, information corresponding to the Reference header can be included in the utterance notification. .

  Although a method of handling an utterance record in units of utterances has been described here, it goes without saying that the same processing can be performed by using dialogue IDs in the case of handling in units of dialogs.

  General e-mail includes other elements such as a transmission source address, a transmission destination address, a transmission date, a subject, and a text. These elements correspond to the respective elements of the speaker identifier, receiver ID, speech time, topic, and speech content in the utterance targeted by the present invention. Among these, elements other than the topic corresponding to the subject of the electronic mail can be acquired and managed with the above-described configuration. The topic can be estimated to some extent by performing speech recognition, keyword extraction, etc. from the utterance content, and there are many cases where there is no problem even if there is no particular problem.

  As described above, since e-mail and utterance have common elements, as long as each record retention format is shared or converted, e-mail and utterance can be combined with a unified interface. It becomes possible to handle. It can be easily understood by imagining a state in which utterance records are lined up in a mailer that is currently used, mixed in an email (described later). Each communication record can be divided and displayed on separate pages, but can also be displayed mixed on a single page.

  When listing utterances and emails by a user, it is necessary to recognize both the user's speaker identifier and email address, and the machine may recognize that their accounts are the same person desirable. For example, when searching for the communication record of "Mr. Yamada", the search will be performed using Yamada's email address yamada@dokodemo.net and speaker identifier: Y094897870, but the user will recognize each account. Even without it, it is desirable for the machine to retrieve the appropriate account and perform the search.

  The user does not need to be aware of the difference in communication, for example, it is easy to create an e-mail as a response to a certain utterance record (in this case, the utterance ID of the corresponding utterance record is included in the In-Reply-To header) Write). When replying to an utterance having a certain utterance ID by e-mail, it is desirable that an e-mail address corresponding to the utterance ID is automatically assigned. As described above, in order to integrate and use e-mail records and utterance records, a framework for converting and identifying each account with each other is necessary. This can be solved by holding a correspondence table.

Next, real-world communication management technology according to the fifteenth embodiment of the present invention will be described. In the information processing terminal according to the present embodiment, the communication record management means manages the communication record performed through the network in addition to the utterance record,
The communication record presenting means presents a part or all of the communication records and dialog records managed by the communication record managing means in a time series based on a communication start time or a dialog occurrence time, and presents them to the user. To do. That is, if other communication means such as telephone, chat, and video conference can be integrated in addition to electronic mail, various communications performed by the user can be handled in a unified manner. The information processing terminal according to the present embodiment records a user's virtual and real world communications, and enables access to any communication record with an integrated user interface. The record of human communication is expected to record much of the person's life. Even after the person dies, he can learn many parts of his life by unraveling his communication record.

  Considering the case where telephone call records are integrated, telephone call / call records (source, destination, call date / time, call time), etc. are already available. You can also record the contents of a call if you record it at the same time. Therefore, the call record can be handled by an interface integrated with the above-mentioned utterance and electronic mail. To enhance interoperability, each call must be given a unique ID, and a user account (in this case, a telephone number) can be accounted for other communication means (eg email address, utterer ID of the utterance) Need to be mapped.

As described above, according to each embodiment of the present invention, it is possible to record who the user has communicated with, so that communication in the real world is handled in the same way as communication in the virtual world. By reviewing the communication record later, it becomes possible to confirm user behavior in the real world.
Hereinafter, a more detailed and specific description will be given.

  The sent and received utterance notifications must be recorded. It is not always necessary to record all transmitted and received packets as they are, and only necessary portions may be stored in a format that is easy to manage.

  The present embodiment includes an utterance detection unit that detects an utterance of a user to be monitored, and an utterance notification unit that notifies an utterance of the user to other nearby terminals when the utterance of the user is detected by the utterance detection unit. The information notified by the utterance notification means further includes an utterance reception means for receiving an utterance notification transmitted from another terminal in the information processing apparatus characterized by including an ID for identifying a speaker. And a communication record management means for recording the utterance notice of the utterance notified by the utterance and the other utterance notice received by the utterance accepting means.

  FIG. 4A is a diagram illustrating an overview of the information processing apparatus according to the present embodiment. The information processing terminal 400 includes a wireless LAN communication module 405 and can perform ad hoc wireless communication with other nearby terminals. Further, a microphone 406 is provided to record the conversation contents. The recorded conversation content is stored in a storage device 408 stored in the device, and can be output to an external terminal via the external output connector 404 or the wireless LAN communication module 405.

  As shown in FIG. 4B, the user attaches an utterance detection pad 407 for detecting the vibration of the vocal cords to the throat. When the vocal cord vibration is detected by the utterance detection pad 407, the information processing terminal 400 generates a message called speech.msg in order to notify the utterance to the nearby terminal, and then sends the message to the neighboring terminal via the wireless LAN communication module 405. Send .msg. The speech.msg is transmitted at regular intervals while the utterance continues, and is stopped when the utterance ends. On the other hand, during the continuation of the utterance, the utterance content is acquired by the microphone 406 and is recorded in the storage device 408 in association with the information of speech.msg. As shown in the figure, the display screen 401 displays a speaker identifier (EA9206454592), a speaker name (Sacchan), an utterance duration (02'13 "), and a recorded voice level. If so, the speaker becomes himself.

  On the other hand, the information processing terminal 400 can receive speech.msg transmitted by the proximity terminal via the wireless LAN module 404. Since speech.msg includes an ID for identifying the speaker, the speaker can be known by measuring speech.msg. This terminal holds a conversion table of known IDs and names in the internal storage device, and the known IDs are converted into names and displayed to the user as shown in FIG. Furthermore, while the utterance other than the self is measured by the microphone 406 and speech.msg is received, the recorded utterance content is stored in the storage device 408 in association with the information of speech.msg.

  Since the utterance record stored in the storage device 408 can be displayed in the display screen 401, a desired utterance record is selected by operating the input device 403, and the recorded utterance content is displayed on the speaker 402. Can be heard through.

  FIG. 20A shows an example in which the utterance records recorded on April 1, 2005 (Friday) are sorted based on the utterance start time and displayed on the display screen 2001. From the left side of the screen, information is displayed in the order of utterance start time (TIME), speaker identifier or name (NAME), and utterance duration (DUR). The user can select a desired utterance record and listen to the utterance content by moving up and down the cursor line (highlighted display) by operating the input device 403 shown in FIG.

  In this example, utterances (ID: 340087B and ID: 129086C) by users who do not have a name corresponding to the speaker identifier are also displayed, but the terminal has a name corresponding to the speaker identifier. It is possible to control not to display an utterance having an unknown unknown utterance ID, or not to record from the beginning. By controlling in this way, information that is not useful can be filtered and the S / N ratio can be improved. Furthermore, it is possible to not display or delete an utterance whose duration is less than a certain time. Utterances such as greetings that are completed in a few seconds are often not important, so it is easier to find important utterances by not displaying them. Here, the sort is performed on the utterance record of one day, but since the utterance record is recorded in association with the utterance start date and time, the same applies to the utterance record in one week, one month, an arbitrary period, etc. You can make a presentation.

  A display screen 2002 shown in FIG. 20B is a diagram showing an example in which the utterance records recorded on April 1 are sorted and displayed by the utterance duration. Since important utterances often take a long time, sorting by utterance duration has the effect of making it easier to find important utterances. The display screen 2002 displays information in the order of the utterance start time (TIME), the speaker identifier or name (NAME), and the utterance duration (DUR) from the left of the screen, and is sorted in descending order of the utterance duration. . Here, the sort is performed on the utterance records of one day, but the same presentation can be performed on the utterance records in one week, one month, or an arbitrary period.

  FIG. 21C is a diagram showing a display example 2101 in which the utterance records recorded on April 1 are sorted and displayed by total utterance time for each user. Since a person who speaks for a long time is estimated to be an important person, sorting by the total utterance time has an effect of making it easier to find important utterances and important persons. Here, the sort is performed on the utterance records of one day, but the same presentation can be performed on the utterance records in one week, one month, or an arbitrary period.

  FIG. 21D is a diagram showing a display example 2102 in which an utterance of a specific user (here, “Sacchan”) is retrieved from the utterance record and displayed. Here, the data is sorted and displayed in time series, but may be sorted and displayed according to the utterance duration. Here, sorting is performed for utterance records for several days, but similar presentation can be made for utterance records in an arbitrary period of one week or January.

  FIG. 1 is a functional block diagram showing a configuration example of an information processing apparatus according to each embodiment of the present invention. As shown in FIG. 1, the information processing apparatus includes an utterance notification unit including a communication unit 101a and an utterance notification unit 102, an utterance detection unit including an utterance state detection unit 103, a conversation partner acquisition unit 104, and a communication unit 101b. Including a notification receiving means, a receiving notification means including a reception notification section 110 and a communication section 101d, a communication record management means and utterance identifier assigning means including a dialog management section 105 and a dialog recording section 108, and a communication section 101c and name management. A user name acquisition unit including a unit 106 and a name resolution unit 107, an utterance content recording unit including a dialogue acquisition unit 110, a communication record output unit including a communication unit 101, and a user interface unit 109. ing.

  The user always carries the information processing apparatus A including each unit shown in FIG. The utterance detection unit including the utterance state detection unit 103 monitors the user's utterance, and when detecting the user's utterance, notifies the utterance notification unit 102 which is an utterance notification unit of detection of the utterance. The utterance notification unit 102 generates a message (speech.msg) shown in FIG. 5 in order to notify an utterance to other nearby terminals, and transmits the message (speech.msg) to the neighboring terminals via the communication unit 101a. At the same time, the utterance state detection unit 103 transmits the detection of the utterance to the dialogue management unit 105 which is a communication record management unit. The dialogue management unit 105 records the occurrence of the utterance in the dialogue recording unit 108 in association with the time.

  On the other hand, the conversation partner acquisition unit 104 corresponding to the notification receiving unit receives speech.msg transmitted from another terminal via the communication unit 101b, and stores information on the speaker such as a speaker identifier included in the speech.msg. Tell the dialog manager 105. The dialogue management unit 105 records the occurrence of another person's utterance in the dialogue recording unit 108 together with the time and information about the utterer. The dialogue recording unit 108 is connected to the communication unit 101 as an utterance record output unit, and can output the dialogue record recorded via the communication unit 101 to an external terminal. Here, the dialogue acquisition unit 101 is an utterance content recording unit, and acquires voice by a microphone in synchronization with transmission / reception of an utterance notification. The acquired voice is associated with the utterance notification by the dialog management unit 105 and stored in the dialog recording unit 108.

  Furthermore, the correspondence information between the user ID and the name acquired via the communication unit 101 is managed by the name resolution unit, and is converted into a known user ID included in the dialogue record handled by the dialogue management unit 105 and the dialogue recording unit 108. Associate names with each other (user name acquisition means). The dialogue records handled by the dialogue management unit 105 and the dialogue recording unit 108 are presented to the user via the user interface 109.

  FIG. 5 is a diagram illustrating a data configuration example of speech.msg used as a transmission notification. speech.msg includes an utterance ID, an utterance number, a speaker identifier, and other information. The utterance ID 501 is a unique ID added to a series of utterances. In the case of an utterance that continues for a long time, a plurality of speech.msg may be transmitted while the utterance is continuing, in which case the same utterance ID is described. When the utterance is interrupted, a new utterance ID is added to the next utterance.

  The utterance No. 502 is a serial number of speech.msg in a series of utterances having the same utterance ID 501, starting from 1 and incremented. The utterance No. 502 increases when speaking continuously for a long time, but returns to 1 when the utterance is interrupted. The speaker identifier 503 is an ID for identifying a speaker. As described above, the speaker identifier 503 may be included in the speech ID 501.

Other information 504 includes the following.
1) Utterance start time: Describe the utterance start time when a series of utterances started.
2) Utterance duration: Describe the duration of a series of utterances.
3) Speech notification transmission interval: Describe the time interval for transmitting the speech notification. When the utterance notification is received and the utterance notification cannot be received at the next expected time, it is considered that the utterance has ended.
4) Size of the utterance notification area: Describes the size of the utterance notification area.
5) Speaking place: Describe the place where the utterance is made.
6) Reception notification request: Describes whether reception notification is necessary.
7) Destination notification destination address: Describes the address to which the reception notification should be transmitted.
8) Speaker ID: Describes the user ID recognized by the utterance terminal as the speaker. The user ID that has received the incoming call notification for the preceding speech.msg in a series of utterances is described.
9) In-Reply-To: Describes the utterance ID of the previous utterance when a dialogue is formed. In the case of a response to an e-mail, the Message-ID of the corresponding e-mail is described.
10) References: Describes the utterance ID to be cited. When citing an e-mail, describe the Message-ID of the corresponding e-mail.
11) Subject: Describes the topic. Corresponds to email subject. When replying to an e-mail, the subject can be automatically generated with the prefix “Re:”, but it is difficult to automatically generate a subject in a general conversation. However, the user can set the subject by himself / herself. For example, in order to measure the timing of talking to a popular person surrounded by many people, the subject is clearly stated and the person is talked to. A popular subject's terminal is set in advance with an interesting subject. When an utterance having the subject is detected, a method of notifying the user can be assumed.
11) Keyword: The keyword is the same as described above, and may be automatically extracted from speech by speech recognition.
12) Related URL: Pointer to related information. Serves as a link to utterances. If a utterance is only outlined and you want detailed information, you can follow a link set in the utterance and browse a Web page with detailed information.

  10 and 11 are diagrams showing the exchange of utterance notifications between the terminals A and D in FIG. While each user is speaking (speaking state), each terminal sends an utterance notification to terminals existing in the utterance notification area every predetermined time. Here, the notification is made by distributing speech.msg in the utterance notification area. Since terminals A and D exist in each other's utterance notification area, both terminals receive utterance notifications uttered by the other party. Here, the utterance IDs: A001 and A002 are assigned to the two utterances of the user A, and the utterance ID: D032 is assigned to the utterance of the user D. During the continuation of speech, speech.msg is periodically transmitted. Here, terminal A receives speech.msg transmitted by terminal D five times, and terminal D receives speech.msg transmitted by terminal A ten times. Each speech.msg is assigned an utterance number (number in parentheses), and each speech.msg having the same utterance ID can be distinguished.

  FIG. 6 is a flowchart showing the procedure for transmitting speech.msg. As shown in FIG. 6, processing is started in Step S1 (Start), and when an utterance is detected by the utterance detection means (Step S2), it is determined whether the utterance is a new utterance or a continuation of a series of continuous utterances. Judgment is made (step S3). For example, if the utterance interval by the same speaker is more than 3 seconds apart, it is determined as a new utterance. In the case of a new utterance (Yes), an utterance ID is generated (step S4), and 1 is newly assigned to the utterance No (step S5). If it is not a new utterance (No), the previous utterance ID is continuously used (step S8), and the utterance No plus 1 is assigned (step S9).

  If another utterance notification has been received immediately before the utterance (here, within 3 seconds) (Yes in step S6), the received utterance ID is described in In-Reply-To (step S10). In the example of FIG. 11, since the utterance ID D032 of D is received in response to the utterance ID A001 of A, ID: A001 is described in the In-Reply-To field. Similarly, In-Reply-To: D032 is described in A's utterance ID: A001.

  Furthermore, when the utterance record is viewed immediately before the utterance (here, within 3 seconds) (Yes), the utterance ID corresponding to References is described (step S11). Thereafter, speech.msg is created by adding the other information described above (step S12). speech.msg is transmitted to the terminals existing in the communication area (step S13) and waits for a specified time (200 ms in this case) (step S14). Even when the invention is not detected in step S2 (No), the process proceeds to step S14. The process returns from step S14 to step S2 and is repeated.

  Here, since the utterance notification is transmitted every 200 ms while the utterance is continuing, it can be estimated that the utterance of the corresponding user is heard for each utterance notification. If you look at the utterance No, you can know how much the utterance has continued, and when you start receiving from the utterance No in the middle, you can know that you interrupted the middle of the utterance.

  In FIG. 10, since the receiving side does not respond to the transmission of speech.msg and is a one-way notification, the speaker knows which terminal is listening to his / her utterance. I can't. However, considering that the normal dialogue is performed by mutual utterances, it is assumed that the listener of the utterance speaks before and after that time, so that the listener can be estimated with high accuracy. In the example of FIG. 10, both terminals A and D can recognize that a dialogue has been performed between terminals A-D.

  If you want to manage the listeners more strictly, such as when you want to manage people who just listen, etc., when each terminal receives an utterance notification, it returns a reception notification including the corresponding utterance ID and user ID. Can be. FIG. 11 shows message exchange when ACK, which is an incoming call notification, is returned to the utterance terminal (ACK is indicated by a dotted arrow). By introducing the reception notification, the speaker terminal can accurately manage which terminal has received the corresponding utterance.

FIG. 12 shows the exchange of speech.msg between the user A and an unrelated user F who passed nearby while interacting with another user E during the dialog of the user AD in FIG. Since the utterance notification with the user F is noise, it is desirable to filter such information as much as possible in order to improve the S / N ratio. This method is shown below.
1) First, if the user ID of user F is unknown to user A, there is a solution that ignores the utterance notification. 2) If a series of utterance numbers did not start from 1, but started in the middle, it is also effective to determine that the interruption occurred in the middle of the utterance and perform filtering. 3) Further, as shown in FIG. 12, when the utterance timings overlap for a long time, it can be determined that each of the utterances is talking to another party and filtered.

  FIG. 30 is a diagram showing a flow of transmitting a reception notification. The process is started (step S31: START), and the terminal that has received the utterance notification in step S32 records the utterance notification in step S33 and determines whether or not the reception notification is necessary in step S34. If the terminal setting allows the notification of the incoming notification and the incoming notification request included in the outgoing notification is true (Yes), an incoming notification including the corresponding outgoing speech ID and receiver ID is generated (step S35), an incoming call notification is transmitted (step S36). The transmission destination of the reception notification is normally the transmission source terminal of the utterance notification. However, when another reception notification transmission destination is specified in the utterance notification, it is followed. If no incoming call notification is required (step S34: No), the process is terminated (step S37).

  FIG. 31 is a diagram showing a flow when an incoming call notification is received. The process is started (step S41: START). When the reception notification is received in step S42, the utterance ID and the receiver ID included in the reception notification are identified, and the utterance and the receiver are specified (step S43). Subsequently, the receiver is recorded in the utterance record corresponding to the corresponding utterance ID (step S44), and the process is terminated (step S45).

  As described above, FIG. 15 is a diagram showing an example of accumulated utterance records. Here, a user A 1508, a user B 1509, and a user C 1510 are talking around the sixth table 1511 in the cafe Famille, and an unknown store clerk 1511 passes near the sixth table 1511. Here, as shown in FIG. 11, since the reception notification for the utterance notification is introduced, the information of the receiver for each utterance is also acquired. Although not described in the table of FIG. 15, various information included in the speech.msg is also managed in the same manner. As shown in this table, if each utterance is managed, the record becomes enormous. Therefore, it is easier for the user to manage the utterances together as a dialogue to some extent. FIG. 16 is a diagram showing an example in which utterances are collectively displayed as a dialogue. FIG. 16 is a table corresponding to FIG. 15, and is summarized by a dialog ID 1601, a main dialog person 1602, a dialog time 1603, a dialog place 1604, a dialog duration 1605, and a voice 1606. The dialogue at the coffee shop Famille in FIG. 15 is summarized into two dialogues between TALK0001 and TALK0002. Which of FIG. 15 and FIG. 16 is easy to understand depends on the purpose, but when looking at FIG. 16 and wanting to know the details of any of the items, it is common to refer to FIG. It is a use.

  FIG. 26 shows a functional block diagram of a content reproducing apparatus that performs utterance notification when reproducing content including an utterance record. It has a content reproduction unit having a content management unit 2605 and a content reproduction unit 2604, and the utterance detection unit 2603 and the utterance notification unit (communication unit 2601a, utterance notification unit 2602). Since it is usually difficult to detect utterances in the content, information for identifying the utterance and the utterer is described in the content in advance, or an utterance notification is generated in advance and passed to the content playback device Is desirable. The content reproduction apparatus performs appropriate utterance notification in synchronization with content reproduction.

  In the lower part of FIG. 26, an incoming call notification receiving means (including an incoming call notification receiving unit 2606 and a communication unit 2601c) for transferring an incoming call notification corresponding to the outgoing call notification to a predetermined destination and an incoming call notification transfer unit (receiving incoming call notification). Part 2607 and communication part 2601). Since they are completely independent from the upper part, they do not have to be on the same terminal, but it is desirable that they also serve as a device configuration.

  FIG. 27 is a diagram showing a processing flow of the content reproduction apparatus. The processing is started (step S21: START), and when the utterance in the content is detected during playback of the initial content (step S22) (YES in step S23), the speaker is specified (step S24), and the utterance notification is generated. Transmit (steps S25 and S26). In step S27, it is determined whether or not the content is finished, and the reproduction of the content (step S22) or the reproduction of the content is finished (step S28). In this flow, the content playback device generates the utterance notification. However, if the utterance notification is embedded in the content or distributed simultaneously, the content playback device does not need to generate the utterance notification. Will be transferred. By introducing such a content reproduction apparatus, it is possible to record and manage not only real human utterances but also utterances via machines by using the same communication recording method.

  FIG. 24 is a display example of a communication client that uses an utterance record, an e-mail record, and a telephone call record in an integrated manner. Since the utterance record is managed by the table as described in the first embodiment, it is basically the same as the mail management by the mailer. For example, there are cases where there is no subject description in telephones and conversations, but this is not a big problem. As shown in FIG. 24, the display 2401 displays the window of the communication client 2402. The left column 2403 displays an entrance to the mail browsing divided into items such as transmission, reception, trash, company / university / friends. Reference numeral 2404 displays menus such as mail, speech, telephone, TV conference, and chat. Here, “all” is displayed, but the type, subject, partner, and date are displayed. As the type, items such as telephone and dialogue are added in addition to mail (2409). In the field 2410, when an utterance is selected, for example, an utterance record shown in FIG. 15 is displayed.

  FIG. 29 is a diagram showing an overview of a PC on which a communication client operates. As shown in FIG. 29, a PC 2901 on which a communication client operates includes a PC main body 2903 having a CPU, memory, HDD, and the like, a display 2905 having a display screen 2907, an input unit 2911 such as a keyboard, and a recording medium. And a slot 1913 that can be attached and detached, and although not shown, for example, the Internet can be accessed by a wireless communication unit or the like. That is, by adding the above functions to this apparatus by hardware or software, the apparatus can be used as a communication recording apparatus described in this specification.

  FIG. 28 is a functional block diagram showing a configuration example of the apparatus shown in FIG. Communication records acquired via the communication unit 2802b, which is communication recording means, are accumulated and managed by a communication record management unit 2804, which is communication record management means. The user name acquisition means includes a communication unit 2801a, an account management unit 2802, and an account resolution unit 2803. As described above, the user can recognize various accounts included in the communication record recorded in the communication record management unit. It plays a role of integrated conversion into an easy format. The communication record management unit is connected with various communication means such as e-mail (2808), telephone (2807), and chat (2806), and these communication records are also integrated. The communication records stored in the communication record management unit 2804 are presented to the user or operated by the user via the user interface unit 2805. When an operation such as a response to an existing communication record is performed, the communication record management unit 2804 calls a necessary communication utilization unit in cooperation with the account resolution unit 2803. Various communication records performed by humans, including utterance records (dialog records), are integrated into one by the communication record management means, and an environment that can be handled without difficulty is realized.

  What is performed in the management of e-mail can be performed in almost the same manner in an environment in which utterance records and call records are integrated. In order to integrate and manage e-mail management and utterance record / call record management, it is necessary to perform account conversion processing in each communication. By preparing a conversion table, it can be read as necessary.

  Next, a fourth embodiment of the present invention will be described. FIG. 17 is a display example of a communication client that handles an utterance record and an e-mail record, schedule information, position information, a digital camera photograph, and the like. The communication client operates on the PC, and takes and manages the utterance record acquired by the information processing apparatus shown in the first embodiment via the network. The user can select when to view communication records on the calendar 1701. The communication records of the selected date are mapped in time series. The names 1706 of the main speakers included in each dialogue are mapped and displayed corresponding to the utterance start time. In FIG. 17, the user names with longer utterance durations included in the dialogue are displayed larger. Furthermore, since location information 1704 of users such as offices and canteens is also displayed in chronological order, the user can intuitively know when and where the dialogue was performed. Click on the user name to hear the corresponding dialog. Further, by displaying together schedule information 1705 such as a meeting or a welcome party, meaning can be given to time and place.

  A photograph 1702 is a photograph taken at that place for the time, and is mapped in time series by the photographing time. Based on the utterance such as “Yes, cheese”, the ID of the person shown in the photo can be acquired, which can also be used for managing the photo. For example, if the apparatus shown in FIG. 4 is a mobile phone, using a camera, a movie, and a microphone for voice recording, the photograph and speech are described in the above-described manner, for example, photo data is added to other information in the format shown in FIG. It can be managed by associating. These pieces of information are also useful records in terms of communication records. Of course, it is possible to map other media, such as a video recording, to the same time axis even if it is not a photograph.

  An e-mail 1703 is an e-mail of a reply to a certain utterance included in the dialogue represented by the user name “Koji Moji”. By expanding the time axis and looking at each utterance, it is possible to see which utterance is actually handled.

  In a rectangle 1707, a selection range selected by the user with the mouse pointer is shown. An e-mail can be created with a user included in the selection range as a destination. For example, when the minutes of a meeting, etc., are transmitted to a conference participant (actually a speaker), an e-mail can be transmitted by selecting the user who was in the conference room at that time. That is, it is possible to provide a tool for transmitting an electronic mail or the like to a participant in a certain dialog or a certain speaker. Even in this case, if a management tool as shown in FIG. This can also be regarded as an e-mail addressed to any time and space. If not only the past time and space but also the future time and space are designated by the calendar 1701 and the time axis and the e-mail is sent, the e-mail is sent to the person in the designated space when the designated time comes. Is sent. This can be used as, for example, a reminder or a mail transmission reservation tool.

  Next, a fifth embodiment of the present invention will be described. The fifth embodiment relates to dialogue categorization. FIG. 9 shows a schematic diagram of communication between users A and B by speech. In FIG. 9A, the user A is speaking (speaking) to the user B. In this case, since user A is unilaterally speaking to user B, this communication can be said to be information transmission. FIG. 9B shows a state in which there is a response from user B's utterance to user A's utterance. Here, it is presumed that a dialogue has been established, and that closer communication than the information transmission shown in FIG. FIG. 9C illustrates a case where the conversation is continuously generated, and closer communication is performed than the simple conversation illustrated in FIG. 9B. The tightness of communication can be estimated by the amount of interaction (time), and it is reasonable to assume that the more conversations are made, the closer communication is done. Furthermore, the communication when the amount of dialogue between both parties is biased can be categorized as an information transmission type (Fig. 9 (D)) and a discussion type (Fig. 9 (C)) when the amount of dialogue is balanced. is there. Such information can be used as a clue when the user browses the information by presenting the utterance record to the user according to the user.

  The present invention can be used for various information processing apparatuses related to speech.

It is a functional block diagram which shows the structural example of the information processing apparatus by each embodiment of this invention. It is a conceptual diagram at the time of using the omnidirectional short-distance radio | wireless communications system in the speech notification by this Embodiment. In this Embodiment, it is a figure which shows the concept at the time of using a directional short-distance radio | wireless communications system for speech notification. It is the figure which showed the external appearance of the information processing apparatus by a present Example. It is a figure which shows the data structural example of speech.msg utilized as a transmission notification. It is a flowchart figure which shows the transmission procedure of speech.msg. This is an example of electronic business card data described in the vCard method that is most often used as an electronic business card format. It is a figure which shows the example of the dialogue by user A, B, C, and a horizontal axis is a time axis. This is a schematic diagram of communication between users A and B by utterances. FIG. 3 is a diagram illustrating exchange of utterance notifications between terminals A and D in FIG. 2. FIG. 3 is a diagram illustrating exchange of utterance notifications between terminals A and D in FIG. 2. FIG. 2 shows the exchange of speech.msg between the user A and an unrelated user F who passed nearby while interacting with another user E during the conversation of the user A-D. It is a figure which shows the timing of continuation of a series of utterances, and utterance notification. 6 is a diagram illustrating an example of an electronic signature added to a dialogue record exchanged between a user A and a user B. FIG. It is a figure which shows an example of the table | surface which manages an utterance record. It is a figure which shows the example which displayed utterances collectively as a dialog. This is a display example of a communication client that handles an utterance record and an e-mail record, schedule information, position information, digital camera photograph, and the like in an integrated manner. It is a table possessed by the information processing apparatus, and is an ID conversion table composed of an ID 1801 and a name 1802. This is a passage from Soseki Natsume's “Kokoro”. This is an example in which recorded utterance records are sorted on the basis of the utterance start time and displayed on the display screen. This is an example in which recorded utterance records are sorted on the basis of the utterance start time and displayed on the display screen. 20 is a dialogue record corresponding to FIG. It is the example which represented the utterance record shown in FIG. 22 per dialog. It is a display example of a communication client that uses an utterance record, an e-mail record, and a telephone call record in an integrated manner. When the user is making a long-distance utterance via a communication path, the utterance notification is being transmitted via a communication path used for transmitting the utterance (FIG. 25A), via another communication path. It is a figure which each shows a mode (FIG. 25B) being transmitted. It is a functional block diagram of the content reproduction apparatus which performs utterance notification when reproducing content including utterance records. It is the figure which showed the processing flow of the content reproduction apparatus. It is a functional block diagram which shows the internal structural example of PC in which a communication client operates. It is a figure which shows the external view of PC in which a communication client operates. It is the figure which showed the flow which transmits an incoming call notification. It is the figure which showed the flow at the time of receiving an incoming call notification.

Explanation of symbols

A ... Information processing device 101a, 101b, 101c ... Communication unit 102 ... Utterance notification unit 103 ... Utterance state detection unit 104 ... Dialog partner acquisition unit 105 ... Dialog management unit 106 ... Name management unit 107 ... Name Solution unit 108. Dialog recording unit 109 109 User interface unit.

Claims (21)

A communication record management means for managing an utterance record including an utterer identifier for identifying an utterer and an utterance occurrence time;
Utterance integration means for grouping utterance records close in time managed by the communication record management means as dialog records,
An information processing apparatus comprising communication record presenting means for presenting a part or all of a conversation record generated by the utterance integration means to a user in a time series based on a conversation occurrence time. A representative extracting means for extracting a representative speaker in the dialogue based on the utterance time or the number of utterances of each utterer among the utterers of the utterance included in the dialogue record generated by the utterance integration unit;
The information processing apparatus according to claim 1, wherein the dialogue record generated by the utterance integration unit is presented together with the representative extracted by the representative extraction unit. The utterance record managed by the communication record management means further includes information indicating the utterance recipient,
The information processing apparatus according to claim 1, wherein the dialogue record generated by the utterance integration unit and the utterance receiver are presented together.   The information processing according to any one of claims 1 to 3, further comprising an ID-name conversion mechanism that converts an identifier corresponding to a speaker or a receiver of the utterance into an actual name or a nickname. apparatus. The utterance record managed by the communication record management means further includes information indicating an utterance occurrence location,
The information processing apparatus according to claim 1, wherein the dialogue record generated by the utterance integration unit is presented together with a place where the dialogue occurs. The utterance record managed by the communication record management means further includes utterance contents,
The information processing apparatus according to claim 1, wherein the dialogue record generated by the utterance integration unit is presented together with the dialogue content including the utterance content. The utterance record managed by the communication record management means is:
Further includes estimated information of the mental state of the speaker,
The information processing apparatus according to claim 1, wherein the dialogue record generated by the utterance integration unit is presented together with the estimation information. A dialog duration calculating means for calculating a duration of each dialog with respect to the dialog record generated by the utterance integration means;
The information processing apparatus according to claim 1, wherein the dialogue record is presented together with a duration calculated by the dialogue duration calculation unit. The communication record presenting means is:
9. The information processing apparatus according to claim 8, wherein only the dialogs whose dialog duration time calculated by the dialog duration calculation unit is equal to or greater than a predetermined threshold value are presented. The communication record presenting means is:
The information processing apparatus according to claim 9, wherein the dialogs are presented in ascending order or descending order according to the dialog duration calculated by the dialog duration calculation unit.   The user search means for searching only an utterance record or a dialog record including a specific utterance designated by a user from the utterance record recorded by the communication record management means. The information processing apparatus according to any one of claims. The communication record management means manages the email record in addition to the utterance record,
The communication record presenting means presents a part or all of the electronic mail record and the conversation record managed by the communication record management means in a time series based on the electronic mail transmission time and the conversation occurrence time, and presents it to the user. The information processing apparatus according to claim 1. Information for identifying a speaker included in the utterance record managed by the communication record management unit and an e-mail address included in the e-mail record managed by the communication record management unit are managed in association with each other. Have account management means,
When replying to an utterance record by e-mail, an e-mail address corresponding to the utterer of the utterance record is acquired by the account management means, and the acquired e-mail address is set as a reply mail destination The information processing apparatus according to claim 12.   When replying to a dialogue record by email, the account management means obtains email addresses corresponding to some or all of the participants in the dialogue, and uses the obtained email address as the reply email destination. The information processing apparatus according to claim 13, wherein the information processing apparatus is set.   15. The information processing apparatus according to claim 13, wherein an ID for identifying a reply source utterance record or dialogue record is described in the reply mail.   16. When citing an utterance record or dialogue record in an e-mail, an ID for identifying the utterance record or dialogue record of the citation source is described in the citation destination e-mail. The information processing apparatus according to any one of claims.   User search means for searching only an e-mail record, a utterance record or a dialogue record including a specific user designated by the user from the e-mail record and the utterance record recorded by the communication record management means, The information processing apparatus according to any one of claims 12 to 15. The communication record management means manages the communication record performed through the network in addition to the utterance record,
The communication record presenting means presents a part or all of the communication record and dialog record managed by the communication record management means in a time series based on the communication start time or dialog occurrence time, and presents it to the user. The information processing apparatus according to claim 1. An information management method using a computer,
Recording an utterance record that includes a speaker identifier identifying the speaker and the time of occurrence of the utterance;
Grouping utterance records that are close in time managed by the communication record management means as dialogue records;
And a step of presenting a part or all of the grouped dialogue records in time series to the user based on the dialogue occurrence time.   A program for causing a computer to execute the steps according to claim 19. A recording medium recording a program for causing a computer to execute the steps according to claim 19.

Images