JP2011180645A - Conversation management system and management server - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a conversation management system accurately recognizing the start and end of a conversation made by a plurality of general users, and recording only a necessary conversation. <P>SOLUTION: A wearable terminal 20 acquires partner identification information for identifying a general user being a conversation partner by a camera 40. At least either the wearable terminal 20 or an utterance content management server 10 identifies the general user being a conversation partner from partner identification information by a face recognition part 12. The wearable terminal 20 acquires by a microphone 30 at least the utterance of the general user to be used. The utterance content management server 10 records the utterance of a plurality of general users by a voice recognition result management part 13 by using a point of time when partner identification information is mutually acquired as the starting end of the conversation. When such a state that partner identification information is not mutually acquired, is continued for a prescribed time, the recording of the conversation is ended by an interview history management part 14. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a conversation management system and a management server having a plurality of wearable terminals individually used by a plurality of general users who have a conversation, and one management server that performs data communication with the plurality of wearable terminals.

  At present, when a conversation is recorded and it is desired to search and browse later, as a conventional method, there is a method of recording it by an explicit operation with an IC (Integrated Circuit) recorder or the like and writing the voice later.

  In this method, certain important conversations such as psychological resistance and annoyance to explicit operation, transfer of voice data to PC (Personal Computer), troublesome work of converting voice to character data, etc. It has only been implemented.

  On the other hand, wearable terminals are becoming widespread, and a person wearing a wearable terminal can always record his / her speech with a microphone attached to the wearable terminal.

  In addition, a technique for identifying an individual by face authentication has been developed. More specifically, the size of the face image is normalized, the contour line is detected from the face image, and includes a general feature including a closed region where the contour line is closed, a line segment where the contour line is open, and an isolated point. The unique feature is detected, and the registered feature-processed face image file is verified by pattern matching to authenticate the person.

  For this reason, it is possible to simplify the pattern matching process by collating the registered face image that has undergone the feature processing by the image processing and the face image that has undergone the feature processing from the captured face image of the visitor (Patent Document 1).

  Furthermore, as a technique as described above, there is a proposal of a communication recording system that can increase the proof value of recorded contents while allowing continuous calls and continuous recording in any free situation.

  In that technology, the headset detects the voice of the wearer of the headset and generates a first audio signal, and transmits the first audio signal to the device by short-range wireless communication compliant with the Bluetooth standard. Transmitting means.

  The device receives the first audio signal from the headset by short-range wireless communication compliant with the Bluetooth standard, and generates the second audio signal by detecting the audio of a person other than the headset wearer Information for generating information to be stored by associating time information with the first and second audio signals, time information acquiring means for acquiring the generation times of the first and second audio signals, A creation means and a storage means for storing the created information are provided (Patent Document 2).

JP 2005-242432 A Japanese Patent Laying-Open No. 2005-237017

  In the case of a conversation in which both speakers wear a wearable terminal, there is a need to efficiently record the contents of conversations with people and to search and view them later. However, in the conventional method, only the user's remarks are recorded on the wearable terminal, and therefore, the remarks of the other party cannot be searched and viewed.

  The technique of Patent Document 2 can solve the above-described problems. However, it is difficult to accurately recognize the start and end of a conversation by a plurality of general users and record only the necessary conversation.

  The present invention has been made in view of the above problems, and a conversation management system and management server capable of accurately recognizing the start and end of conversations by a plurality of general users and recording only necessary conversations. Is to provide.

  The conversation management system of the present invention includes a plurality of wearable terminals individually used by a plurality of general users who have a conversation, and a single management server that performs data communication with the plurality of wearable terminals. And a wearable terminal and a management server, at least one of the wearable terminal and the management server has user identification means for identifying a general user of a conversation partner from the partner identification information, and wearable The terminal has a statement acquisition unit that acquires at least a statement of a general user to be used, and the management server starts recording of conversations of a plurality of general users when the partner identification information is mutually acquired, Recording end means for ending the recording of conversation when a state in which the partner identification information is not mutually acquired has elapsed until a predetermined time.

  The management server of the present invention is a management server of the conversation management system of the present invention, and recording start means for starting recording of conversations of a plurality of general users when the partner identification information is mutually acquired, and the partner identification information is Recording ending means for ending the recording of the conversation when the mutually unacquired states elapse until a predetermined time.

  It should be noted that the various components of the present invention need only be formed so as to realize their functions. For example, dedicated hardware that exhibits a predetermined function, contents of statements in which a predetermined function is given by a computer program It can be realized as a management server, a predetermined function realized in the statement content management server by a computer program, an arbitrary combination thereof, or the like.

  The various components of the present invention do not necessarily have to be independent of each other. A plurality of components are formed as a single member, and a single component is formed of a plurality of members. It may be that a certain component is a part of another component, a part of a certain component overlaps with a part of another component, or the like.

  In the conversation management system of the present invention, the wearable terminal acquires partner identification information for identifying the general user of the conversation partner by the identification acquisition means. At least one of the wearable terminal and the management server identifies the general user of the conversation partner with the user identification means from the partner identification information. The wearable terminal acquires at least a general user's speech to be used by the speech acquisition means. When the management server obtains the partner identification information from each other, the recording server records the conversations of a plurality of general users. When the state in which the other party identification information is not mutually acquired elapses until a predetermined time, the recording of the conversation is ended by the recording end unit. For this reason, it is possible to accurately recognize the start and end of a conversation by a plurality of general users and to record only the necessary conversation.

It is a typical block diagram which shows the logical structure of the data processing system of embodiment of this invention. It is a typical time chart which shows the recording process of conversation. It is a schematic diagram which shows the data structure of a certain conversation log. It is a typical time chart which shows the recording process of conversation of two general users. It is a schematic diagram which shows the data structure of ID of a general user. It is a flowchart which shows the process of recognizing the start of conversation. It is a schematic diagram which shows the data structure of the several conversation recorded. It is a schematic diagram which shows data structures, such as the utterance content in one conversation. It is a flowchart which shows the process of recognizing completion | finish of conversation. It is a schematic diagram which shows the state by which the conversation history display menu was displayed on the spectacles type display. It is a schematic diagram which shows the state by which the search result was displayed on the spectacles type display. It is a typical time chart which shows the recording process of a conversation of three general users.

  An embodiment of the present invention will be described below with reference to the drawings. Referring to FIG. 1, a conversation management system of the present invention includes a plurality of wearable terminals 20 that are individually used by a plurality of general users who have a conversation, and a message content management server 10 that performs data communication with the plurality of wearable terminals 20. Have.

  The wearable terminal 20 includes a camera 40 that is identification acquisition means for acquiring partner identification information for identifying a general user of a conversation partner. The statement content management server 10 includes a face recognition unit 12 that is a user identification unit that identifies a general user of a conversation partner from the partner identification information.

  The wearable terminal 20 includes a microphone 30 that is a speech acquisition unit that acquires at least a general user's speech to be used. The speech content management server 10 is a state where voice recognition result management unit 13 which is a recording start means for starting recording of conversations of a plurality of general users when the partner identification information is mutually acquired, and the partner identification information is not acquired mutually. Has a meeting history management section 14 which is a recording end means for ending the recording of conversation when a predetermined time elapses.

  The meeting history management unit 14 stores the conversation in the voice recognition result management unit 13 with the end when the other party identification information is not acquired after the predetermined time has passed. However, since the wearable terminal 20 has the speech recognition unit 21 that recognizes the acquired speech as text data and converts it into text data, the speech recognition result management unit 13 records the conversation as text data.

  The message content management server 10 causes the wearable terminal 20 to browse a conversation recorded by a general user identified as a conversation partner. The microphone 30 of the wearable terminal 20 acquires a search key that is spoken by a general user to use, and the message content management server 10 searches and browses the conversation using the acquired search key.

  The wearable terminal 20 acquires a face image of a general user who is a conversation partner as partner identification information. Wearable terminal 20 can also acquire partner identification information from wearable terminal 20 of the general user of the conversation partner.

  Further, as will be described in detail later, the voice recognition result management unit 13 records the conversation even when the partner identification information of three or more general users is mutually acquired, and the meeting history management unit 14 records the conversation. The recording of the conversation is terminated when a predetermined time elapses when the partner identification information of all three or more persons who have been acquired is not mutually acquired.

  More specifically, the message content management server 10 and the wearable terminal 20 operate under program control and are connected to each other via a data network 100 such as the Internet.

  The message content management server 10 is an information processing device such as a workstation server installed on the Internet, and includes an ID management unit 11, a face recognition unit 12, a voice recognition result management unit 13, a visit history management unit 14, and a search. Units 15 and so on are logically included.

  The wearable terminal 20 is an information processing apparatus such as a small personal computer, and includes a voice recognition unit 21, a microphone 30, a camera 40, a glasses-type display 50, operation keys 60, and the like.

  The wearable terminal 20 has a function of transmitting information captured by the camera 40 to the message content management server 10 via the data network 100. Further, it has a function of receiving data transmitted by the message content management server 10 and displaying it on the screen on the glasses-type display 50.

  The voice recognized by the microphone 30 is converted into character information by the voice recognition unit 21 and transmitted to the message content management server 10. Here, the speech recognition unit 21 may be provided on the statement content management server 10 depending on the configuration, and the wearable terminal 20 may transmit the voice data to the statement content management server 10 as it is.

  Next, the operation of this embodiment will be described in detail with reference to FIGS. As shown in FIG. 2, when Mr. A and Mr. B have a conversation until 10: 00-10: 02, the details will be described with reference to FIG. 3 and FIG.

  First, Mr. A's wearable terminal 20 transmits Mr. B's video to the message content management server 10 via the camera 40 (step A1). Here, the wearable terminal 20 operates so as to always transmit video to the message content management server 10 at regular intervals.

  Next, the message content management server 10 matches the received video with the ID information stored in the ID management unit by face recognition or ID recognition of the wearable terminal 20 by the face recognition unit 12, and determines Mr. B (step A2).

  Here, each ID is assigned to the wearable terminal and is managed by the ID management unit 11 in a state where an individual can be specified as shown in FIG. The ID recognition of the wearable terminal 20 can be expressed by, for example, a two-dimensional code on the front surface of the glasses-type display 50 (not shown).

  Next, the meeting history management unit 14 sets the recognized time to the start time and the end time of the conversation as shown in FIG. 7 according to the determination logic as shown in FIG. Is set to the speaker 2 and the state is awaiting recognition (step A3).

  Similarly, when Mr. B's wearable terminal 20 transmits the video of Mr. A to the message content management server 10 through the camera 40, the meeting history management unit 14 indicates that Mr. A and Mr. B are waiting for recognition. It is determined that the bidirectional recognition of Mr. A and Mr. B has been completed and the conversation has started, the recognized time is set as the conversation end time, and the state is defined as being in conversation.

  Next, Mr. A's wearable terminal 20 converts the voice input through the microphone 30 into text by the voice recognition unit 21 (step A4), and transmits the text to the message content management server 10 (step A5).

  Next, the statement content management server 10 stores the received text in the speech recognition result management unit 13 as shown in FIG. 8 (step A6). In this way, the conversation is continued, but when the face-to-face history management unit 14 is referred to by the determination logic as shown in FIG. 9 and face recognition is not performed for a certain period of time, and the conversation end time is not updated, It is determined that the conversation has ended, and the state is ended.

  Next, a conversation history display menu is displayed on the glasses-type display 50 as shown in FIG. At this time, the wearable terminal 20 requests the speech content management server 10 to search the conversation history (step A7).

  Next, the search unit 15 in the statement content management server 10 extracts the conversation time and conversation partner from the visit history management unit 14, and extracts the comment at that time from the voice recognition result management unit 13 (step A8), and transmit to wearable terminal 20 (step A9).

  Next, wearable terminal 20 displays the conversation history on glasses-type display 50 (step A10). Here, as shown in FIG. 11, it is also possible to adopt a mechanism in which a keyword is input by voice input or the like and a utterance or other information is searched.

  In the conversation management system of the present embodiment, as described above, wearable terminal 20 acquires partner identification information for identifying a conversational partner general user with camera 40. The message content management server 10 identifies the general user of the conversation partner by the face recognition unit 12 from the partner identification information.

  The wearable terminal 20 acquires at least a general user's remarks to be used by the microphone 30. The speech content management server 10 records the utterances of a plurality of general users in the speech recognition result management unit 13 at the beginning of the conversation when the partner identification information is mutually acquired, and a state in which the partner identification information is not mutually acquired is predetermined. When the time has elapsed, the conversation history management unit 14 ends the conversation recording.

  For this reason, in the conversation management system of the present embodiment, it is possible to accurately recognize the start and end of conversations by a plurality of general users and to record only necessary conversations. Furthermore, the voice recognition result text recorded on the conversation partner's terminal can be displayed on the own terminal.

  The reason is that ID information and time specified by face recognition or ID recognition are used to determine the conversation time, and only the conversation during the conversation can be shared with the conversation partner. is there.

  Moreover, even if it is a conversation of two or more persons, all who participated in the conversation can refer to each other's remarks. The reason is that when a plurality of one-to-one conversation groups are established at the same time, it is considered that a plurality of persons have conversations.

  With reference to FIG. 12, the example in the case of having a conversation with three people is given. When the conversation between Mr. A and Mr. B is established, the conversation between Mr. A and Mr. C is established, and the conversation between Mr. B and Mr. C is established, it is determined that the conversation between the three persons is established. Furthermore, the end logic of FIG. 9 determines that the conversation has ended when a certain time has passed without any of the three persons recognizing their faces.

  The present invention is not limited to the present embodiment, and various modifications are allowed without departing from the scope of the present invention. For example, in the above embodiment, the face recognition unit 12 as user identification means for identifying a general user of a conversation partner from the partner identification information is exemplified in the statement content management server 10. However, such a face recognition unit 12 may be in the wearable terminal 20.

  In addition, the speech recognition unit 21 that recognizes the acquired speech as described above and converts it into text data is exemplified in the wearable terminal 20. However, such a speech recognition unit 21 may be provided in the statement content management server 10.

  Furthermore, in the present embodiment, it has been exemplified that each part of the statement content management server 10 and the wearable terminal 20 is logically realized as various functions by a computer program. However, each of these units can be formed as unique hardware, or can be realized as a combination of software and hardware.

  In the above embodiment, the current Internet is exemplified as the data network 100. However, this may be a next generation network (NGN) which is the next generation Internet.

  Needless to say, the above-described embodiment and a plurality of modifications can be combined within a range in which the contents do not conflict with each other. Further, in the above-described embodiments and modifications, the structure of each part has been specifically described, but the structure and the like can be changed in various ways within a range that satisfies the present invention.

DESCRIPTION OF SYMBOLS 10 Statement content management server 11 ID management part 12 Face recognition part 13 Voice recognition result management part 14 Visit history management part 15 Search part 20 Wearable terminal 21 Voice recognition part 30 Microphone 40 Camera 50 Glasses type display 60 Operation key 100 Data network

Claims (10)

A plurality of wearable terminals individually used by a plurality of general users having a conversation, and a single management server in data communication with the plurality of wearable terminals,
The wearable terminal has identification acquisition means for acquiring partner identification information for identifying the general user of the conversation partner,
At least one of the wearable terminal and the management server has user identification means for identifying the general user of the conversation partner from the partner identification information,
The wearable terminal has a speech acquisition unit that acquires at least the speech of the general user to be used,
When the management server obtains the partner identification information from each other, a recording start unit that starts recording the conversations of the plurality of general users, and a state in which the partner identification information is not mutually acquired until a predetermined time elapses. A conversation ending system for ending the recording of the conversation. The recording start unit records the conversation even when the partner identification information of three or more general users is acquired mutually,
2. The conversation management according to claim 1, wherein the recording ending unit ends the recording of the conversation when a state in which the partner identification information of all of the three or more persons in which the conversation is recorded is not mutually acquired has elapsed for a predetermined time. system.   The statement content management server according to claim 1, further comprising: a conversation storage unit that stores the conversation when the partner identification information is not acquired mutually after the predetermined time has elapsed.   The conversation management according to any one of claims 1 to 3, wherein the management server further includes browsing permission means for allowing the general user identified as the conversation partner to browse the conversation recorded on the wearable terminal. system. The speech acquisition means acquires a search key that the general user to use speaks,
The conversation management system according to claim 4, wherein the browsing permission unit searches and browses the conversation with the acquired search key. At least one of the wearable terminal and the management server further includes voice recognition means that recognizes the acquired speech as text data,
The conversation management system according to any one of claims 1 to 5, wherein the recording start unit records the conversation as the text data.   The conversation management system according to any one of claims 1 to 6, wherein the wearable terminal further includes image acquisition means for acquiring a face image of the general user of the conversation partner as the partner identification information.   The conversation management system according to claim 1, wherein the wearable terminal acquires the partner identification information from the wearable terminal of the general user who is a conversation partner. The management server of the conversation management system according to any one of claims 1 to 8,
Recording start means for starting recording of the conversations of a plurality of general users when the partner identification information is mutually acquired;
A recording ending unit for ending the recording of the conversation when a state in which the partner identification information is not mutually acquired elapses until a predetermined time;
Management server having   The management server according to claim 9, further comprising user identification means for identifying the general user of the conversation partner from the partner identification information for identifying the general user of the conversation partner acquired by the wearable terminal.

Images