JP2016006607A - Interactive communication system, terminal device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for performing an interactive call via a network, in which the interactive call performed via the network is more smoothly performed.SOLUTION: A conversation state monitoring unit 102 monitors a conversation state in interactive communication by a plurality of terminal devices 101 connected via a network; an extraction/recording unit 103 extracts words 104 that appear in conversation in the interactive communication. A position information analyzing/recording unit 107 obtains images that appear in the conversation or position data of the terminal devices 101, analyzes them, and records them as position information 108. A topic providing unit 106 extracts, when the conversation state is a stagnation state, a word 105 matching a first condition from the words 104 recorded by the extraction/recording unit 103; extracts position information 109 matching a second condition from the position information 108 recorded by the position information analyzing/recording unit 107; retrieves a topic 110 matching or relating to the word 105 and position information 108 by accessing a server device 111 of a search site, for example; and provides it to the terminal devices 101.

Description

  The present invention relates to a technique for making an interactive call via a network.

  In recent years, communication tools using the Internet such as IP (Internet Protocol) telephone, video chat, and SKYPE (“SKYPE” is a registered trademark of “Skype”) have been developed, and these are widely used as alternatives to conventional telephones. Has been. In particular, peer-to-peer (P2P) network systems are difficult to concentrate on specific nodes, and the processing is distributed and stable even when the number of clients and the data capacity increase, making them suitable for video chats that exchange real-time moving images. ing.

  On the other hand, it is said that communication is effective for preventing dementia, and the communication tools as described above are expected to be useful for preventing dementia.

  For example, a system that can detect a smiling face of a dementia patient and provide an appropriate conversation according to the detected smile has been proposed (for example, a technique described in Patent Document 1). This system detects the smiles of patients with dementia during a conversation, adjusts the conversation time according to the appearance frequency of the smiles, or changes the content of the conversations. Concentrated conversation can be provided to patients with dementia.

In addition, a conversation service technology that heals the minds of elderly people residing in nursing care facilities has been proposed (see, for example, Patent Document 2). In this technology, the day care center terminal transmits the scheduled date and time of the event to the conversation service server. The conversation service server stores the scheduled date and time data in the daycare database. The care facility terminal accesses the conversation service server, selects an event desired to be received, and transmits an image reception reservation application for the selected event to the conversation service server. The conversation service server stores the image reception reservation application in the reservation list in association with the scheduled date and time data. When the scheduled date and time is reached, the conversation service server causes video chat between the daycare terminal and the nursing facility terminal.
Furthermore, there is a topic information providing system that provides a topic suitable for the preference of the caller during a call (for example, a technique described in Patent Document 3).

JP 2011-223369 A JP 2005-196640 A JP 2003-115951 A

However, the conventional conversation system for dementia patients described above has a problem in that the development of conversation is lacking because the server only transmits a conversation data group unilaterally.
In addition, the above-described conventional conversation service system has a problem in that it does not sufficiently develop conversational conversations, but only allows the elderly and children to interact.
In addition, from the viewpoint of prevention of dementia among the elderly, it is preferable not to provide unilateral conversation, but it is preferable to talk to the elderly, families such as children and grandchildren, and staff of care facilities, etc. There was also a problem that the topic did not continue in the first place even when trying to make a conversation using.
Further, the technique described in Patent Document 3 merely provides a topic regardless of the situation of the caller.

  Accordingly, an object of the present invention is to more smoothly advance an interactive call performed via a network.

As an example of an aspect of the present invention, the invention described in claim 1 includes a conversation state monitoring unit that monitors an active / stagnation state of conversation in interactive communication by a plurality of terminal devices connected via a network;
An extraction / recording unit that extracts and records predetermined information in the interactive communication, and a topic providing unit that supplies topics related to the recorded predetermined information to the terminal device when the conversation state is stagnant And providing an interactive communication system.
An eleventh aspect of the present invention provides a terminal device used in the interactive communication system according to any one of the first to ninth aspects.

  According to a twelfth aspect of the present invention, there is provided a conversational activity / stagnation state monitoring process for monitoring a conversational activity / stagnation state in interactive communication with a computer of a terminal device connected to another terminal device via a network; An extraction / recording process for extracting / recording predetermined information in interactive communication, and when the conversation state is stagnant, a topic related to the extracted predetermined information is accessed to a server device of a search site. A program for executing a topic providing process for searching and supplying the searched topic to the terminal device is provided.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to advance the interactive telephone call performed via a network more smoothly.

It is a functional block diagram of embodiment of an interactive communication system. 1 is a configuration diagram of a network system on which an embodiment of an interactive communication system is based. It is a block diagram of embodiment of a terminal device. It is a block diagram of embodiment of a server apparatus. It is a figure which shows the hardware structural example of the computer which can implement | achieve a terminal device or a server apparatus. It is a flowchart which shows the example of an interactive communication process. It is operation | movement explanatory drawing of a conversation state monitoring process. It is a flowchart which shows the detailed example of an active state process. It is a flowchart which shows the detailed example of a stagnation state process. It is a figure which shows the example of a display of the interactive communication (the 1). It is a figure which shows the example of a display of the interactive communication (the 2). It is a block diagram of other embodiments of a terminal unit. It is a block diagram of other embodiments of a server apparatus.

  Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. FIG. 1 is a functional configuration diagram of an embodiment of an interactive communication system.

  The embodiment of the present invention is an interactive communication system in which a plurality of terminal devices 101 connected via a network such as the Internet or a local area network perform interactive communication by, for example, peer-to-peer connection.

  In the embodiment of the interactive communication system, the conversation state monitoring unit 102 monitors the conversation state in interactive communication by a plurality of terminal devices 101 connected via a network.

  The phrase extraction / recording unit 103 extracts a phrase 104 that appears in a conversation in interactive communication.

  The position information analysis / recording unit 107 acquires an image that appears in a conversation or position data of the terminal device 101, analyzes it, and records it as position information 108.

  The topic providing unit 106 extracts the phrase 105 that matches the first condition from the phrase 104 recorded by the phrase extraction / recording unit 103 when the conversation state monitoring unit 102 determines that the conversation state is stagnant. Then, the position information 109 meeting the second condition is extracted from the position information 108 recorded by the position information analysis / recording unit 107. Then, the topic providing unit 106 supplies the terminal device 101 with a topic 110 that matches or is related to both of the extracted words 105 and the position information 109 (which may be either). Specifically, the topic providing unit 106 accesses the search site server device 111 to search for a topic related to the extracted word 105 and position information 109, and searches the terminal device 101 for the searched topic. Supply. The topic that matches the extracted word / phrase 105 and the position information 109 is a search for news or articles including the word / phrase and position information from a search site, and the related topic is the same as the word / phrase and position information. Even if you don't do it, you can search by tag or genre. For example, a search for news “J-League match results” from the phrase “World Cup soccer” can be given.

  Due to the functional configuration of the embodiment of the interactive communication system described above, the topic 110 that matches the phrase 104 or the position information 108 that has previously appeared in the interactive communication conversation when the conversation state in the interactive communication becomes stagnant. Is retrieved from the server device 111 at the search site and displayed on the display screen of the terminal device 101. As a result, a user who is performing interactive communication, for example, sees the display and the like, and recalls, for example, the recent topic 110 or the topic 110 that has been collected more than six months ago, for example. It is possible to proceed to.

  A more specific embodiment of the interactive communication system will be described below. FIG. 2 is a configuration diagram of a network system on which the embodiment of the interactive communication system is based. The embodiment of the interactive communication system 200 is a network in which a plurality of terminal devices 220 (corresponding to the terminal device 101 in FIG. 1) capable of performing interactive communication with each other, for example, by peer-to-peer connection are the Internet or a local area network. Via a plurality of relay devices 210 including router devices, server devices, and the like forming 250, communication is established. In addition, the server apparatus 100 that provides a topic to each terminal apparatus 220 in the present embodiment is connected to the relay apparatus 210 of the network 250. Further, for example, a search site server device 230 and a news site server device 240 accessible from the Internet are connected to the relay device 210 of the network 250.

  Under the configuration of the interactive communication system 200 as described above, for example, three terminals of Mr. A's terminal device 220A, Mr. B's terminal device 220B, and Mr. C's terminal device 220C execute interactive communication with each other. be able to.

  FIG. 3 is a block diagram of an embodiment of the terminal device 220 of FIG. The terminal device 220 is, for example, a general personal computer that can be connected to the network 250, a tablet terminal device, or a smartphone terminal device.

  The interactive communication control unit 301 issues an authentication request to the server device 100 and receives permission for interactive communication from the server device 100, and then establishes interactive communication by peer-to-peer connection with another terminal, Controls the execution of interactive communication using text data, audio data, image data, and other file data.

  The voice input / output unit 302 is a unit that inputs voice uttered by the user through, for example, a microphone, and emits voice from another terminal to the user through, for example, a speaker or an earphone.

  The imaging unit 303 is, for example, a digital camera unit that captures an image of the user.

  The display unit / operation input unit 304 is a touch panel display unit, or a display unit and a keyboard unit that displays text and images by interactive communication or a topic provided by the server device 100 to the user.

  A GPS (Global Positioning System) receiving unit 305 receives signals from several GPS satellites in the sky and calculates latitude / longitude data where the terminal device 220 itself is located.

  The data storage unit 306 stores image data and other file data exchanged through interactive communication.

  The voice analysis unit 307 inputs voice data of conversation in interactive communication input / output to / from the voice input / output unit 302 via the interactive communication control unit 301, acoustically analyzes the voice data, and outputs voice information. To do. The acoustic analysis is, for example, a frequency analysis process for extracting a human formant component included in voice data and a process for calculating the average power of the voice data. The voice analysis unit 307 outputs, as frequency information, feature vector data obtained by frequency analysis of voice every 10 to several tens of milliseconds as voice information. More specifically, the feature vector data includes, for example, a cepstrum coefficient obtained by cepstrum analysis of speech, a PARCOR count obtained by parsing speech with PARCOR (Partial Auto Correlation Coefficients), or LSP (Line Spectral Pairs) analysis of speech. 10 to 13 real coefficient sets composed of LSP coefficients obtained as described above. In addition, the voice analysis unit 307 calculates the average power of the voice data at regular time intervals and outputs this value as voice information.

  The position data acquisition unit 308 appears in a conversation of interactive communication, and is included in, for example, still image data or moving image data metadata input / output to / from the data storage unit 306, for example, Exif (EXchangeable Image File format) data. The latitude / longitude data indicating the position where the image was taken is acquired as position data via the interactive communication control unit 301. Alternatively, the position data acquisition unit 308 acquires latitude / longitude data detected by the GPS receiver 103 as position data via the interactive communication control unit 301.

  The communication interface unit 309 provides a connection function between the network 250 of FIG. The communication interface unit 309 transmits text data, voice data, image data, and other file data for interactive communication that the interactive communication control unit 301 inputs and outputs to other terminals via the network 250 in FIG. The terminal device 220 transmits and receives. Further, the communication interface unit 309 transmits the feature vector data output by the voice analysis unit 307 every 10 to several tens of milliseconds to the server apparatus 100 via the network 250 in FIG. Furthermore, the communication interface unit 309 transmits the position data output from the position data acquisition unit 308 as needed to the server apparatus 100 via the network 250 in FIG.

  FIG. 4 is a block diagram of an embodiment of the server device 100 of FIG.

  The communication interface unit 401 provides a connection function between the network 250 of FIG. The communication interface unit 401 receives the authentication request, voice information, and position data transmitted by the terminal device 220 in FIG. 2 and passes them to the authentication unit 402, the word extraction unit 405, and the position information analysis unit 407, respectively.

  When the authentication unit 402 receives the authentication request from the terminal device 220 in FIG. 2, the authentication unit 402 collates the user name and password included in the authentication request with the user name and password stored in the user database (user DB) 403. If the collation result is OK, interactive communication is permitted to the terminal device 220.

  The conversation state monitoring unit 404 monitors the conversation state in the interactive communication by each terminal device 220. More specifically, the conversation state monitoring unit 404 inputs voice average power information of the voice information input from the communication interface unit 401 to the word extraction unit 405 from the word extraction unit 405, and the value of the voice average power information. Based on the above, it is monitored whether the conversation of the interactive communication in the terminal device 220 that has transmitted the value is in an active state or a stagnation state.

  For example, when the conversation state monitoring unit 404 detects an active state, the word extraction unit 405 uses the voice information input from the terminal device 220 from which the state has been detected via the communication interface unit 401 to perform a dialog in the terminal device 220. Extract words that appear in type communication conversations. Then, the word extraction unit 405 records the extracted word, identification information of interactive communication from which the word is extracted, and date / time information from which the word is extracted, in the word database (word DB) 406. If the same word has already been registered in the word DB 406, the word extraction unit 405 overwrites and records the interactive communication identification information and date / time information of the word with new information.

  For example, when the conversation state monitoring unit 404 detects an active state, the position information analysis unit 407 corresponds to the position data from the position data input via the communication interface unit 401 from the terminal device 220 in which the state is detected. For example, a position name on the map is analyzed as position information. Then, the position information analysis unit 407 stores, in the position information database (position information DB) 408, the extracted position information, identification information of interactive communication from which the position information is extracted, and date / time information from which the position information is extracted. Record. When the same position information is already registered in the position information DB 408, the position information analysis unit 407 overwrites and records the interactive communication identification information and date / time information of the position information with new information. .

  When the conversation state monitoring unit 404 detects a stagnation state, the topic providing unit 409 extracts, for example, the words 410 registered within the last few months from the word DB 405, for example, and also from the position information DB 408, for example, For example, the position information 411 registered within several months is extracted at random. Then, the topic providing unit 409 searches for the topic 412 that matches both the extracted word 410 and the position information 411 (which may be either) via the communication interface unit 401 and the network 250 of FIG. The site server device 230 and the news site server device 240 are accessed and searched. The topic providing unit 409 transmits the searched topic 412 to the terminal device 101 performing interactive communication in which the conversation state monitoring unit 404 has detected a stagnation state.

  In FIG. 3, when the interactive communication control unit 301 in the corresponding terminal device 220 receives the above-described topic 412 via the communication interface unit 309, the interactive communication control unit 301 displays text data or image data included in the topic 412 on the display unit / It is displayed on the interactive communication screen of the operation input unit 304.

In the configuration described above, the conversation state monitoring unit 404 in FIG. 4 corresponds to the conversation state monitoring unit 102 in FIG. Also, the voice analysis unit 307 in FIG. 3 and the word extraction unit 405 in FIG. 4 correspond to the phrase extraction / recording unit 103 in FIG. Also, the word DB 405 in FIG. 4 corresponds to the recorded phrase 104 in FIG. The position data acquisition unit 308 in FIG. 3 and the position information analysis unit 407 in FIG. 4 correspond to the position information analysis / recording unit 107 in FIG. Further, the position information DB 408 in FIG. 4 corresponds to the recorded position information 108 in FIG. Further, the topic providing unit 409 in FIG. 4 corresponds to the topic providing unit 106 in FIG. The search site server apparatus 230 in FIG. 2 corresponds to the search site server apparatus 111 in FIG. 2, 3, and 4, the voice analysis unit 307 and the position data acquisition unit 308 are added to the terminal device 220, the conversation state monitoring unit 404, the word extraction unit 405, the word DB 405, the position The information analysis unit 407, the position information DB 408, and the topic providing unit 106 are distributed and arranged on the server device 100, respectively. In this case, the data transmission amount of the voice information and the position data transmitted from the voice analysis unit 307 and the position data acquisition unit 308 in the terminal device 220 to the server device 100 is far more than that of transmitting the voice data or the image data itself. Less transmission is required.
Thus, for example, a user who is performing interactive communication with the terminal device 220 in the same manner as in the functional configuration of FIG. By looking at the display of the topic 412 displayed in the section 304 (FIG. 3), for example, the recent topic 412 or the topic 412 that has been recorded more than six months ago, for example, can be recalled again, and the interactive call can be promoted more smoothly. It becomes possible. In particular, elderly people may remember more old topics than the most recent ones.

  FIG. 5 is a diagram illustrating a hardware configuration example of a computer capable of realizing the terminal device 220 in FIG. 3 or the server device 100 in FIG.

  When the computer shown in FIG. 5 is implemented as the server apparatus 100 of FIG. 4, the CPU (central processing unit) 501, the memory 502, the input apparatus 503, the output apparatus 504, the external storage apparatus 505, A portable recording medium driving device 506 into which a portable recording medium 509 is inserted is provided, and a communication interface circuit 507 is connected to each other by a bus 508. Further, when the computer shown in FIG. 5 is implemented as the terminal device 220 of FIG. 3, in addition to the same configuration as that of the server device 100 described above, the camera unit 510 and the GPS reception on the bus 508 are added. The machine 511 is further connected. The configuration shown in FIG. 5 is an example of a computer that can implement the terminal device 220 or the server device 100 of the interactive communication system 200, and such a computer is not limited to this configuration.

  The communication interface unit 309 in FIG. 3 or the communication interface unit 401 in FIG. 4 corresponds to the communication interface circuit 507 in FIG. The display unit / operation input unit 304 and the voice input / output unit 302 in FIG. 3 correspond to the input device 503 and the output device 504 in FIG. 5. The imaging unit 303 and the GPS receiving unit 305 in FIG. 3 correspond to the camera 510 and the GPS receiver 511 in FIG. The functions of the data storage unit 306 in FIG. 3, the word DB 405 in FIG. 4, the location information DB 408 in FIG. 4, and the user DB 403 in FIG.

  The portable recording medium driving device 506 accommodates a portable recording medium 509 such as an SD card, a compact flash, or an optical disk, and has an auxiliary role for the external storage device 505.

  The CPU 501 controls the entire computer. The memory 502 is a memory such as a RAM that temporarily stores a program or data stored in the external storage device 505 (or the portable recording medium 509) when executing a program, updating data, or the like. The CUP 501 performs overall control by reading a program for interactive communication processing described later into the memory 502 and executing it. Thereby, in the terminal device 220, the functions of the interactive communication control unit 301, the voice analysis unit 307, and the position data acquisition unit 308 of FIG. 3 are realized. In the server device 100, the functions of the authentication unit 402, the conversation state monitoring unit 404, the word extraction unit 405, the location information analysis unit 407, and the topic providing unit 106 in FIG. 4 are realized.

  The system according to the present embodiment is realized by the CPU 501 executing a program having functions realized by the flowcharts of FIGS. 6, 8, and 9. For example, the program may be recorded and distributed in the external storage device 505 or the portable recording medium 509, or may be acquired from the network by the communication interface 507.

  FIG. 6 is a flowchart showing an example of interactive communication processing realized as processing in which the CPU 501 in the server device 100 and the CPU 501 in the terminal device 220 execute the respective interactive communication processing programs stored in the memory 502.

  First, the terminal device 220A of FIG. 2 operated by the user A starts using the interactive communication service (step S2). As a result, the CPU 501 of the terminal device 220A transmits an authentication request and a connection request to the terminal device 220B of FIG. 2 operated by the user B to the server device 100 via the communication interface circuit 507 and the network 250 (FIG. 2). (Communication L2).

  As a result, in the server apparatus 100, when the CPU 501 receives the authentication request via the communication interface circuit 507, the CPU 501 starts an authentication process (step S4). The CPU 501 collates the user name and password included in the authentication request with the user name and password stored in the user DB 403 (FIG. 4) in the external storage device 205. If the collation result is OK, the CPU 501 transmits a connection notification to the connection destination terminal device 220B via the communication interface circuit 507 and the network 250 (FIG. 2) (communication L4).

  The processing in step S4 described above realizes the function of the authentication unit 402 in FIG.

  In the terminal device 220B, when the CPU 501 receives the connection notification via the communication interface circuit 507, the CPU 501 executes a use start process by peer-to-peer connection with the terminal device 220A (step S6). Thereafter, in the terminal device 220B, the CPU 501 notifies the server device 100 of the start of use via the communication interface circuit 507 and the network 250 (FIG. 2) (communication L6).

  In the server device 100, when the CPU 501 receives the use start via the communication interface circuit 507, the CPU 501 starts a process for providing a topic (step S8).

  After the start of interactive communication, the CPU 501 of the terminal devices 220A and 220B periodically executes the processes of the voice analysis unit 307 and the position data acquisition unit 308 described above with reference to FIG. The voice information and the position data are transmitted to the server device 100 via In the server apparatus 100, the above-described voice information and position data are periodically received from the terminal apparatuses 220A and 220B in the iterative process from step S12 to step S30 (the communication L9).

  The processing of the communication L9 described above realizes the functions of the voice analysis unit 307 and the position data acquisition unit 308 in FIG.

  In the server apparatus 100, the CPU 501 executes the conversation state monitoring process in the repetition process from step S12 to step S30 (step S12), so that the conversation state of the interactive communication by the terminal devices 220A and 220B is Whether it is in an active state or a stagnation state is monitored (step S14). FIG. 7 is an operation explanatory diagram of the conversation state monitoring process in steps S12 and S14. The CPU 501 of the server device 100 monitors voice average power information included in the voice information periodically transmitted from the terminal devices 220A and 220B (whichever is acceptable). For example, as shown in the period T1, if the value 701 of the voice average power information is between the upper limit value VH and the lower limit value VL that are threshold values, the CPU 501 has a normal conversation state in the period T1. (This determination may not be performed). Further, as shown in the period T2, if the value 701 of the voice average power information is below the lower limit value VL, the CPU 501 determines that the conversation state in the period T2 is a stagnation state. Furthermore, as shown in the period T3, if the value 701 of the voice average power information exceeds the upper limit value VH, the CPU 501 determines that the conversation state in the period T3 is an active state.

  The processes in steps S12 and S14 described above realize the function of the conversation state monitoring unit 404 in FIG.

  In the server device 100, when the CPU 501 determines in step S14 that the conversation state between the terminal devices 220A and 220B is active, the CPU 501 executes active state processing (step S16).

  FIG. 8 is a flowchart illustrating a detailed example of the active state process in step S16 of FIG. 6 executed by the CPU 501 of the server apparatus 100.

  In FIG. 8, first, the CPU 501 receives the voice information transmitted from each of the terminal devices 220A and 220B by the communication L9 of FIG. 6 described above via the communication interface circuit 507 (step S801). Note that the terminal devices 220 </ b> A and 220 </ b> B may operate so that only one of the overlapping audio information is transmitted to the server device 100.

  Next, the CPU 501 executes speech recognition processing for recognizing the speech indicated by the speech information as text data, using the above-described feature vector data in the received speech information (step S802). In this process, for example, a speech recognition process using a hidden Markov model is executed using a set of feature vector data for a plurality of consecutive periods.

  Next, the CPU 501 outputs a plurality of morphemes constituting the text data by performing morphological analysis on the text data obtained in step S802 (step S803).

  Next, the CPU 501 executes word selection processing (step S804). Here, the CPU 501 extracts a morpheme corresponding to a predetermined part of speech as a word from the plurality of morphemes obtained in step S803.

  Next, the CPU 501 executes word registration processing (step S805). Here, the CPU 501 identifies the word selected in step S804, identification information of interactive communication between the terminal devices 220A and 220B from which the word is extracted (identified in step S8 in FIG. 6), The date and time information from which the word is extracted is registered in the word DB 405 (see FIG. 4) in the external storage device 505. If the same word is already registered in the word DB 406, the CPU 501 overwrites and records the identification information and date / time information of the interactive communication of the word with new information.

  The processes from steps S801 to S805 described above realize the function of the word extraction unit 405 in FIG.

  Subsequently, the CPU 501 receives the position data transmitted from each of the terminal devices 220A and 220B through the communication L9 of FIG. 6 described above via the communication interface circuit 507 (step S806). Note that the terminal devices 220 </ b> A and 220 </ b> B may operate so that only one of the overlapping position data is transmitted to the server device 100.

  Next, the CPU 501 executes address information inquiry processing (step S807). Here, the CPU 501 is connected to the network 250 in FIG. 2 via the communication interface circuit 507 using the latitude / longitude data, which is the position data received in step S806, as a key, or is not particularly illustrated. By making an inquiry to the map database, address information corresponding to the position data is acquired.

  Next, the CPU 501 executes representative place name selection processing (step S808). Here, the CPU 501 selects a representative place name from the address information acquired in step S807.

  Finally, the CPU 501 executes position information registration processing (step S809). Here, the CPU 501 uses the representative place name selected in step S808 as position information, the position information, the identification information of the terminal devices 220A and 220B from which the position information is extracted, and the date and time information from which the position information is extracted. And recorded in the position information DB 408 (see FIG. 4) in the external storage device 505. When the same position information is already registered in the position information DB 408, the CPU 501 overwrites and records the identification information and date / time information of interactive communication of the position information with new information.

  The processing from step S806 to S809 described above realizes the function of the position information analysis unit 407 in FIG.

  After the process of step S809, the process of the flowchart of FIG. 8 is terminated, and the active state process of step S16 of FIG. 6 is terminated.

  As described above, in the present embodiment, when the conversation state is in the active state, it is determined that the word or position information that has appeared in the conversation at that time is a cause of the conversation, and the word DB 405 and the position in FIG. Accumulated in the information DB 408.

  On the other hand, in the server device 100, when the CPU 501 determines in step S14 in FIG. 6 that the conversation state between the terminal devices 220A and 220B is stagnant, it executes a stagnant state process (step S20).

  FIG. 9 is a flowchart illustrating a detailed example of the stagnation state process in step S20 of FIG. 6 executed by the CPU 501 of the server apparatus 100.

  In FIG. 9, first, the CPU 501 extracts, for example, the words 410 (see FIG. 4) registered within the last few months, for example, at random from the word DB 405 in the external storage device 505 (step S901).

  Next, the CPU 501 extracts, for example, the position information 411 (see FIG. 4) registered within the last few months from the position information DB 408 in the external storage device 505, for example, at random (step S902). ).

  Next, the CPU 501 transmits the topic 412 (see FIG. 4) that matches both the word 410 extracted in step S901 and the position information 411 extracted in step S902 via the communication interface circuit 507 and the network 250 in FIG. Then, the server device 230 at the search site shown in FIG. 2 is accessed and searched (step S903).

  In step S903, the CPU 501 determines whether a search result has been answered from the server device 230 at the search site (step S904).

  If the determination in step S904 is YES, the CPU 501 outputs the search result as the topic 412 (see FIG. 4) (step S909). Thereafter, the processing of the flowchart of FIG. 9 ends.

  If the determination in step S904 is NO, the CPU 501 extracts, for example, a word 410 (see FIG. 4) whose registration date and time is less than six months from the word DB 405 in the external storage device 505, for example, at random (step S905).

  Next, the CPU 501 accesses the news site server apparatus 240 in FIG. 2 via the communication interface circuit 507 and the network 250 in FIG. 2 and acquires the latest keyword of the news site (step S906).

  Next, the CPU 501 transmits a topic 412 that matches both the word 410 extracted in step S905 and the recent keyword of the news site acquired in step S906 via the communication interface circuit 507 and the network 250 in FIG. The server device 230 at the search site shown in FIG. 2 is accessed and searched (step S907).

  In step S907, the CPU 501 determines whether a search result has been answered from the server device 230 at the search site (step S908).

  If the determination in step S908 is YES, the CPU 501 outputs the search result as the topic 412 (see FIG. 4) (step S909). Thereafter, the processing of the flowchart of FIG. 9 ends.

  If the determination in step S908 is NO, the CPU 501 returns to the process in step S901 and repeats random extraction of new keywords from the word DB 405.

  After the output processing of the topic 412 in step S909, the CPU 501 ends the processing in the flowchart in FIG. 9, and returns to the stagnation state processing in step S20 in FIG. When the topic 412 is obtained by the above-described processing in the stagnation state processing, the CPU 501 transmits it to the terminal devices 220A and 220B via the communication interface circuit 507 and the network 250 (communication L10 in FIG. 6).

  In FIG. 3, each of the CPUs 501 in the terminal devices 220 </ b> A and 220 </ b> B receives the above topic 412 via the communication interface circuit 507, the text data or image data included in the topic 412 is displayed on the monitor of the output device 204. Is displayed on the type communication screen (steps S22 and S24 in FIG. 6).

  FIGS. 10A and 10B are diagrams illustrating examples of interactive communication displayed on the monitors of the terminal device 220A and the terminal device 220B of the present embodiment by the above-described interactive communication processing, respectively. In FIG. 10, 2204 corresponds to the camera 510 of FIG. Reference numeral 2206 denotes a speaker. Reference numeral 2202 denotes a display area for interactive communication. Among these, 2210 is a display area of the user camera image of the counterpart terminal. Reference numeral 2212 denotes a display area for the own camera image. Reference numeral 2214 denotes a topic display area. For example, the recently-talked word randomly extracted in step S901 in FIG. 9 is “Sakura”, and the image shared in the recent interactive communication randomly extracted in step S902 in FIG. If the location information is “Karuizawa”, a search on the search site using the keywords “Sakura” and “Karuizawa” as a key results in the topic 412 “Sakura in Karuizawa is in full bloom!”, For example. It is displayed in the topic display area 2214 of the terminal devices 220A and 220B. As a result, the users A and B who are performing interactive communication can view the topic display, recall the Karuizawa cherry blossoms again, and enjoy the conversation, thereby enabling the conversational conversation to proceed more smoothly. Although the short text search result is displayed as an example, the entire corresponding home page may be displayed. Also, a homepage link may be displayed.

  FIGS. 11A and 11B are diagrams showing display examples of home pages in interactive communication displayed on the monitors of the terminal device 220A and the terminal device 220B of the present embodiment by the above-described interactive communication processing, respectively. is there. Each part constituting the screen is the same as in FIG. As in the case of FIG. 10, for example, the recently-talked word randomly extracted in step S901 of FIG. 9 is “Sakura”, and the recent dialog randomly extracted in step S902 of FIG. When the position information of the image shared by the type communication is “Karuizawa”, as a result of searching on the search site using the keywords “Sakura” and “Karuizawa” as a key, the content of the home page including those keywords is the topic 412. And is displayed in the topic display area 2214 of the terminal devices 220A and 220B as shown in FIGS. 11 (a) and 11 (b). As a result, users A and B performing interactive communication can come up with a more specific image related to “Sakura” and “Karuizawa”.

  Next, another embodiment of the terminal device 220 and the server device 100 in FIG. 2 will be described.

  FIG. 12 is a block diagram of another embodiment of the terminal device 220 of FIG. In FIG. 12, the same number is attached | subjected to the part of the same function as the case of one Embodiment of the terminal device 220 of FIG. 12 differs from the configuration of FIG. 3 in that the terminal device 220 includes a sensor group 1201 in addition to or instead of the GPS reception unit 305, and in addition to or instead of the position data acquisition unit 308. The sensor data group acquisition unit 1202 is provided.

  The sensor group 1201 is, for example, a set of one or more environmental sensors, and includes an air temperature sensor, an atmospheric pressure sensor, a humidity sensor, an illuminance sensor, and the like, and each has a terminal device 220 such as an air temperature, an atmospheric pressure, a humidity, and an illuminance. Detect environmental data.

  Alternatively, the sensor group 1201 is a set of one or more environmental sensors, for example, a body temperature sensor, a pulse sensor, a blood pressure sensor, and the like, and operates the terminal device 220 such as a user's body temperature, pulse, blood pressure, etc. Detect data on the physical condition of elderly users.

  The sensor data group acquisition unit 1202 displays the detected values such as temperature, pressure, humidity, illuminance, or body temperature, pulse, blood pressure, etc. detected by the sensor group 1201 when the interactive communication is performed. The data is acquired as a sensor data group via 301 and output to the communication interface unit 401.

  In this case, the communication interface unit 309 transmits the sensor data group output from the sensor data group acquisition unit 1202 as needed to the server apparatus 100 via the network 250 in FIG.

  FIG. 13 is a block diagram of another embodiment of the server apparatus 100 of FIG. 13, parts having the same functions as those in the embodiment of the server apparatus 100 shown in FIG. 13 differs from the configuration of FIG. 4 in that the server apparatus 100 adds a sensor information analysis unit 1301 and a sensor information database (sensor information DB) in addition to or instead of the position information analysis unit 407 and the position information DB 408. ) 1302.

  For example, when the conversation state monitoring unit 404 detects an active state, the sensor information analysis unit 1301 detects the sensor data group from the sensor data group input via the communication interface unit 401 from the terminal device 220 in which the state is detected. The environment or physical condition corresponding to is analyzed as sensor information. Examples of the environmental state include “hot temperature”, “low pressure approaching”, “easy”, “daytime or night”, and the like. The state of physical condition is, for example, information related to physical condition such as “having fever”, “pulse is fast”, “blood pressure is high or low”. Then, the sensor information analysis unit 1301 records the extracted sensor information, identification information of interactive communication from which the sensor information is extracted, and date / time information from which the sensor information is extracted, in the sensor information DB 1302.

  When the conversation state monitoring unit 404 detects a stagnation state, the topic providing unit 409, for example, in addition to the word 410 extracted from the word DB 405 and the position information 411 extracted from the position information DB 408, as described with reference to FIG. Sensor information related to the environment of the latest terminal device 220, such as “temperature is hot”, “low pressure approaching”, “bullying”, “daytime or night”, and the physical condition of the user of the latest terminal device 220 Sensor information 1303 such as “having fever”, “pulse is fast”, “blood pressure is high or low”, for example, is extracted at random. Then, the topic providing unit 409 converts the extracted word 410 and the topic 412 that matches or relates to all of the position information 411 and / or the sensor information 1303 (which may be either) into the communication interface unit 401 and the figure. The search site server device 230 and the news site server device 240 are accessed and searched via the second network 250. The topic providing unit 409 transmits the searched topic 412 to the terminal device 101 performing interactive communication in which the conversation state monitoring unit 404 has detected a stagnation state.

  In this manner, according to the other embodiment of the terminal device 220 of FIG. 12 and the server device 100 of FIG. 13, the state of the immediate environment or the condition of the elderly user is being performed during interactive communication or It is possible to provide a topic according to the state.

  Although this form is suitable for the video chat system for the elderly for the purpose of preventing dementia, for example, it is not restricted to this. For example, in English conversation classes using video chat, beginners often find that English conversation is stagnant because English words do not come out of their mouths. In such a case, when the present embodiment is applied and English words (or Japanese) which are hints of a topic are displayed, the conversation can be carried smoothly and a larger amount of conversation can be handled within the specified time.

  The function sharing shown in FIGS. 3 and 4 between the terminal device 220 and the server device 100 is based on various functions sharing in consideration of the efficiency of peer-to-peer connection and the efficiency by distributing the server devices in a plurality of ways. It is possible to adopt a configuration.

Regarding the above embodiment, the following additional notes are disclosed.
(Appendix)
(Appendix 1)
A conversation state monitoring unit that monitors the active / stagnation state of conversation in interactive communication by a plurality of terminal devices connected via a network;
An extraction / recording unit for extracting / recording predetermined information in the interactive communication;
A topic providing unit that supplies a topic related to the recorded predetermined information to the terminal device when the conversation state is a stagnation state;
An interactive communication system comprising:
(Appendix 2)
The topic providing unit searches for a topic related to the extracted predetermined information by accessing a server device of a search site, and supplies the searched topic to the terminal device.
The interactive communication system according to appendix 1, wherein
(Appendix 3)
The extraction / recording unit extracts / records words appearing in conversation in interactive communication as the predetermined information,
The topic providing unit extracts a phrase that matches a predetermined condition from the recorded phrase, and supplies a topic related to the extracted phrase to the terminal device. Interactive communication system.
(Appendix 4)
The extraction / recording unit further includes a position information analysis / recording unit that acquires and analyzes word data that appears in the conversation in interactive communication as the predetermined information, or position data of the terminal device, and records the position data as position information. Prepared,
The interactive communication system according to appendix 1 or 2, wherein the topic providing unit supplies a topic related to the extracted position information to the terminal device.
(Appendix 5)
The position information analysis / recording unit
A position data acquisition unit that acquires latitude / longitude data included in the metadata of an image that appears in the conversation as the position data;
The interactive communication system according to supplementary note 4, wherein
(Appendix 6)
The position information analysis / recording unit
An address information inquiry unit that acquires address information corresponding to the position data by making an inquiry to the map database using the position data acquired by the position data acquisition unit as a key;
A representative place name selection unit for selecting a representative place name from the acquired address information;
The representative place name is registered in the position information database as the position information;
The interactive communication system according to appendix 5, which includes:
(Appendix 7)
The extraction / recording unit extracts and records environment information in interactive communication as the predetermined information,
The interactive communication system according to appendix 1 or 2, wherein the topic providing unit supplies a topic related to the extracted / recorded environment information to the terminal device.
(Appendix 8)
The extraction / recording unit includes a sensor information analysis / recording unit that acquires and analyzes environmental data around the terminal device at the time of the conversation or physical condition data of a user of the terminal device and records it as sensor information
The topic providing unit further extracts sensor information that matches a predetermined condition from the recorded sensor information when the conversation state is a stagnation state, and a topic related to the extracted sensor information is extracted from the terminal device. To supply,
The interactive communication system according to appendix 7, which is characterized in that.
(Appendix 9)
The environmental data is at least one of temperature, atmospheric pressure, humidity, and illuminance,
The physical condition data is at least one of body temperature, pulse, blood pressure,
The interactive communication system according to appendix 8, wherein the interactive communication system is characterized.
(Appendix 10)
10. The appendix 1 to 9, wherein the conversation activity / stagnation state monitoring unit, the extraction / recording unit, and the topic providing unit are integrally provided in the terminal device. Interactive communication system.
(Appendix 11)
A terminal device used in the interactive communication system according to any one of appendices 1 to 9.
(Appendix 12)
To a terminal device computer connected to another terminal device via a network,
Conversation activity / stagnation status monitoring processing for monitoring conversation activity / stagnation status in interactive communication;
An extraction / recording process for extracting / recording predetermined information in the interactive communication;
When the conversation state is a stagnation state, a topic providing process for searching for a topic related to the extracted predetermined information by accessing a server device of a search site and supplying the searched topic to the terminal device When,
A program that executes

DESCRIPTION OF SYMBOLS 100 Server apparatus 101,220,220A, 220B, 220C Terminal apparatus 102,404 Conversation state monitoring part 103 Phrase extraction and recording part 104 Recorded phrase 105 Extracted phrase 106,409 Topic provision part 107 Location information analysis and recording part 108 Recorded location information 109 Extracted location information 110, 412 Topic 111, 230 Search site server device 240 News site server device 250 Network 301 Standing sensor unit 302 Voice input / output unit 303 Imaging unit 304 Display unit / operation input Unit 305 GPS receiver 306 Data storage unit 307 Voice analysis unit 308 Position data acquisition unit 309, 401 Communication interface unit 402 Authentication unit 403 User DB (database)
405 Word extraction unit 406 Word DB (database)
407 Location information analysis unit 408 Location information DB (database)
410 extracted word 411 extracted position information 501 CPU (central processing unit)
502 Memory 503 Input Device 504 Output Device 505 External Storage Device 506 Portable Recording Medium Drive Device 507 Communication Interface Circuit 508 Bus 509 Portable Recording Medium 510 Camera 511 GPS Receiver 1201 Sensor Group 1202 Sensor Data Group Acquisition Unit 1301 Sensor Information Analysis Unit 1302 Sensor information DB (database)

Claims (12)

A conversation state monitoring unit that monitors the active / stagnation state of conversation in interactive communication by a plurality of terminal devices connected via a network;
An extraction / recording unit for extracting / recording predetermined information in the interactive communication;
A topic providing unit that supplies a topic related to the recorded predetermined information to the terminal device when the conversation state is a stagnation state;
An interactive communication system comprising: The topic providing unit searches for a topic related to the extracted predetermined information by accessing a server device of a search site, and supplies the searched topic to the terminal device.
The interactive communication system according to claim 1. The extraction / recording unit extracts / records words appearing in conversation in interactive communication as the predetermined information,
The topic providing unit extracts a phrase that matches a predetermined condition from the recorded phrase, and supplies a topic related to the extracted phrase to the terminal device. Interactive communication system. The extraction / recording unit further includes a position information analysis / recording unit that acquires and analyzes word data that appears in the conversation in interactive communication as the predetermined information, or position data of the terminal device, and records the position data as position information. Prepared,
The interactive communication system according to claim 1, wherein the topic providing unit supplies a topic related to the extracted position information to the terminal device. The position information analysis / recording unit
A position data acquisition unit that acquires latitude / longitude data included in the metadata of an image that appears in the conversation as the position data;
The interactive communication system according to claim 4. The position information analysis / recording unit
An address information inquiry unit that acquires address information corresponding to the position data by making an inquiry to the map database using the position data acquired by the position data acquisition unit as a key;
A representative place name selection unit for selecting a representative place name from the acquired address information;
The representative place name is registered in the position information database as the position information;
The interactive communication system according to claim 5, comprising: The extraction / recording unit extracts and records environment information in interactive communication as the predetermined information,
The interactive communication system according to claim 1, wherein the topic providing unit supplies a topic related to the extracted and recorded environmental information to the terminal device. The extraction / recording unit includes a sensor information analysis / recording unit that acquires and analyzes environmental data around the terminal device at the time of the conversation or physical condition data of the user of the terminal device and records it as sensor information,
The topic providing unit further extracts sensor information that matches a predetermined condition from the recorded sensor information when the conversation state is a stagnation state, and a topic related to the extracted sensor information is extracted from the terminal device. To supply,
The interactive communication system according to claim 7. The environmental data is at least one of temperature, atmospheric pressure, humidity, and illuminance,
The physical condition data is at least one of body temperature, pulse, blood pressure,
The interactive communication system according to claim 8.   The active / stagnation state monitoring unit of the conversation, the extraction / recording unit, and the topic providing unit are integrally provided in the terminal device. Interactive communication system.   A terminal device used in the interactive communication system according to claim 1. To a terminal device computer connected to another terminal device via a network,
Conversation activity / stagnation status monitoring processing for monitoring conversation activity / stagnation status in interactive communication;
An extraction / recording process for extracting / recording predetermined information in the interactive communication;
When the conversation state is a stagnation state, a topic providing process for searching for a topic related to the extracted predetermined information by accessing a server device of a search site and supplying the searched topic to the terminal device When,
A program that executes