KR20160100811A - Method and device for providing information - Google Patents

Abstract

Provided are a method and a device for providing information. The method for providing information includes a step of obtaining speech information provided by at least one participant participating in conversation by using a communication terminal; a step of obtaining state information which shows the present state of the at least one participant by using the speech information and/or stored data; a step of determining a virtual speech provided to the at least one participant based on the speech information and the state information; and a step of outputting the determined virtual speech. So, speech recognition can be carried out.

Description

[0001] The present invention relates to a method and a device for providing information,

The present disclosure relates to a device and method for providing information, and more particularly, to a device and method for providing speech information.

Along with the development of electronic technology, calls using smart phones, PCs, tablet computers, and the like have become commonplace. In addition, it is common that a call is made between two persons. In the currency, communication is generally performed through voice.

Also, in connection with speech recognition technology, a method is used in which a user's voice is received as input, the received user's voice is analyzed, and an output corresponding to the user's voice is generated.

Specifically, a speech recognition technology is a technique for a computer to analyze or recognize an acoustic speech signal, such as a speech uttered by a human being. The speech recognition technique converts an uttered voice into an electrical signal, And extracts the frequency characteristic of the speech.

Due to the spread of computers and the increase of users, computers are widely used in daily life such as education, games, and business. Recently, various technologies using computer have been developed due to the development of computer technology. In addition, speech recognition technology using a computer and a computer is used in various fields.

Along with the development of electronic technology, calls using smart phones, PCs, tablet computers, and the like have become commonplace. In addition, it is common that a call is made between two persons. In the currency, communication is generally performed through voice.

Also, in connection with speech recognition technology, a method is used in which a user's voice is received as input, the received user's voice is analyzed, and an output corresponding to the user's voice is generated.

Specifically, a speech recognition technology is a technique for a computer to analyze or recognize an acoustic speech signal, such as a speech uttered by a human being. The speech recognition technique converts an uttered voice into an electrical signal, And extracts the frequency characteristic of the speech.

Due to the spread of computers and the increase of users, computers are widely used in daily life such as education, games, and business. Recently, various technologies using computer have been developed due to the development of computer technology. In addition, speech recognition technology using a computer and a computer is used in various fields.

According to a first aspect of the present invention, there is provided an information providing method for acquiring speech information provided by at least one participant participating in a conversation using a communication terminal through the communication terminal ; Obtaining status information indicating the current status of the at least one participant using the speech information and / or pre-stored data; Determining a virtual speech to be provided to the at least one participant based on the speech information and the status information; And outputting the determined virtual speech.

In addition, the step of acquiring the speech information through the communication terminal may include outputting a virtual speech inquiring to acquire information on the at least one participant; And obtaining the speech information including the additional speech information obtained through the inquiring virtual speech.

The status information may further include emotion information indicating a current emotional state of the at least one participant, personal information of the at least one participant, topic information of the conversation, Schedule information of one participant, and personality information of the at least one participant.

In addition, the speech information may include voice information of the at least one participant acquired by the communication terminal.

The determining of the virtual speech may include obtaining the content of the virtual speech using the speech information; Determining a presentation method of the virtual speech using the status information; And obtaining the virtual speech in which the obtained content is represented by the determined presentation method.

In addition, the step of determining the virtual speech may include: determining whether the speech information includes a predetermined keyword; And if the speech information includes the predetermined keyword, obtaining the virtual speech including content for a subject different from the subject of the conversation.

In addition, the status information may include emotion information of the at least one participant,

The step of determining the virtual speech may acquire the virtual speech of the content and the presentation method determined according to the emotion information.

In addition, the emotion information may include information on the type of emotion and information on the level of emotion.

The determining of the virtual speech may include determining whether the emotion state of the at least one participant is abnormal using the emotion information, And if the emotional state is determined to be abnormal, obtaining the virtual speech including content for a subject different from the subject of the conversation.

In addition, the step of determining whether the emotional state is abnormal may determine whether the emotional state is abnormal based on the type of emotions of the at least one participant and the duration of the emotional state of the kind.

Further, as a technical means for achieving the above-mentioned technical object, a second aspect of the present disclosure relates to a method for acquiring speech information provided by at least one participant participating in a conversation using a communication terminal, Acquiring status information indicating a status using the speech information and / or pre-stored data, and determining a virtual speech to be provided to the at least one participant based on the speech information and the status information; And an output unit outputting the determined virtual speech.

The output may also output a virtual speech query to obtain information about the at least one participant and the control may obtain the speech information comprising additional speech information obtained via the querying virtual speech have.

The status information may further include emotion information indicating a current emotional state of the at least one participant, personal information of the at least one participant, topic information of the conversation, Schedule information of one participant, and personality information of the at least one participant.

In addition, the speech information may include voice information of the at least one participant acquired by the communication terminal.

The control unit may acquire the content of the virtual speech using the speech information, determine a presentation method of the virtual speech using the status information, and transmit the virtual content to the virtual Speech can be obtained.

In addition, the control unit may determine whether the speech information includes a predetermined keyword, and when the speech information includes the predetermined keyword, the control unit may transmit the virtual speech including contents for a subject different from the subject of the conversation Can be obtained.

In addition, the status information may include emotion information of the at least one participant, and the controller may obtain the virtual speech of the content and the presentation mode determined according to the emotion information.

In addition, the emotion information may include information on the type of emotion and information on the level of emotion.

The control unit may determine whether the emotion state of the at least one participant is abnormal by using the emotion information, and when the emotion state is determined to be abnormal, The virtual speech can be obtained.

As a technical means for achieving the above technical object, a third aspect of the present disclosure provides an information providing communication terminal, which uses the communication terminal to acquire speech information provided by at least one participant participating in a conversation , Using the speech information and / or pre-stored data to obtain status information indicative of the current status of the at least one participant, and generating virtual speech to be provided to the at least one participant based on the speech information and the status information ; And an output unit outputting the determined virtual speech.

As a technical means for achieving the above technical object, a fourth aspect of the present disclosure provides an information providing server for acquiring speech information provided by at least one participant participating in a conversation using a communication terminal, Obtaining status information indicating the current status of at least one participant using the speech information and / or pre-stored data, and determining a virtual speech to be provided to the at least one participant based on the speech information and the status information A control unit; And an output unit outputting the determined virtual speech.

In addition, the fifth aspect of the present disclosure can provide a computer-readable non-volatile recording medium recording a program for executing the method of the first aspect in a computer.

1 is a diagram illustrating an example in which a device according to one embodiment provides information.
2 is a flow diagram of a method for a device according to an embodiment to obtain virtual speech.
3 is a flow diagram of a method according to an embodiment for obtaining and outputting a virtual speech.
4 is a flow diagram of a method according to an embodiment for obtaining a virtual speech using speech information and state information.
5 is a flow diagram of a method for a device according to an embodiment to obtain virtual speech according to whether a keyword preset in the speech information is included.
6 is a flow diagram of a method for a device according to an embodiment to obtain virtual speech based on emotion information.
7 is a diagram illustrating an example in which a device according to an embodiment acquires virtual speech using a keyword included in speech information.
8 is a diagram illustrating an example in which a device according to an embodiment acquires virtual speech using a keyword included in speech information.
9 is a block diagram illustrating a configuration of a device for providing information according to an embodiment.
10 is a block diagram illustrating a configuration of a receiving unit according to an embodiment.
11 is a block diagram showing a configuration of a control unit according to an embodiment.
12 is a block diagram showing a configuration of a speech synthesis unit according to an embodiment.
13 is a block diagram showing a configuration of a control unit according to an embodiment.
14 is a block diagram showing the configuration of a control unit according to an embodiment.
15 is a block diagram illustrating a configuration of a database according to an embodiment.
16 is a block diagram showing a configuration of a database according to an embodiment.
17 is a diagram showing an example of emotion information according to an embodiment.
18 is a diagram illustrating an example in which a server according to an embodiment provides information.
19 is a diagram illustrating an example in which a communication terminal according to an embodiment provides information.

Brief Description of the Drawings The advantages and features of the present invention, and how to accomplish them, will become apparent with reference to the embodiments described hereinafter with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims.

The terms used in this specification will be briefly described and the present invention will be described in detail.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. Also, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term, not on the name of a simple term, but on the entire contents of the present invention.

When an element is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements as well, without departing from the spirit or scope of the present invention. Also, as used herein, the term "part " refers to a hardware component such as software, FPGA or ASIC, and" part " However, "part" is not meant to be limited to software or hardware. "Part" may be configured to reside on an addressable storage medium and may be configured to play back one or more processors. Thus, by way of example, and not limitation, "part (s) " refers to components such as software components, object oriented software components, class components and task components, and processes, Subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functions provided in the components and "parts " may be combined into a smaller number of components and" parts " or further separated into additional components and "parts ".

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

Hereinafter, an electronic apparatus related to the present invention will be described in more detail with reference to the drawings. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

An embodiment "or" an embodiment "of the principles of the present invention as used herein is intended to mean the particular features, structures, features, etc., which are described in connection with the embodiments included in at least one embodiment of the principles of the invention . Therefore, the appearances of the phrase "in one embodiment" or "in an embodiment" appearing in various places throughout this specification are not necessarily all referring to the same embodiment.

Figure 1 is an illustration of an example in which device 100 provides information in accordance with one embodiment.

The currency may be performed on two humans, or on three humans. For example, two human conversations can be performed using the first communication terminal 110 and the second communication terminal 120, and in other examples, the first communication terminal 110, the second communication terminal 120, And the device 100 may be used to perform a three-party call.

2 When a human call is performed, tension, inconvenience, and the like may be caused. However, when three human calls are performed, relative tension and inconvenience may be solved. For example, according to one embodiment, when the device 100 participates in a call as a third participant during a two-person call through the first communication terminal 110 and the second communication terminal 120, The atmosphere of a closed-and-tit-for-tat two-person call can be transformed into a calm atmosphere. The third participant participating in the call by the device 100 according to one embodiment may be a virtual character. For example, the voice signal transmitted to the first communication terminal or the second communication terminal by the device 100 may be the virtual speech of the virtual character.

The device 100 may be implemented as a separate server from the first communication terminal 110 or the second communication terminal 120 so that the first communication terminal 110 and the second communication terminal 120 It can intervene in communication.

The device 100 may be arranged within the first communication terminal 110 to intervene in communication between the first communication terminal 110 and the second communication terminal 120 according to one embodiment.

The device 100 may be arranged within the second communication terminal 120 to intervene in communication between the first communication terminal 110 and the second communication terminal 120 according to one embodiment.

2 is a flow diagram of a method for a device 100 according to one embodiment to obtain a virtual speech.

In step S210, the device 100 according to an embodiment acquires, via the communication terminal, speech information provided by at least one participant participating in the conversation using the communication terminal.

Speech information according to one embodiment may include information obtained via a voice signal. For example, the speech information may include information obtained from a voice signal of a communication terminal user received via the communication terminal. As another example, the speech information may include keyword information obtained through analysis of the speech signal of the communication terminal user. As another example, the speech information may include voice information obtained through analysis of the speech signal of the communication terminal user. In another example, the speech information may include speech information of at least one participant that the communication terminal acquires.

The device 100 according to one embodiment may obtain speech information from one or more communication terminals. For example, the device 100 may obtain speech information from the first communication terminal and / or the second communication terminal.

The communication terminal according to an exemplary embodiment may include a microphone and a speaker. In addition, the communication terminal according to one embodiment may include, but is not limited to, a smart phone, a tablet computer, a personal digital assistant, a laptop computer, a desktop computer, and the like.

A communication terminal that provides speech information to the device 100 according to one embodiment may perform a call function. A communication terminal according to an exemplary embodiment may include a local communication terminal or a remote communication terminal in a call. For example, the first communication terminal may be a local communication terminal, and the second communication terminal may be a remote communication terminal. The device 100 according to one embodiment may be implemented in a server form, a form embedded in a communication terminal, or a separate device placed outside a communication terminal.

The device 100 according to one embodiment may analyze voice signals obtained through a communication terminal to obtain speech information.

Alternatively, the communication terminal according to an exemplary embodiment may acquire a voice signal and transmit the result of analyzing the acquired voice signal to the device 100. The device 100 can acquire the speech information using the speech signal analysis result received from the communication terminal.

The communication terminal or device 100 can receive a voice signal. For example, the user's voice signal can be received through the microphone of the communication terminal. For example, when the user's voice signal of the first communication terminal is received through the microphone of the first communication terminal, the device 100 can acquire the voice signal of the first communication terminal user through the first communication terminal . As another example, when the user's voice signal of the second communication terminal is received via the microphone of the second communication terminal, the device 100 can acquire the voice signal of the user of the second communication terminal through the second communication terminal. As another example, when the voice signal of the user of the local communication terminal is received through the microphone of the local communication terminal, the device 100 can acquire the voice signal of the user of the local communication terminal through the local communication terminal. As another example, when the user's voice signal of the remote communication terminal is received through the microphone of the remote communication terminal, the device 100 can acquire the voice signal of the user of the remote communication terminal through the remote communication terminal.

A user's voice signal input to the remote communication terminal may be transmitted to the device 100 or the local communication terminal through a wireless connection or a wired connection. Non-audio information (eg musical notation) can be edited via a keyboard, button input, etc., or converted into audio information.

Wireless connections include, but are not limited to, 2G / 3G / 4G connections, WiFi connections, Bluetooth connections, WiMAX connections, ZigBee connections, Ultra Wide Band (UWB) and other wireless connections now known or later developed.

In step S220, the device 100 according to one embodiment obtains status information indicating the current status of at least one participant using speech information and / or pre-stored data.

The status information according to one embodiment includes emotion information indicating the current emotional state of at least one participant and / or virtual character, personal information of at least one participant and / or virtual character, At least one of topic information, schedule information of at least one participant, and personality information of at least one participant and / or a virtual character.

The emotion information according to one embodiment may include information on the type of emotion and information on the level of emotion. Types of emotions may include, but are not limited to, emotion states such as elation, anger, sorrow, depression and mildness.

The device 100 according to one embodiment may obtain status information using the speech information obtained in step S210.

For example, when the first communication terminal acquires a voice signal of " There is a cold wind today ", the device 100 transmits, through the speech information obtained from the voice signal acquired by the first communication terminal, The user can obtain information about the current health status of the user. As another example, when the remote communication terminal acquires a voice signal "I fought my friend yesterday", the device 100 receives the voice signal acquired by the remote communication terminal through the wireless connection, State information indicating that the emotional state may be inconvenient.

The device 100 according to one embodiment may obtain status information using pre-stored data.

For example, the age information of the user can be obtained through the information of the date of birth of the user. The pre-stored data may be stored in the device 100, stored in the communication terminal, or stored in the server.

The device 100 according to an exemplary embodiment may acquire status information using the speech information and the pre-stored data obtained in step S210.

For example, if data for users A and B is stored beforehand, and user A refers to user B as father, the status information about the relationship between users A and B is as follows: The device 100 can acquire the < / RTI >

The acquired state information according to one embodiment may be stored in a memory of the communication terminal, a memory or a server (not shown) of the device 100, or the like. For example, the status information may be stored in a local communication terminal and / or a telecommunications terminal. As another example, the status information may be stored in the internal or external memory of the device 100, or in a server connected to the device 100. [

When the status information is stored in the server, the communication terminal or the device 100 can acquire status information from the server via the wired / wireless connection.

At step S230, the device 100 according to one embodiment obtains or determines a virtual speech to be provided to at least one participant based on the speech information and / or the status information. For example, the device 100 according to one embodiment may determine one of a plurality of virtual speech to be a virtual speech to be provided to the participant, and may generate a virtual speech to be provided to the participant.

The virtual speech according to an exemplary embodiment may be determined according to the speech information obtained in step S210 and the state information acquired in step S220.

According to one embodiment, the device 100 may provide virtual speech through a virtual character. According to one embodiment, the device 100 may determine the virtual speech based on the gender, personality, voice, etc. of the virtual character.

The device 100 according to an exemplary embodiment may determine the character, tone, etc. of the virtual character according to the status information, and the character of the virtual character and the like may be changed according to the changed status information.

The state information according to one embodiment may include a personality variable and / or a state variable. For example, the state information according to one embodiment may be determined according to a personality variable and / or a state variable.

The personality variable may be used to indicate the general tendency of the virtual character corresponding to the speech information input to the communication terminal, and may be changed through communication between the user of the communication terminal and another person. For example, personality variables can be classified as preference / sensitivity subject, preference / sensitivity keyword, sympathy, accent, adaptability, alterness, curiosity, opposition, eloquence, idioms, But is not limited to, at least one of loquacity, peculiarities, responsiveness, sentimentality, and sleep time. As an example, the preference / sensitivity theme may be used when the device 100 determines whether the virtual character will actively participate or rarely participate in the subject. As another example, the preference / sensitivity keywords may be used to indicate whether the virtual character is interesting (such as sports) or not (such as terrorism) for the subject of interest. As another example, compassion can be used for a person, object, or object when the virtual character indicates whether to give a positive comment or to give a negative comment. As another example, accents may be used to indicate the possible accent types and degrees of virtual characters. In another example, adaptability can be used to indicate the degree of long term variation in the character of the virtual character. As another example, the arousal level may be used to indicate the degree of sensitivity of the virtual character to input speech information. As another example, curiosity can be used when the virtual character indicates a questioning motion. As another example, the opposition may be used to indicate an action in which a virtual character performs an instruction. As another example, persuasive power can be used to indicate the tendency to use the virtual character's fluent and decorated sentences. As another example, an idiom can be used to represent a phrase or sentence that is often used by a virtual character. As another example, the discursiveness can be used to indicate the degree to which a virtual character speaks a lot. As another example, the substrate may be used to indicate a particular reaction mode of the virtual character for a particular subject. In another example, responsiveness can be used to indicate the aggressiveness of a virtual character's request or inquiry. As another example, emotion may be used to indicate a tendency for a virtual character to develop strong mood. As another example, sleep time can be used to indicate a tendency for a virtual character to have low responsiveness during day time.

The state variable according to an embodiment may be used to determine a behavior characteristic of a virtual character. The state variable may be determined according to the previous state variable, the speech information input through the communication terminal, the personality variable, and the like. For example, aggressiveness can indicate how aggressively a virtual character speaks about an issue. High aggressiveness can often indicate spontaneously speaking, long and fast. In another example, the emotional state may indicate the type of emotion (including joy and depression) and intensity that the virtual character represents through speech. In another example, the speech scheme represents the current speech of the virtual character. Speech schemes can include the degree and frequency of adaptation to a particular type of dialect, the degree of formality and familiarity, and the request for a particular voice. As another example, improvisation can indicate how aggressively a virtual character starts a conversation.

In step S230, the device 100 according to one embodiment obtains or determines a virtual speech to be provided to at least one participant based on the speech information and the status information. For example, the device 100 according to one embodiment may determine one of a plurality of virtual speech to be provided to at least one participant, and may generate virtual speech to be provided to at least one participant.

The device 100 according to an exemplary embodiment may generate a virtual speech having an expression attribute corresponding to the speech information obtained in step S210 using the state information acquired in step S220. The presentation attribute of the virtual speech or the presentation attribute of the speech information may be used to indicate expression related information such as the expression style and emotion of the speech and may include information that can be used to indicate the emotional state and / have.

The emotional state included in the presentation attribute according to an exemplary embodiment may include, but is not limited to, types such as elation, anger, sorrow, depression, and mildness. Each type of emotional state can be defined at a different level. For example, emotion states of elation may be defined at several levels such as low, medium, or high. The presentation mode according to one embodiment may include, but is not limited to, phrasing style, accent type, dialect frequency, dialect degree, dial tone, scene mode or background music.

According to one embodiment, the content of the virtual speech may be determined according to the status information, and the content and / or the presentation attribute of the speech information may be input to the communication terminal. For example, the device 100 according to one embodiment may analyze audio information input to a communication terminal with an audio processing technique (e.g., speech recognition technology) to obtain content, Lt; RTI ID = 0.0 > a < / RTI > In another example, device 100 according to one embodiment may analyze speech information that is input to or obtained from a communication terminal to obtain a presentation attribute of the speech information. The device 100 according to one embodiment may then generate virtual speech with presentation attributes determined according to analyzed speech information and status information.

For example, if the speech information input to the communication terminal includes a subject of football, and the subject of football is a preference topic of a virtual character (according to an exemplary embodiment, the preference subject may include a personality variable included in the status information of the virtual character , The content of the virtual speech to be generated can be determined to be related to the subject of the football and the type of emotion state of the presentation attribute of the virtual speech to be generated can be determined with joy.

As another example, when the presentation attribute of the speech information input to the communication terminal is analyzed as being included in the emotion state of the " grief " type, the content of the virtual speech may be generated to be related to the subject of " up ". In this case, the type of emotional state of the presentation attribute of the virtual speech to be generated may be determined to be " gentle ".

According to an embodiment, when the content of the speech information input to the communication terminal includes a predetermined type of keyword, the content of the virtual speech may include immediate information corresponding to the predetermined type. For example, if the content of the communication includes a keyword of the address type, the content of the virtual speech may include quick information relating to the address type such as a quick address update or a meeting place address.

According to one embodiment, the content of the virtual speech may be determined by the presentation attribute (such as an emotional state) of the speech information input to the communication terminal. An emotion state for a local user and a remote user in the communication terminal or the device 100 can be analyzed and obtained and the contents of the virtual speech can be adjusted according to the emotion state. For example, when the emotion state of the speech information is abnormal, the contents of the virtual speech may include subject information different from the current subject or predetermined instant information. The abnormal emotional state may be determined according to the abnormal emotional state type and / or the abnormal emotional state duration. Emotional state types can include neutral types such as positive types such as joy, excitement, joy, etc., negative types such as sadness, depression, anger, terrorism, and mildness. For example, a negative type of emotional state may be determined to be an abnormal emotional state. As another example, if the emotional state of a negative type such as sadness or depression continues for a predetermined period (such as one minute), the emotional state may be determined to be abnormal. As another example, if a positive type of emotional state such as interest or excitement persists for a predetermined period of time (such as 10 minutes), the emotional state may be determined to be abnormal.

According to one embodiment, when the person referred to in the speech information input to the communication terminal is a participant of the current call (e.g., a local user or a remote user participating in the call) or if the subject includes a predetermined topic, May include immediate information generated according to the emotional state of the speech information, or subject information different from the current subject. According to one embodiment, the predetermined subject may be a subject that changes the emotional state of the participant, or it may be a subject that gives the participant an antagonism. The subject may be stored in a database used to record information related to the participant. For example, if the speech information input to the communication terminal by the local user is related to the remote user of the current communication and the type of the emotion state of the speech information is anger, the content of the virtual speech may include the emotion of the local user And may include immediate information to persuade the local user to control it. For example, if the subject of the local user's speech information includes a subject that causes the remote user's despair (e.g., the age of the remote user), the content of the virtual speech may include a topic that is different from the subject .

The presentation attributes according to an embodiment may be obtained or generated by adjusting the previous presentation attributes of the virtual speech. Since the presentation attribute of the virtual speech includes the emotional state and / or emotional mode, adjusting the previous emotional attribute may include adjusting the emotional state and / or adjusting the presentation mode.

Adjusting the emotional state in accordance with an embodiment may include suppressing the emotional state and / or improving the emotional state. The suppression of the emotional state may include adjusting the positive type of emotional state to a neutral or negative type of emotional state, such as adjusting the happy emotional state to a mood or depressed emotional state, and / To a negative type of emotional state. In accordance with one embodiment, suppression of the emotional state may include adjusting the emotional state from high to low, such as from high to low levels of joy in the emotional state. According to one embodiment, the enhancement of the emotional state may include adjusting the emotional state of the negative type to an emotional state of a neutral or positive type, such as modulating the emotional state of the emotional state from mild or depressive to pleasurable, To an emotional state of the state. Improving the emotional state in accordance with an embodiment may include adjusting the emotional state from a low state to a high state, such as adjusting the degree of joy emotional state from low to high.

According to one embodiment, the presentation attribute of the virtual speech can be adjusted according to the content of the speech information input to the communication terminal. For example, if the content of the speech information includes an interesting keyword previously set in the status information of the virtual character, the emotional state of the virtual speech can be improved. As another example, if the content of the speech information includes a biased keyword preset in the status information, the previous emotion state of the virtual speech may be suppressed. For example, when the content of the speech information input to the communication terminal includes a positive sentence, the emotional state of the previous virtual speech can be improved. Here, a positive sentence can be a sentence that can be used to greet or answer a greeting. The emotional state of the virtual virtual speech can be improved when the content of the speech information input to the communication terminal includes greeting the private character, the virtual character. As another example, if the content of the speech information includes a statement, the emotional state of the previous virtual speech can be suppressed. The statement may be a sentence used to request or command. When the content of the speech input to the communication terminal includes a statement structure such as " lower the voice number ", the emotional state of the virtual speech of the virtual character can be suppressed.

According to one embodiment, the emotional state contained in the presentation attribute of the virtual speech may be determined by the state information obtained in step S220. For example, if the state information includes compassion towards the person who entered the speech information, the emotional state can be adjusted. For example, when the state information includes compassion toward the person who has input the speech information, the emotion state can be improved according to the setting of the state information. As another example, when the state information does not include compassion toward the person who entered the speech information, or when the state information includes compassion toward the other party of the person who has input the speech information, depending on the setting of the state information, Can be suppressed.

When the content of the speech information input to the communication terminal includes a predetermined interesting subject, the device 100 according to an embodiment can improve the emotional state. When the content of the speech information input to the communication terminal includes a subject causing a predetermined half-tone, the device 100 according to one embodiment can suppress the emotional state.

Interesting topics and antipatterns can be determined by personality variables, which determine the status information of each virtual character. For example, a personality variable may include variables that determine whether it is a preference / sensitivity topic. For example, a preference variable included in a personality variable may increase when a keyword for a predetermined interesting subject is acquired from the speech information, and may decrease when a keyword for the subject causing the predetermined antipathy is obtained from the speech information. For example, a preference variable included in a personality variable increases when a keyword for art (predetermined interesting topic) is acquired from speech information, and decreases when a keyword for terrorism (a subject causing a predetermined antipathy) is acquired from speech information can do.

According to one embodiment, when the person referred to in the speech information input to the communication terminal is one of the participants of the current call, the device 100 according to one embodiment can improve the emotional state of the virtual character.

The device 100 according to an exemplary embodiment may determine the status information using the emotion state acquired from the speech information input to the communication terminal. The device 100 according to an exemplary embodiment may determine the emotional state of the virtual character using the status information. For example, when the emotion state acquired from the speech information input to the communication terminal is abnormal or the abnormal emotion state continues for a predetermined period or more, the device 100 according to the embodiment can update the current state information. The fact that the emotion state acquired from the speech information is abnormal may mean a case where the emotion state of one of the call participants is determined to be abnormal or a case where two or more emotion states of the call participants are determined to be abnormal.

The device 100 according to an exemplary embodiment may determine the emotional state of a participant participating in a call through the speech information input to the communication terminal by the participant. For example, if the emotion state acquired from the participant's speech information is a negative type (e.g., anger type), the emotion state of the participant may be represented abnormally. As another example, if the emotional states acquired from two participants participating in the call are both negative (eg, one is anger type and the other is depressive), the emotional state of the two participants may be represented as bilateral abnormal have. As another example, if the emotion states acquired from the two participants participating in the call are positive and negative types, respectively (e.g., one is anger type and the other is a joy type), the emotion states of the two participants Can be represented as abnormal or mutually abnormal.

The device 100 according to an exemplary embodiment may determine whether the emotion state is normal using the emotion type as well as the duration of the predetermined emotion. For example, it is possible to determine whether the participant's emotional state is normal considering both the type of the emotional type, the intensity of the emotional type, and the duration of the emotional type.

When it is determined that the emotional state of one or more participants of the call participant is abnormal, the device 100 according to the embodiment obtains the virtual speech determined according to the content and expression attributes necessary for switching the emotional state of the participant to the normal state, can do. In addition, the device 100 according to one embodiment can determine whether the emotional state of the call participant changes according to the virtual speech provided by the device 100. [

The virtual speech obtained and output by the device 100 according to an embodiment may be determined according to the presentation attributes of the virtual speech and the virtual speech. The presentation attribute may include information on how to represent the virtual speech.

The presentation attribute according to one embodiment can be determined according to the presentation attribute obtained from the speech information input to the communication terminal. For example, when the frequency and degree of the attribute for the A local dialy are high as a result of analysis of the presentation attribute obtained from the speech information input to the communication terminal, the expression attribute of the virtual speech provided by the device 100 is also A It can include the expression attribute of a dialect.

The device 100 according to one embodiment can determine the presentation attribute of the virtual speech provided by the device by determining one of a plurality of preset presentation attributes. For example, when the predetermined family mode is selected, the device 100 according to an exemplary embodiment can obtain virtual speech according to the expression attributes according to the dialects and language habits used between the family members. As another example, in addition to the family mode, the predetermined plurality of presentation modes may include a business mode, a rest mode, and the like, but are not limited thereto.

FIG. 3 is a flow diagram of a method in which a device 100 according to one embodiment obtains and outputs a virtual speech.

Steps S330 and S340 correspond to steps S220 and S230 described above in FIG. 2, respectively, so that a detailed description will be omitted for the sake of simplicity.

In step S310, the device 100 according to one embodiment outputs a virtual speech query to obtain information about at least one participant.

For example, the device 100 according to one embodiment may output virtual speech querying a first participant's current schedule through virtual speech during a conversation between a first participant and a second participant. In another example, if a conversation is about a sport between a first participant and a second participant, the device 100 according to one embodiment may provide the first participant with content that asks whether the sport is a subject that the first participant is interested in And can output the included virtual speech.

In step S320, the device 100 according to one embodiment obtains speech information that includes additional speech information obtained via the querying virtual speech.

The device 100 according to an exemplary embodiment may acquire speech information provided by a participant corresponding to the virtual speech output in step S310. The device 100 according to an exemplary embodiment may obtain additional speech information, which is speech information provided by a participant corresponding to the virtual speech output in step S310. In addition, the device 100 according to an embodiment may acquire the speech information including the additional speech information through the communication terminal, and may analyze the additional speech information included in the speech information to obtain additional information. Additional information may include, but is not limited to, information about the participant.

In step S350, the device 100 according to one embodiment outputs the virtual speech obtained in step S340.

For example, the device 100 according to one embodiment may output the virtual speech obtained in step S340 in a voice form. As another example, the device 100 according to one embodiment may output the virtual speech obtained in step S340 in a character form.

The device 100 according to one embodiment may provide the virtual speech obtained in step S340 only to one of the two participants in the call. Or device 100 according to one embodiment may provide the virtual speech obtained in step S340 to both participants in the call. The device 100 according to an exemplary embodiment may output the virtual speech obtained in step S340 to a single participant or a plurality of participants through a speaker or the like.

The device 100 according to an exemplary embodiment may generate a virtual speech in a presentation format according to the state information obtained in S330, and then output virtual speech generated through the communication terminal. For example, the virtual speech may be output directly through the speaker of the local communication terminal. The virtual speech may be transmitted to the remote communication terminal through the telephone network after the encoding process is performed, and may be output by the remote communication terminal through the speaker of the remote communication terminal.

The device 100 according to an embodiment may be configured to delay output of speech information if i) content of speech information input via a communication terminal includes a predetermined sensitive keyword for a virtual character, or ii) , It is possible to output the speech information. The output command may be transmitted by the communication terminal or automatically transmitted by the communication terminal after a predetermined time interval. For example, in a case where the speech information input to the local communication terminal includes a predetermined sensitive keyword (as opposed to) that causes a dispute during communication, the virtual character of the local communication terminal transmits the speech information to the remote communication terminal Delay, change the subject, or suggest emotions to the local user. The device 100 in accordance with one embodiment may communicate with a local user or a remote user using virtual speech in a private mode while the transmission of speech information is delayed.

4 is a flow diagram of a method in which a device 100 according to one embodiment obtains virtual speech using speech information and status information.

In step S410, the device 100 according to one embodiment uses the speech information to obtain the content of the virtual speech.

The device 100 according to an exemplary embodiment may analyze the speech information obtained from the communication terminal to determine a subject, and may acquire content corresponding to the determined subject and situation. For example, if two participants are talking on a topic about football, the device 100 according to one embodiment may determine the content of a recent soccer article as the content of a virtual speech.

In step S420, the device 100 according to an embodiment determines the presentation method of the virtual speech using the status information.

The presentation method according to an exemplary embodiment may be a method in which a virtual speech is expressed by voice, such as a voice tone or a speech speed. The presentation method according to one embodiment may be determined according to the presentation property.

An expression attribute includes an expression style, an emotional state of a virtual character, a presentation mode, and the like, and can be determined according to state information. For example, the device 100 according to an exemplary embodiment may determine a presentation attribute corresponding to the status information, and determine a presentation method of the virtual speech according to the determined presentation attribute.

For example, the device 100 according to an exemplary embodiment may determine the presentation method of the virtual speech to express the content through the soft and slow tones when the emotional state between the participants participating in the call is excessively furious according to the status information .

In step S430, the device 100 according to the embodiment obtains the virtual speech in which the content obtained in step S410 is expressed by the expression determined in step S420.

For example, the device 100 according to one embodiment may obtain a virtual speech representing the content of the soccer article determined in step S410 according to the expression determined in step S420.

5 is a flow diagram of a method for a device 100 according to one embodiment to obtain virtual speech according to whether a keyword preset in speech information is included.

In step S510, the device 100 according to one embodiment determines whether the speech information includes a preset keyword.

For example, the device 100 in accordance with one embodiment may determine whether the speech information obtained via the communication terminal includes a keyword for the subject of the call participant avoidance. As another example, the device 100 according to one embodiment may determine whether the speech information obtained via the communication terminal includes a keyword for the talk participant's preferred topic.

In step S520, the device 100 according to one embodiment obtains a virtual speech including content for a subject different from the subject of the conversation, if the speech information includes the predetermined keyword.

For example, if the speech information acquired by the device 100 according to the embodiment through the communication terminal includes a keyword of a talk participant's avoiding topic, a virtual speech including content for a subject not related to the current topic Can be obtained and output.

6 is a flow diagram of a method for a device 100 according to one embodiment to obtain virtual speech based on emotion information.

In step S610, the device 100 according to one embodiment determines whether the emotion state of at least one participant is abnormal using emotion information.

The state information according to one embodiment may include emotion information. The emotion information according to one embodiment may include information on the type of emotion and information on the level of emotion. Types of emotions may include, but are not limited to, emotion states such as elation, anger, sorrow, depression and mildness.

For example, the emotion information may include information about the participant's emotional state and the angry state. The device 100 according to one embodiment may obtain emotion information through the speech information obtained through the communication terminal. The device 100 according to one embodiment can determine how much the emotion state of one of the plurality of emotion states is, through the speech information obtained through the communication terminal.

In step S620, the device 100 according to one embodiment obtains a virtual speech that includes content for a subject different from the subject of the conversation if the emotional state is determined to be abnormal.

For example, the device 100 according to one embodiment may determine that the emotional state is abnormal when the participant confirms that he or she is maintaining a predetermined emotional state for a predetermined period or more. In addition, the device 100 according to one embodiment may obtain virtual speech including content for a topic different from the current conversation subject, if the emotional state of the participant in the call is determined to be abnormal. For example, device 100 according to one embodiment may obtain and output virtual speech for a different sport than the current conversation subject, if it has confirmed that the participant's anxious emotional state is maintained for more than one minute.

7 is a diagram illustrating an example in which the device 100 according to an embodiment acquires virtual speech using a keyword included in speech information.

According to one embodiment, when the content of the speech information input to the communication terminal includes a predetermined sensitive keyword in the status information of the virtual character, the contents of the virtual speech may include prompt information ) Or topic information. The predetermined sensitive keyword according to an exemplary embodiment may be stored in the sensitive keyword field of the personality variable of the status information. If the content of the call according to one embodiment is associated with a predetermined sensitive keyword, the call may be negatively affected. For example, if the content of the speech information includes a keyword of terrorism, and the keyword of terrorism is one of the sensitive keywords of the status information of the virtual character, the content of the virtual speech to be generated is " , Or may include subject information that is different from the current subject, such as a " sports " subject, directly.

For example, in the case where the speech information transmitted from the first communication terminal 110 to the second communication terminal 120 includes the content " Terror situation is 710 ", the device 100 according to one embodiment To the first communication terminal 110 and / or the second communication terminal 120, the virtual speech including the content " Replace the subject " (720).

As another example, if the speech information transmitted from the first communication terminal 110 to the second communication terminal 120 includes content such as " Terror situation " 710, (730) to the first communication terminal (110) and / or the second communication terminal (120).

8 is a diagram illustrating an example in which the device 100 according to an embodiment acquires virtual speech using a keyword included in speech information.

According to one embodiment, the predetermined type may include a numeric type and / or a temporal type. If the content of the speech information input to the communication terminal includes a numeric type keyword, the content of the virtual speech may include immediate information related to the contact list update and numeric conversion. If the content of the speech information includes a keyword of a time type, the content of the virtual speech may include immediate information related to schedule conflict, a time reminder, a reminder and a time difference or a travel reminder. For example, when the content of the speech information includes the keyword " 7:00 am tomorrow ", the communication terminal can search for the schedule information of the user tomorrow at 7:00 am. If duplicates are detected, the content of the generated virtual speech may contain immediate information that it is a conflict.

For example, if the speech information transmitted from the first communication terminal 110 to the second communication terminal 120 includes contents such as " Let's have a cafe at 7:00 am tomorrow morning? (830) The device 100 analyzes the schedule for the user of the first communication terminal and, according to the analysis result, transmits the virtual speech including the content " There is already the meeting schedule at 7 am tomorrow " (820) To the control unit 810.

A device 100 according to one embodiment may be located within a first communication terminal 110 and may be located within a second communication terminal 120 and may include a first communication terminal 110 and a second communication terminal 120 As shown in FIG.

9 is a block diagram illustrating a configuration of a device 100 for providing information in accordance with one embodiment.

9, a device 100 according to one embodiment includes a receiving unit 910, a control unit 920, and an output unit 930. Only the components associated with this embodiment are shown in the device 100 shown in Fig. Therefore, it will be understood by those skilled in the art that other general-purpose components other than the components shown in FIG. 9 may be further included.

A receiving unit 910 according to an embodiment acquires speech information provided by at least one participant participating in a conversation through a communication terminal using a communication terminal.

Speech information according to one embodiment may include information obtained via a voice signal. For example, the speech information may include information obtained from a voice signal of a communication terminal user received via the communication terminal. As another example, the speech information may include keyword information obtained through analysis of the speech signal of the communication terminal user. As another example, the speech information may include voice information obtained through analysis of the speech signal of the communication terminal user.

The receiving unit 910 according to an embodiment may acquire the speech information from one or more communication terminals. For example, the receiving unit 910 can acquire the speech information from the first communication terminal and / or the second communication terminal.

The communication terminal according to an exemplary embodiment may include a microphone and a speaker. In addition, the communication terminal according to one embodiment may include, but is not limited to, a smart phone, a tablet computer, a personal digital assistant, a laptop computer, a desktop computer, and the like.

The controller 920 according to an exemplary embodiment may analyze speech signals obtained through the communication terminal to obtain speech information. The controller 920 may include a receiver 910 according to an exemplary embodiment of the present invention. For example, the control unit may use the communication terminal to obtain speech information provided by at least one participant participating in the conversation.

Alternatively, the communication terminal according to an exemplary embodiment may acquire a voice signal and transmit the result of analyzing the acquired voice signal to the device 100. The control unit 920 can acquire the speech information using the speech signal analysis result received from the communication terminal.

The communication terminal or controller 920 can receive the voice signal. For example, the user's voice signal can be received through the microphone of the communication terminal. For example, when the user's voice signal of the first communication terminal is received through the microphone of the first communication terminal, the receiving unit 910 can acquire the voice signal of the first communication terminal user through the first communication terminal . As another example, when the user's voice signal of the second communication terminal is received through the microphone of the second communication terminal, the receiving unit 910 can acquire the voice signal of the user of the second communication terminal through the second communication terminal. As another example, when the voice signal of the user of the local communication terminal is received through the microphone of the local communication terminal, the device 100 can acquire the voice signal of the user of the local communication terminal through the local communication terminal. As another example, when the user's voice signal of the remote communication terminal is received through the microphone of the remote communication terminal, the receiving unit 910 can acquire the voice signal of the user of the remote communication terminal through the remote communication terminal. The receiver 910 may be located within the local communication terminal, may be located within the remote communication terminal, or may be external to the local communication terminal and the remote communication terminal.

The controller 920 according to an embodiment obtains state information indicating the current state of at least one participant using the speech information and / or the pre-stored data.

The status information according to one embodiment includes emotion information indicating at least one participant's current emotional state, at least one participant's personal information, topic information about the conversation, at least one Schedule information of a participant, and personality information of at least one participant.

The emotion information according to one embodiment may include information on the type of emotion and information on the level of emotion. Types of emotions may include, but are not limited to, emotion states such as elation, anger, sorrow, depression and mildness.

The controller 920 according to an exemplary embodiment may obtain status information using the speech information.

For example, when the first communication terminal obtains a voice signal of " There is a cold wind today ", the controller 920, through the speech information obtained from the voice signal acquired by the first communication terminal, The user can obtain information about the current health status of the user. As another example, when the remote communication terminal acquires a voice signal "I fought a friend yesterday", the controller 920 receives the voice signal acquired by the remote communication terminal through the wireless connection, State information indicating that the emotional state may be inconvenient.

The controller 920 according to an exemplary embodiment may acquire status information using pre-stored data.

For example, the age information of the user can be obtained through the information of the date of birth of the user. The pre-stored data may be stored in the controller 920, stored in the communication terminal, or stored in the server.

The control unit 920 according to an exemplary embodiment may acquire status information using the speech information and pre-stored data acquired by the receiving unit 910.

For example, if data for users A and B is stored beforehand, and user A refers to user B as father, the status information about the relationship between users A and B is as follows: The control unit 920 can acquire the control information.

The obtained status information according to an embodiment may be stored in a memory of the communication terminal, a memory or a server (not shown) of the controller 920, or the like. For example, the status information may be stored in a local communication terminal and / or a telecommunications terminal. As another example, the status information may be stored in the internal or external memory of the control unit 920, or in a server connected to the control unit 920. [

When the status information is stored in the server, the communication terminal or the control unit 920 can obtain the status information from the server through the wired / wireless connection.

The control unit 920 according to an embodiment obtains or determines a virtual speech to be provided to at least one participant based on the speech information and the status information. For example, the device 100 according to one embodiment may determine one of a plurality of virtual speech to be provided to at least one participant, and may generate virtual speech to be provided to at least one participant.

The virtual speech according to an exemplary embodiment may be determined according to the speech information obtained by the receiving unit 910 and the state information acquired by the controller 920.

According to one embodiment, the controller 920 may provide virtual speech through the virtual character. According to an embodiment, the controller 920 can determine the virtual speech based on the sex, personality, voice, etc. of the virtual character.

The controller 920 according to an exemplary embodiment may determine the character, tone, etc. of the virtual character according to the status information, and the character of the virtual character and the like may be changed according to the changed status information.

The state information according to one embodiment may include a personality variable and / or a state variable. For example, the state information according to one embodiment may be determined according to a personality variable and / or a state variable.

The personality variable may be used to indicate the general tendency of the virtual character corresponding to the speech information input to the communication terminal, and may be changed through communication between the user of the communication terminal and another person. For example, personality variables can be classified as preference / sensitivity subject, preference / sensitivity keyword, sympathy, accent, adaptability, alterness, curiosity, opposition, eloquence, idioms, But is not limited to, at least one of loquacity, peculiarities, responsiveness, sentimentality, and sleep time. For example, a preference / sensitivity topic can be used to determine whether a virtual character is actively involved or rarely participating in a subject. As another example, the preference / sensitivity keywords may be used to indicate whether the virtual character is interesting (such as sports) or not (such as terrorism) for the subject of interest. As another example, compassion can be used for a person, object, or object when the virtual character indicates whether to give a positive comment or to give a negative comment. As another example, accents may be used to indicate the possible accent types and degrees of virtual characters. In another example, adaptability can be used to indicate the degree of long term variation in the character of the virtual character. As another example, the arousal level may be used to indicate the degree of sensitivity of the virtual character to input speech information. As another example, curiosity can be used when the virtual character indicates a questioning motion. As another example, the opposition may be used to indicate an action in which a virtual character performs an instruction. As another example, persuasive power can be used to indicate the tendency to use the virtual character's fluent and decorated sentences. As another example, an idiom can be used to represent a phrase or sentence that is often used by a virtual character. As another example, the discursiveness can be used to indicate the degree to which a virtual character speaks a lot. As another example, the substrate may be used to indicate a particular reaction mode of the virtual character for a particular subject. In another example, responsiveness can be used to indicate the aggressiveness of a virtual character's request or inquiry. As another example, emotion may be used to indicate a tendency for a virtual character to develop strong mood. As another example, sleep time can be used to indicate a tendency for a virtual character to have low responsiveness during day time.

The state variable according to an embodiment may be used to determine a behavior characteristic of a virtual character. The state variable may be determined according to the previous state variable, the speech information input through the communication terminal, the personality variable, and the like. For example, aggressiveness can indicate how aggressively a virtual character speaks about an issue. High aggressiveness can often indicate spontaneously speaking, long and fast. In another example, the emotional state may indicate the type of emotion (including joy and depression) and intensity that the virtual character represents through speech. In another example, the speech scheme represents the current speech of the virtual character. Speech schemes can include the degree and frequency of adaptation to a particular type of dialect, the degree of formality and familiarity, and the request for a particular voice. As another example, improvisation can indicate how aggressively a virtual character starts a conversation.

The output unit 930 according to an embodiment outputs the virtual speech obtained by the controller 920. [

For example, the output unit 930 according to an exemplary embodiment may output the virtual speech obtained by the controller 920 in a voice form. As another example, the output unit 930 according to an exemplary embodiment may output the virtual speech obtained in the controller 920 in a character form.

The output 930 according to one embodiment may provide virtual speech to only one of the two participants in the call. Or an output unit 930 according to an exemplary embodiment may provide the virtual speech obtained by the controller 920 to both participants in a call. The output unit 930 according to an exemplary embodiment may output virtual speech obtained by the controller 920 to a single or a plurality of participants through a speaker or the like.

The output unit 930 according to an exemplary embodiment may output the virtual speech of the presentation method according to the state information acquired by the controller 920. [ For example, the virtual speech may be output directly through the speaker of the local communication terminal. The virtual speech may be transmitted to the remote communication terminal through the telephone network after the encoding process is performed, and may be output by the remote communication terminal through the speaker of the remote communication terminal.

10 is a block diagram illustrating a configuration of a receiving unit 910 according to an embodiment.

Referring to FIG. 10, a receiving unit 910 according to an embodiment includes a speech parser 1010 and a state information obtaining unit 1020. Only the components related to the present embodiment are shown in the receiving unit 910 shown in Fig. Therefore, it will be understood by those skilled in the art that other general-purpose components other than the components shown in FIG. 10 may be further included.

The speech parser 1010 according to an exemplary embodiment may extract speech information by analyzing a speech signal input to the device 100. [

State information obtaining unit 1020 according to an embodiment of the present invention.

The control unit 920 may generate virtual speech having presentation attributes according to the speech information and the status information. The output unit 930 can output the virtual speech. The speech information and the virtual speech may include a presentation attribute used to represent non-content information, such as the emotion and the phrasing style of the speech. The presentation attribute may include an emotional state and / or a presentation mode. The speech information may include other information such as a person's voice characteristics that is input as audio.

The emotional state of an expression attribute may include, but is not limited to, gladness, anguish, sadness, depression, and mildness. The presentation mode of the presentation attribute may include at least one of an expression style, an accent type, a dialect frequency, a dialect degree, a dial tone, a situation mode, and a background sound.

The speech parser 1010 according to one embodiment may analyze the audio input to the communication terminal and extract speech information. The extracted speech information may include content information (e.g., subject, keyword), presentation mode information (e.g. accent), and emotion state information (e.g., emotional state of the joy of the person entering the audio) Do not. The speech parser 1010 also includes a speech recognition module (not shown) for recognizing content information from the audio input to the electronic device and an expression attribute recognition module (not shown) for recognizing the presentation attribute information from the audio .

The state information obtaining unit 1020 according to an embodiment may be used to store state information. The status information may include information used to indicate the behavior and nature of the virtual character. The status information can be updated in accordance with the speech information obtained by the speech parser 1010 or the like. For each participant who actually participates in the communication, the virtual character can participate in the call using virtual speech. The generation of the virtual speech of the virtual character according to one embodiment may be determined according to the state information stored in the state information obtaining unit 1020. [

According to one embodiment, the state information may include personality variables and state variables. The personality variable can be used to indicate the disposition of the virtual character corresponding to the speech information input to the communication terminal and can be changed through a long time communication between the user of the communication terminal and another person. For example, personality variables include preference / sensitivity topics, preference / sensitivity keywords, sympathy, accent, adaptability, alterness, curiosity, opposition, eloquence, idioms, but is not limited to, at least one of loquacity, peculiarities, responsiveness, sentimentality, and sleeping time. The state variable is used to indicate the behavior characteristic of the virtual character, and may be changed depending on the previous state variable, the speech information input through the communication terminal, the personality variable described above, and the like. For example, the state variable may include, but is not limited to, at least one of activeness, emotional state, expression mode, and spontaneity. Personality and status variables are either set by default or can be obtained by the user. For example, the user of the device 100 may send a copy / update command to the device 100 to copy the status information of the user's preferred virtual character, and may transmit status information of the status information acquisition unit of the device 100 Can be updated.

Under the control of the controller 920, the state information obtaining unit 1020 according to an exemplary embodiment may store the state information stored in the state information obtaining unit 1020 and the stored state information according to the speech information obtained by the speech parser 1010 Can be updated. The control unit 920 may update the personality variable of the status information according to at least one of the speech information obtained by the speech parser 310 and the update command from the user. In addition, the control unit 920 may update the state variable of the state information according to at least one of the personality variable of the state information, the speech information obtained by the speech parser 1010, and the update instruction from the user.

The controller 920 according to an exemplary embodiment may update the personality variable according to an update command from the user. For example, the control unit 920 can copy the personality variable of the user's preferred virtual character and update the personality variable of the status information acquisition unit 1020 by receiving a copy / update command from the user of the communication terminal . In addition, the controller 920 may update the personality variable according to the speech information obtained by the speech parser 1010. For example, the control unit 920 may perform analysis and statistics of the content of the speech information, and may renew or enrich the idioms in the personality variable by determining high frequency idioms in the content of the speech information.

The controller 920 according to an exemplary embodiment may update the state variable according to the relationship between the personality variable and the state variable. For example, in personality variables, arousal, curiosity, preference motives, preference keywords, compassion, talk and responsiveness can positively affect the aggressiveness of state variables. For example, it may be more aggressive if it is high or strong in arousal, curiosity, preference theme, preference keyword, compassion, talk and response. In personality variables, sleep time can negatively affect aggressiveness. For example, aggressiveness may be low during sleep time. The peculiarities of personality variables can affect positively or negatively depending on the situation.

The controller 920 according to an exemplary embodiment may update the state variable according to the speech information obtained by the speech parser 1010. [ For example, if a user entering the above-described speech information frequently communicates with a virtual character, the aggressiveness of the state variable may increase. If the user entering the above-mentioned speech information rarely communicates with the virtual character or places an emphasis on the other, the aggressiveness of the state variable may be reduced. In addition, personality variable data and state variables can be directly determined by the user. For example, the aggressiveness can be adjusted to a specific value according to the input from the user.

11 is a block diagram illustrating a configuration of a control unit 920 according to an embodiment.

11, the controller 920 includes a planner 1110 and a speech synthesizer 1120. [ Only the components related to the present embodiment are shown in the control unit 920 shown in Fig. Therefore, it will be understood by those skilled in the art that other general-purpose components other than the components shown in FIG. 11 may be further included.

The controller 920 may include a planner 1110 and a speech synthesizer 1120. [ 11 shows a diagram of the structure of the controller 920 according to an embodiment. As shown in FIG. 11, the controller 920 may include a planner 1110 and a speech synthesizer 1120. Planner 1110 determines i) state information and content of virtual speech to be generated in accordance with the state information stored in state information obtaining unit 1020 and the speech information obtained by speech parser 1010, or ii) To generate a text descriptor according to the content, or iii) to generate a presentation attribute descriptor according to the presentation attribute. The speech synthesizer 1120 can generate the virtual speech according to the text descriptor and the presentation attribute descriptor. The planner 1110 analyzes the speech information obtained by the speech parser 1010 and recognizes information such as a person, a subject, a keyword, a sentence structure, and the like according to the presentation attribute of the content of the virtual speech and the pitch information And iii) determine the contents of the virtual speech according to the speech information.

The content of the virtual speech generated according to the determination of the planner 1110 according to an embodiment may include voluntary content and interactive content. Voluntary content may include, but is not limited to, at least one of greeting, an instruction to a user, an event reminder, a comment, and a question. The interactive content may include an answer to at least one of a greeting, an instruction to the user, an event reminder, a comment and a question. For example, if the identity of the user input audio is recognized according to the voice information of the speech information described above (e.g., the identity is recognized in the user information database), the virtual speech generated according to the decision of the planner 1110 Voluntary content of the greeting may include responding to or greeting the greeting, and the content of the greeting may include the name of the user. If an interesting subject is detected in the above-mentioned speech information, the mutual content of the virtual speech generated according to the determination of the planner 1110 may include a comment on the subject.

12 is a block diagram showing a configuration of a speech synthesizer 1120 according to an embodiment.

12, the speech synthesis unit 1120 includes a front end processing module 1210, a front end prosody processing module 1220, and a back-end waveform synthesizer 1230. The front- Only the components related to the present embodiment are shown in the speech synthesis unit 1120 shown in Fig. Therefore, it will be understood by those skilled in the art that other general-purpose components other than the components shown in FIG. 12 may be further included.

The speech synthesis unit 1120 according to an embodiment includes a front end text processing module 1210 for generating a phonetic label according to a text descriptor, ii) a front end text processing module 1210 for generating a phonetic label according to a prosody modifying descriptor a front-end prosodic processing module 1220 for generating a prosody modulation descriptor, and iii) a back-end waveform synthesizer 1230 for generating virtual speech in accordance with phonetic symbols and prosody modulation descriptors. The phonetic symbol can be used to indicate features such as articulation intonation of each unit of speech to be generated. The rhythm scheme modulation symbols can be used to express features such as rhythm emotion, rhyme scheme of words and sentences in the speech to be generated.

According to one embodiment, the controller 920 may determine one of the audio input to the device 100 or the audio input to the device 100 to be superimposed on the virtual speech, as the virtual speech to be output. For example, the control unit 920 can select the audio input by the user of the communication terminal as the audio to be output. In this case, the call participant may feel that the virtual character is not participating in the conversation. As another example, the control unit 920 may select to output speech in which the audio input to the device 100 overlaps the virtual speech. In this case, the call participant may feel that the virtual character is participating in the conversation.

13 is a block diagram showing a configuration of the control unit 920 according to an embodiment.

13 is a diagram illustrating a controller 920 that controls the device 100 to output virtual speech according to one embodiment. 13, the first control unit 1362 may be configured to receive a virtual speech 1370 (such as virtual speech that interacts with a local user) and a remote microphone 1311 (E.g., a local user's speaker) over local mixer 1342 via local mixer 1342 and local mixer 1342. [ The second control unit 1361 receives the audio signal 1353 input via the microphone 1312 by the local user through the remote mixer 1341 It is possible to control the device 100 so as to superimpose and output through the remote output unit 1321 (speaker of the remote user). The second control unit 1361 can output the audio 1353 input by the local user to the output of the remote output unit 1321. [ The first control unit 1362 can output the audio 1354 input by the user of the remote terminal to the local output unit 1322. The control unit 920 may output the virtual speech 1370 to the local output unit 1322 or output the virtual speech 1370 to the remote output unit 1321. [ In the process described above, the first control unit 1362 may receive a non-voice input of the user, such as a keyboard or mouse 1330. [

The control unit 920 controls the local output unit 1322 or the remote output unit 1321 to delay the output of the audio output from the device 100. In this case, can do. The control unit 920 according to an exemplary embodiment may control the local output unit 1322 or the remote output unit 1321 to output audio when an output command is received. The control unit 920 according to an exemplary embodiment may delay the output of audio input to the device 100 during communication. During the delay, the controller 920 according to one embodiment may output virtual speech to the singular or plural participants of the communication in private mode. Delayed output can result in the cancellation of one sentence or one paragraph in the communication.

FIG. 14 is a block diagram showing the configuration of the control unit 920 according to an embodiment.

Referring to FIG. 14, a controller 920 according to an embodiment includes a determination unit 1420, a first spatial filter 1430, and a second spatial filter 1440. Only the components related to the present embodiment are shown in the control unit 920 shown in Fig. Therefore, it will be understood by those skilled in the art that other general-purpose components other than the components shown in FIG. 14 may be further included.

If the controller 920 according to an embodiment has determined the superposition of virtual speech and audio as an output, the device 100 may be used to spatially filter audio and virtual speech. In addition, FIG. 14 shows a diagram showing that the control unit 920 filters audio according to an embodiment. As shown in FIG. 14, under the control of the control unit 920, the determination unit 1420 can select one or both of the audio 1411 and the virtual speech 1412 to be output. When the control unit 920 determines the superposition of the audio and the virtual speech as the output, the determination unit 1420 simultaneously selects the audio 1411 and the virtual speech 1412 to be outputted, and performs spatial filtering pseudo-spatial filters).

15 is a block diagram showing a configuration of a database 1500 according to an embodiment.

Referring to FIG. 15, a database 1500 according to an embodiment includes a human database 1510, an object database 1520, and a record database 1530. Only the components related to the present embodiment are shown in the database 1500 shown in Fig. Therefore, it will be understood by those skilled in the art that other general-purpose components other than the components shown in FIG. 15 may be further included.

According to one embodiment, the device 100 may include a database that stores knowledge information. The knowledge information may be information representing a person and an object. Here, the control unit 920 generates virtual speech having a presentation attribute according to i) speech information obtained by the speech parser, ii) status information of the status information obtaining unit 1020, and iii) knowledge information stored in the database . ≪ / RTI > For example, the speech information may include a subject stored in the database, and the control unit 920 may generate a virtual speech that uses subject information and status information related to the subject stored in the database to comment on the subject .

 15 is a diagram showing a structure of a database 1500 according to an embodiment. As shown in FIG. 15, the database 1500 includes a person database 1510 for storing information about a person, an object database 1520 for storing phonetic symbol information and common sense information, information on subjects and events, And a memorandum database 1530 for storing information of objects.

Objects stored and recorded in the person database 1510 according to one embodiment include: i) a user of the electronic device, ii) the user's contacts (e.g., contacts in the contact list), and iii) other connections A co-worker, etc.). The person database 1510 may comprehensively store related data of the objects described above. Related data include the person's name, gender, age, etc.; Information of social relationships to determine the relevance of an object to another object; And social relationship information for subsequent data management (e.g., for a predetermined period of time after communication), and information related to the origin of the person's information. The information in the above-described person database 1510 can be obtained through user input, automatic contact list search, automatic online search, and the like.

The information stored in the object database 1520 according to an exemplary embodiment may include at least one of a phonetic symbol for a speech parser and common knowledge information for knowledge search. The information may include keywords (and homonyms of keywords), commonly known information (e.g., common vocabulary with commonly known persons and places), phonetic symbols, and sources of such words. The information of the object database 1520 may be at least one of user input, public dictionary, and automatic online search.

The historical database 1530 according to one embodiment may store non-common knowledge of objects that are not of a person. In addition to information about objects, information about events, and information about subjects, the history database 1530 can store information related to the sources of the above-described information for subsequent database management. The history database 1530 may be obtained from user input, user calendar (log), analysis.

16 is a block diagram showing a configuration of a database 1500 according to an embodiment.

The database 1500 according to one embodiment may operate in connection with an external database 1610. [

16 shows the relationship between the person database 1510, the object database 1520, and the record database 1530 of the database 1500 according to an embodiment. 10, the person database 1510 includes data of a plurality of contacts of a user contact list stored in an external database 1610, such as a name, a voice characteristic, a social relationship, an age, a telephone number and the like. Commonly known common sense data representing some of the data of the person database 1510 according to one embodiment may be stored in the object database 1520 according to one embodiment. For example, the person database 1520 according to one embodiment may include data representing an item called vodka purchased by a person named Hong Gil Dong. In FIG. 16, the recording database 1530 according to one embodiment may include information such as an event (e.g., Mrs. Lee and Mrs Ho talk about the vodka price), information about the topic (e.g., have. Referring to FIG. 16, if the remote participant of the call is Hong Kil-dong, the controller 920 may generate a virtual speech asking for the wife of Young-hee, who is a wife of Hogil-dong. The generated virtual speech according to one embodiment may include a topic related to the theme, such as a board or a Seoul.

According to one embodiment, the person's information stored in the person database 1510 may include information of a person's speech / voice characteristics. The device 100 may further include a speaker recognizer (not shown) for confirming the identity of a person associated with the audio input to the device 100 according to the voice characteristic information. For example, the device 100 in accordance with one embodiment may extract information about the voice characteristics of a remote user of the communication and may search for a person database 1510 in which a voice feature entry matching the extracted information is stored Thereby identifying the remote user.

The device 100 according to an exemplary embodiment may further include a pattern matching unit (not shown) for extracting information from a sentence matched with a stored syntax pattern according to information stored in a dictionary database. The stored grammatical pattern is not limited as long as it can include a sentence structure of a question, a sentence structure of a statement, and a sentence structure of a statement.

The controller 920 may be used to update the knowledge information of the database 1500 according to one embodiment. In particular, the controller 920 can update the knowledge information actively or manually by at least one of an online search, a question, an automatic interference, a blank filling, an uncertain field matching, a new field discovery, a new value discovery, For example, the control unit 920 may periodically search an uncertain region or an empty value of the database 1500 from the speech information obtained through the device 100, and may fill in an uncertain or emptied value with a predetermined update method And can update the information stored in the database. As another example, the control unit 920 may continue to monitor keywords, main topics, and correctly configured sentences in the communication to gather various knowledge information in the database 1500. [

The control unit 920 according to an exemplary embodiment may perform a management operation on data of the database 1500 after the communication is terminated. For example, if the matching of the voice characteristics of the ranged user to the voice characteristics of the person stored in the person database 1510 is not completed during the communication, even after the communication, the identification information of the person corresponding to the voice feature is found, The control unit 920 may operate to match the voice characteristics of the far user and the voice features of the person database 1510 until all of the information of the voice features in the base 1510 are compared.

17 is a diagram showing an example of emotion information according to an embodiment.

In accordance with an embodiment, Fig. 17 shows the change of variables related to the aggressiveness of the state information. When the virtual character is enabled, the aggressiveness of the state information is changed from a disabled state 1710 (e.g., an aggressive variable value corresponding to 0) to a passive state 1720 (e.g., The corresponding aggressive variable value). Aggressiveness can be changed to an active state 1730 (eg, an aggression variable value corresponding to 2) if the user activates a virtual character such as greeting a virtual character. If the user frequently communicates with the virtual character, the aggressiveness can be changed to a hyperactive state 1740 (e.g., an aggressive variable value corresponding to 3). If the user's attention shifts to the other side, such as when the user does not frequently communicate with the virtual character, the aggressiveness may change to the active state 1730. [ If the user continues to ignore the virtual character or directly commands the virtual character such as " quietly ", the aggressiveness may change to passive state 1720. [ If the user continues to ignore the virtual character or if it no longer communicates with the virtual character, the aggressiveness may change to the inactive state (1710).

18 is a diagram illustrating an example in which the server 1800 provides information according to an embodiment.

The operations performed by the device 100 described above in FIGS. 1-17 may be implemented by the server 1800.

The currency may be performed on two humans, or on three humans. For example, two human conversations can be performed using the first communication terminal 110 and the second communication terminal 120, and in other examples, the first communication terminal 110, the second communication terminal 120, And the server 1800 can be used to perform a three-party call.

2 When a human call is performed, tension, inconvenience, and the like may be caused. However, when three human calls are performed, relative tension and inconvenience may be solved. For example, according to one embodiment, when a server 1800 participates in a call as a third party during a two-person call through the first communication terminal 110 and the second communication terminal 120, The atmosphere of a closed-and-tit-for-tat two-person call can be transformed into a calm atmosphere. The third participant participating in the call by server 1800 according to one embodiment may be a virtual character. For example, the voice signal transmitted to the first communication terminal or the second communication terminal by the server 1800 may be the virtual speech of the virtual character.

The server 1800 may be implemented as a separate server from the first communication terminal 110 or the second communication terminal 120 and may be implemented as a server between the first communication terminal 110 and the second communication terminal 120 It can intervene in communication.

The server 1800 may be arranged within the first communication terminal 110 to intervene in communication between the first communication terminal 110 and the second communication terminal 120 according to one embodiment.

According to one embodiment, the server 1800 may be arranged within the second communication terminal 120 to intervene in communication between the first communication terminal 110 and the second communication terminal 120. [

The server 1800 according to one embodiment may include a receiving unit 910, a control unit 920, and an output unit 930 shown in FIG.

The server 1800 according to an exemplary embodiment may acquire, via the communication terminal, speech information provided by at least one participant participating in a conversation using the communication terminal.

Speech information according to one embodiment may include information obtained via a voice signal. For example, the speech information may include information obtained from a voice signal of a communication terminal user received via the communication terminal. As another example, the speech information may include keyword information obtained through analysis of the speech signal of the communication terminal user. As another example, the speech information may include voice information obtained through analysis of the speech signal of the communication terminal user. In another example, the speech information may include speech information of at least one participant that the communication terminal acquires.

The server 1800 according to one embodiment may obtain status information indicating the current status of at least one participant using speech information and / or pre-stored data.

The status information according to one embodiment includes emotion information indicating the current emotional state of at least one participant and / or virtual character, personal information of at least one participant and / or virtual character, At least one of topic information, schedule information of at least one participant, and personality information of at least one participant and / or a virtual character.

The server 1800 according to one embodiment may obtain or determine the virtual speech to be provided to the at least one participant based on the speech information and / or the status information. For example, the server 1800 according to one embodiment may determine one of the plurality of virtual speeches as virtual speech to be provided to the participant, and may generate virtual speech to be provided to the participant.

The server 1800 according to one embodiment may output the obtained virtual speech.

For example, the server 1800 according to one embodiment may output the obtained virtual speech in a voice form. As another example, the server 1800 according to one embodiment may output the obtained virtual speech in a character form.

The server 1800 according to one embodiment may provide the acquired virtual speech to only one of the two participants in the call. Or the server 1800 according to one embodiment may provide the acquired virtual speech to both participants in the call. The server 1800 according to an exemplary embodiment may output virtual speech obtained to a single participant or a plurality of participants through a speaker or the like.

19 is a diagram showing an example in which the communication terminal 1900 according to an embodiment provides information.

The operations performed by the device 100 described above in FIGS. 1-17 may be implemented by the communication terminal 1900.

For example, the communication terminal 1900 according to one embodiment may be configured to perform the functions of the first communication terminal 110 disclosed in FIG. 1, the functions of the device 100 described in FIGS. 1 through 17, 1 communication terminal 110 according to the present invention.

As another example, the communication terminal 1900 according to one embodiment may correspond to the first communication terminal 110 described above in Fig.

The communication terminal 1900 according to an exemplary embodiment may include a receiving unit 910, a control unit 920, and an output unit 930 shown in FIG.

The communication terminal 1900 according to an exemplary embodiment may use the terminal to acquire speech information provided by at least one participant participating in a conversation.

Speech information according to one embodiment may include information obtained via a voice signal. For example, the speech information may include information obtained from a speech signal of a user of the communication terminal 1900 received via the communication terminal 1900. As another example, the speech information may include keyword information obtained through analysis of the speech signal of the user of the communication terminal 1900. As another example, the speech information may include voice information obtained through analysis of the speech signal of the user of communication terminal 1900. As another example, the speech information may include speech information of at least one participant that the communication terminal 1900 acquires.

The communication terminal 1900 according to one embodiment may obtain status information indicating the current status of at least one participant using speech information and / or pre-stored data.

The status information according to one embodiment includes emotion information indicating the current emotional state of at least one participant and / or virtual character, personal information of at least one participant and / or virtual character, At least one of topic information, schedule information of at least one participant, and personality information of at least one participant and / or a virtual character.

The communication terminal 1900 according to one embodiment may obtain or determine the virtual speech to be provided to at least one participant based on the speech information and / or the status information. For example, the communication terminal 1900 according to one embodiment may determine one of a plurality of virtual speeches as virtual speech to be provided to the participant, and generate virtual speech to be provided to the participant.

The communication terminal 1900 according to one embodiment can output the obtained virtual speech.

For example, the communication terminal 1900 according to one embodiment can output the obtained virtual speech in a voice form. As another example, the communication terminal 1900 according to one embodiment may output the obtained virtual speech in a character form.

The communication terminal 1900 according to an exemplary embodiment may provide the acquired virtual speech to only one participant in the call. Or the communication terminal 1900 according to one embodiment may provide the acquired virtual speech to both participants in the call. The communication terminal 1900 according to an exemplary embodiment may output virtual speech obtained to a single participant or a plurality of participants through a speaker or the like.

One embodiment may also be implemented in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium can include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

Claims (20)

In the information providing method,
Acquiring, through the communication terminal, speech information provided by at least one participant participating in a conversation using a communication terminal;
Obtaining status information indicating the current status of the at least one participant using the speech information and / or pre-stored data;
Determining a virtual speech to be provided to the at least one participant based on the speech information and the status information; And
And outputting the determined virtual speech. The method according to claim 1,
The step of acquiring the speech information through the communication terminal
Outputting a virtual speech query to obtain information about the at least one participant; And
And obtaining the speech information including the additional speech information obtained through the inquiring virtual speech. The method according to claim 1,
Wherein the status information comprises emotion information indicating a current emotional state of the at least one participant, personal information of the at least one participant, topic information of the conversation, And at least one of schedule information of the participant and personality information of the at least one participant. The method according to claim 1,
Wherein the speech information includes voice information of the at least one participant acquired by the communication terminal. The method according to claim 1,
The step of determining the virtual speech
Acquiring contents of the virtual speech using the speech information;
Determining a presentation method of the virtual speech using the status information; And
And obtaining the virtual speech in which the obtained content is represented by the determined presentation method. The method according to claim 1,
The step of determining the virtual speech
Determining whether the speech information includes a predetermined keyword; And
And when the speech information includes the preset keyword, obtaining the virtual speech including content for a subject different from the subject of the conversation. The method according to claim 1,
Wherein the status information includes emotion information of the at least one participant,
The step of determining the virtual speech
And the virtual speech of the content and the presentation method determined according to the emotion information is obtained. 8. The method of claim 7,
The emotion information
Information about the type of emotion and information about the level of emotion. 8. The method of claim 7,
The step of determining the virtual speech
Determining whether the emotion state of the at least one participant is abnormal using the emotion information; And
And if the emotional state is determined to be abnormal, obtaining the virtual speech including content for a subject different from the subject of the conversation. 10. The method of claim 9,
Wherein the step of determining whether the emotional state is abnormal
Determining whether the emotional state is abnormal based on the type of emotions of the at least one participant and the time duration of the emotional state of the kind. In the information providing device,
Acquiring speech information provided by at least one participant participating in a conversation using a communication terminal,
Acquiring status information indicating the current status of the at least one participant using the speech information and / or pre-stored data,
A control unit for determining virtual speech to be provided to the at least one participant based on the speech information and the status information; And
And outputting the determined virtual speech. 12. The method of claim 11,
Wherein the output unit outputs a virtual speech inquiring to obtain information about the at least one participant,
Wherein the control unit obtains the speech information including the additional speech information obtained through the inquiring virtual speech. 12. The method of claim 11,
Wherein the status information comprises emotion information indicating a current emotional state of the at least one participant, personal information of the at least one participant, topic information of the conversation, And at least one of schedule information of the participant and personality information of the at least one participant. 12. The method of claim 11,
Wherein the speech information comprises voice information of the at least one participant acquired by the communication terminal. 12. The method of claim 11,
The control unit
Acquiring contents of the virtual speech using the speech information,
Determines the expression system of the virtual speech using the status information,
And obtains the virtual speech in which the obtained content is represented by the determined expression. 12. The method of claim 11,
The control unit
Determining whether the speech information includes a predetermined keyword,
And if the speech information includes the predetermined keyword, obtains the virtual speech including content for a subject different from the subject of the conversation. 12. The method of claim 11,
Wherein the status information includes emotion information of the at least one participant,
The control unit
And obtains the virtual speech of the content and the presentation method determined according to the emotion information. In the information providing communication terminal,
Acquiring speech information provided by at least one participant participating in a conversation using the communication terminal,
Acquiring status information indicating the current status of the at least one participant using the speech information and / or pre-stored data,
A control unit for determining virtual speech to be provided to the at least one participant based on the speech information and the status information; And
And outputting the determined virtual speech. In the information providing server,
Acquiring speech information provided by at least one participant participating in a conversation using a communication terminal,
Acquiring status information indicating the current status of the at least one participant using the speech information and / or pre-stored data,
A control unit for determining virtual speech to be provided to the at least one participant based on the speech information and the status information; And
And outputting the determined virtual speech. A computer-readable non-volatile recording medium storing a program for executing the method of any one of claims 1 to 10 in a computer.

Images