WO2015156443A1

WO2015156443A1 - Cartoon-type mobile personal secretary service system

Info

Publication number: WO2015156443A1
Application number: PCT/KR2014/003622
Authority: WO
Inventors: 태정수
Original assignee: 네무스텍(주)
Priority date: 2014-04-11
Filing date: 2014-04-24
Publication date: 2015-10-15

Abstract

According to the present invention, a cartoon-type mobile personal secretary service system, which receives a voice command of a user on a mobile device to generate a response to the voice command, thereby displaying the response on a display unit of the mobile device through a virtual personal secretary, comprises: a voice reception module for receiving a user voice command through a microphone of the mobile device; a transcription module for converting the voice command into a transcribed text command by analyzing the voice command; a response module for generating a response to the text command as a transcribed response sentence; and a display module for generating a chat window on the display unit of the mobile device and respectively generating a command window displaying the text command in a cartoon format and a response window displaying the response sentence in the cartoon format such that scrolling is enabled in the chat window.

Description

Cartoon Mobile Personal Assistant Service System

The present invention relates to a cartoon-type mobile personal assistant service system, and more particularly, the user's interest and convenience in recognizing a user's voice in a mobile device, processing a command according to the voice, and displaying the result on a display. The present invention relates to a cartoon type mobile personal assistant service system that provides a cartoon form in order to enhance and effectively express emotions that are difficult to express with letters.

Mobile personal assistant service, such as the iPhone's SIRI service, that sends a voice command to a mobile device to notify the user by voice of the results of processing or processing a search, sending an email, or scheduling an event on the mobile device. Has been put to practical use in recent years.

A conventional personal assistant service generally recognizes a user's voice command as a text command using various voice recognition techniques and processes the user's voice command according to the recognition result. Korean Laid-Open Patent Publication No. 2003-0033890 discloses a system for providing a personal assistant service using such a voice recognition technology.

The conventional personal assistant service converts a voice command into text through the meaning of a word included in a user's voice command and recognizes only the information as a command and responds only by voice or in the form of a simple text.

Such a conventional mobile personal assistant service has a problem that can be felt dry to the user and soon lose the interest of use. As a result, there is a problem that the frequency of use of the user is reduced and the desire for use of the user is also reduced.

The present invention has been made to solve the problems described above, by displaying the user's voice command and the response of the personal assistant service to the mobile device in a cartoon format to improve the user's interest and convenience and effectively convey emotion To provide personal assistant services.

Cartoon-type mobile personal assistant service system of the present invention for achieving the above object, by receiving a user's voice command from the mobile device to generate a response to the voice command of the mobile device through a virtual personal assistant A cartoon type mobile personal assistant service system displayed on a display unit, comprising: a voice receiving module configured to receive a voice command of a user through a microphone of a mobile device; A texting module for analyzing the voice command and converting the voice command into a textual text command; A response module for generating a response to the text command in a characterized response sentence; And a display module configured to generate a chat window on the display unit of the mobile device, generate a command window for displaying the text command in a cartoon form, and a response window for displaying the response sentence in a cartoon form, and scrollably display the chat window. There is a feature in that it includes.

The cartoon-type mobile personal assistant service system of the present invention improves user's interest and improves service satisfaction by displaying commands and responses of a user and a virtual personal assistant on a display of a mobile device in a cartoon format.

1 is a block diagram of a cartoon-type mobile personal assistant service system according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a state in which a chat window of the cartoon-type mobile personal assistant service system illustrated in FIG. 1 is displayed on a display unit of a mobile device.

FIG. 3 illustrates an emotion plane for explaining the cartoon-type mobile personal assistant service system shown in FIG. 1.

4 illustrates another example of a command window and a response window displayed by the cartoon-type mobile personal assistant service system shown in FIG. 1.

5 and 6 show another example of the response window displayed by the cartoon-type mobile personal assistant service system shown in FIG. 1, respectively.

Hereinafter, a cartoon-type mobile personal assistant service system according to the present invention will be described in detail with reference to a preferred embodiment.

Referring to FIG. 1, the cartoon-type mobile personal assistant service system of the present embodiment includes a voice receiving module 110, a texting module 120, a response module 140, and a display module 150.

The voice receiving module 110 receives a voice command of a user through a microphone of the mobile device. The user may speak voice commands such as "What is the weather of the day?", "What is my schedule today?", "What is the phone number of the nearest coffee shop?"

The voice command received by the voice receiving module 110 is transmitted to the texting module 120 and the emotion extraction module 130.

The texting module 120 analyzes the voice commands and converts them into textual text commands. The texting module 120 converts a user's voice command into a textual command using commonly used speech recognition technology.

The emotion extracting module 130 receives and analyzes a voice command from the voice receiving module 110 and extracts a user's emotion by receiving and analyzing a text command from the texting module 120. The emotion extraction module 130 determines the degree of harmony of the user conversation using the text command, and determines the tension of the user using the voice command.

As shown in FIG. 3, the degree of harmony is a value obtained by quantifying the degree of pleasantness and displeasure of user emotion. The emotion extracting module 130 analyzes the words of the text command and analyzes the degree of inclusion of negative morphemes or positive morphemes, the degree of inclusion of negative or positive words in the text command, and the degree of discomfort of the ending of the text command. And the degree of pleasantness are quantified as the degree of harmony. The emotion extraction module 130 digitizes the degree of harmony in consideration of the morpheme, the vocabulary, the presence or absence of a compound, etc.

As shown in FIG. 3, the degree of tension is a numerical value of the degree of tension or excitement of the user. High tension is a state of surprise and awakening; low tension is a state of calm and relaxation. The emotion extraction module 130 analyzes the sound of the voice command and digitizes the degree of tension to the degree of relaxation and awakening. The emotion extracting module 130 recognizes that the sound of the voice command is awake state when the sound of the voice command is higher and faster than the preset sound criterion, and is relaxed when the sound of the voice command is lower than the sound criterion. The emotion extraction module 130 may quantify the tension in consideration of the amplitude of the sound of the voice command, that is, the amplitude of the sound. The emotion extraction module 130 may quantify the degree of tension by further considering the accuracy of the pronunciation of the voice command read by the recognition rate of the voice.

The emotion extraction module 130 may determine the emotion of the user by expressing the harmony and tension as described above as coordinate values on the emotion plane as shown in FIG. 3. On the emotion plane shown in Fig. 3, the degree of unpleasantness and the level of unpleasantness is expressed by the coordinates of the first axis (x-axis), and the tension indicating the degree of excitement of the user is represented by the second axis (y-axis). It is represented by coordinates. The emotion extraction module 130 may classify the type of emotion for each area on the emotion plane. For example, in the state of moderate tension, when the degree of harmony is low, it is judged by the feeling of "unhappy, misery, sadness", and when the degree of harmony is high, it is judged by the feeling of "happy, joy". In the state of high arousal, the harmony is low, the feeling of "anguishment, anguish", the harmony is in the middle of "surprise, stimulation", the feeling of harmony, the feeling of "happy, joy". In the relaxed state of low tension, if the harmony is low, the feeling of "dry, boring", if the harmony is medium, "difference, calm" feeling, if the harmony is high, "loose, satisfied" feeling.

On the other hand, the response module 140 analyzes the text command characterizing the voice command in the text module 120 to process the command and to provide a textual response to the text command. That is, the response module 140 analyzes the text command to determine the meaning of the voice command and performs the command according to the meaning. The response module 140 may search for information necessary by wireless communication, search for a contact stored in the mobile device of the user, and grasp a user's schedule or register a new user.

For example, if the user's voice command is "What is the weather of the day?", The response module 140 retrieves the weather of the day through wireless communication and "rains today." Or a response that says, "I'll tell you today's weather."

The display module 150 generates a chat window 200 as shown in FIG. 2 on the display unit 170 of the mobile device, and communicates the dialog between the user and the virtual personal assistant through the chat window 200 in a cartoon format. ).

The display module 150 displays the command window 210, the response window 220, and the result window 230 in a chat window 200 in a scrollable manner. 2 and 4, the command window 210 is a result of the text module 120 converting a user's voice command into a text command. The display module 150 does not simply display the text command in the command window 210 in letters, but in a cartoon format. The cartoon format means setting a frame that is a frame of the command window 210 like a cartoon format, setting a background image or a background color in the frame, displaying a character representing the user, and using a speech bubble. The user's voice command draws a speech bubble next to the user character and a text command inside the speech bubble. Text commands such as "What is the weather for today?" And "Sua phone number?" Will be displayed in the speech bubble. In some cases, the command window 210 may further display an image of an object suitable for the voice command of the user and the conversation of the personal assistant. As such, by displaying a user's command in response to a virtual personal assistant in a cartoon format, the user's interest can be enhanced and satisfaction with the personal assistant service can be improved. In addition, by expressing the emotion of the user using a cartoon format can have a transmission power more than the character and has the advantage of improving the satisfaction of the user.

2 and 4, the display module 150 displays the response window 220 in a similar manner to the command window 210 described above. The response window 220 displays a response of the personal assistant to the voice command of the user in the chat window 200 in a cartoon form. Like the command window 210, the frame is composed of a background color, a character image of a personal assistant, and text of a response sentence displayed inside a speech bubble. Response sentences such as "I will inform you the weather of today", "I will guide the phone number information" is displayed inside the speech bubble of the response window (220).

Referring to FIG. 2, when the voice command is an instruction requesting information inquiry, the display module 150 displays the result of inquiring the information (response information) in the result window 230. That is, when the voice command is a command for requesting inquiry of response information (eg, a phone number) stored in the mobile device or response information (eg, bus operation information) stored in an external server, the response module 140 may request response information. The result is displayed in the result window 230. The response module 140 receives the response information in the HTML format or processes the response information in the HTML format and transmits the response information to the display module 150, and the display module 150 displays the response information in the result window 230. The display module 150 may simply display the result window 230 in text form according to the content of the HTML format of the response information, and like the command window 210 and the response window 220, the result window 230 may be in a cartoon format. ) Can also be displayed. In the example shown in FIG. 2, the result window 230 displays a picture, a name, and a phone number of the person inquiring.

In particular, the cartoon-type mobile personal assistant service system of the present embodiment can be used in conjunction with an external server that provides various information such as movie timetable, bus operation information, aircraft operation information, weather information, etc. By using the display module 150 displaying the result window 230, the external server may provide response information in various visual ways. By doing so, the operator providing the cartoon-type mobile personal assistant service system of the present embodiment only needs to manage the cartoon image of the command window 210 and the result window 230, and the server manager of the external connection service of the result window 230 Since the result window 230 can be provided in an effective way according to the standard, there is an advantage of improving the operation efficiency of the overall service.

As such, when the external server provides the response information in the HTML format conforming to the standard of the result window 230, the response module 140 pre-examines the size of the result window 230 when inquiring the response information to the external server. Will be sent to the server. The external server transmits the response information in HTML format to the mobile device in consideration of the size of the result window 230.

Meanwhile, the display module 150 links and generates the result window 230 with the related application so as to be linked with the related application by the touch. For example, in the case of a weather-related result window 230, when a user touches the result window 230, a weather application connected to an external server providing weather information is executed on the mobile device. In the case of the result window 230 related to the movie showing time, when the user touches the result window 230, an application connected to an external server that provides the movie showing timetable is executed on the mobile device. In the case of FIG. 2, when the user touches the result window 230, the phonebook application is executed to search for more detailed information desired by the user. In this way, if the user wants to inquire more detailed information from the cartoon-type mobile personal assistant service system of the present embodiment, it is possible to inquire the corresponding information by touching the result window 230.

Meanwhile, the cartoon storage module 160 stores various images used for constructing the command window 210 and the response window 220. Various images of the user character to be used in the command window 210 and various images of the personal assistant character to be used in the response window 220 are stored in the cartoon storage module 160. In some cases, the user character and the personal assistant character are designed with expressions corresponding to various emotions such as joy, anger, and sadness and are stored in the cartoon storage module 160. Various shapes of speech bubbles to be used in the command window 210 and the response window 220 are also stored in the cartoon storage module 160. Speech balloons are also variously designed according to the emotion of the user or personal assistant and stored in the cartoon storage module 160. In some cases, various background colors of the command window 210 and the response window 220 corresponding to the emotion of the user or the personal assistant may also be stored in the cartoon storage module 160 in response to the emotion, and the command window 210 and Various objects such as a clock, a cup, and a book to be displayed in the response window 220 are also stored in the cartoon storage module 160.

The display module 150 configures the command window 210 by inquiring the character image and the speech bubble image of the appropriate user character in the cartoon storage module 160 according to the emotion of the user extracted by the emotion extraction module 130 described above.

The response module 140 determines a response emotion corresponding to the emotion of the user extracted by the emotion extraction module 130 and constructs a response sentence accordingly. The response module 140 determines the response text according to the response feelings. The display module 150 configures the response window 220 by inquiring the character image and the speech bubble image of the personal assistant character who can express the appropriate emotion in the cartoon storage module 160 according to the response emotion determined by the response module 140. .

The process of determining the response feeling in the response module 140 will be described in more detail as follows. First, the coordinates on the emotion plane of the response sentence corresponding thereto are set for each coordinate on the emotion plane of the voice command. The correspondence between the coordinates on the emotion plane of the voice command and the emotion plane of the response emotion may be set in various ways. For example, when the user's emotion is "happiness, joy", the response emotion may also be set to correspond to the user's emotion by "corresponding to" happy, joy ". In addition, when the emotion of the voice command is "unhappy, sad", the response emotion may be set to comfort and alleviate the user's emotion by responding with "difference, calm".

When the response emotion is determined as described above, the response sentence may be configured or the background color of the response window 220 or the shape of the speech bubble may be determined according to the determined emotion. The response module 140 configures the response sentence by adjusting the morpheme, vocabulary, and ending of the response sentence of the response sentence according to the position on the emotion plane of the response emotion.

Referring to FIG. 4, when the user's voice command is recognized as an awakening and unpleasant emotion, the command window 210 is configured and displayed on the display unit 170 in a form that can be expressed. In addition, in constructing the response sentence and the response window 220, the image of the character to be used in the response window 220 according to the response feeling mapped to appropriately correspond to the emotion of the voice command in consideration of the emotion of the voice command of the user. You will choose the type of speech bubble. 5 and 6 illustrate examples of character images, background images, and speech bubbles of personal assistants modified according to response emotions. By combining various images as described above, an appropriate response to the voice command of the user is displayed in the response window 220 in the form of a cartoon, so that the user's interest and satisfaction can be improved as compared to a personal assistant service that has conventionally only responded to text.

Claims

In the cartoon-type mobile personal assistant service system for receiving a user's voice command from the mobile device to generate a response to the voice command to display on the display unit of the mobile device through a virtual personal assistant,

A voice receiving module configured to receive a voice command of a user through a microphone of the mobile device;

A texting module for analyzing the voice command and converting the voice command into a textual text command;

A response module for generating a response to the text command in a characterized response sentence; And

A display module for creating a chat window on a display unit of the mobile device, a command window for displaying the text command in a cartoon form, and a response window for displaying the response sentence in a cartoon form, and scrollably displaying the chat window in the chat window; Cartoon-type mobile personal assistant service system, characterized in that it comprises.
The method of claim 1,

The command window is displayed in a cartoon form by a character displaying the user, a speech bubble displaying the text command, and a background color of the command window.

The response window is cartoon type mobile personal assistant service system, characterized in that the character is displayed in a cartoon form by the character, the speech bubble, the response sentence is displayed and the background color of the response window.
The method according to claim 1 or 2,

When the voice command is stored in an external server connected to the mobile device through a network or requests an inquiry of response information stored in the mobile device,

The response module inquires and receives the response information,

The display module, the cartoon-type mobile personal assistant service system, characterized in that for further displaying a result window for displaying the response information in the chat window generated on the display unit of the mobile device.
The method of claim 3,

The response module receives the response information in HTML format that can be displayed in the result window from the external server,

The display module, Cartoon-type mobile personal assistant service system, characterized in that for configuring the result window to display the response information in the HTML format.
The method of claim 4, wherein

The response module transmits the size of the result window to the external server so that when the response information is inquired to the external server, the external server can transmit the response information in HTML format corresponding to the size of the result window. Cartoon mobile personal assistant service system.
The method of claim 5,

The display module is cartoon type, characterized in that configured to form a form that can be touched and linked to the application so that an application connected to the external server can be executed in the mobile device when the user touches the result window. Mobile personal assistant service system.
The method of claim 4, wherein

An emotion extraction module for analyzing the height and speed of the sound of the voice command and extracting the user's emotion by analyzing the word of the text command; And

And a cartoon storage module in which characters of the user and the personal assistant and the shape of the speech bubble are designed to have expressions corresponding to the emotions of the user extracted by the emotion extraction module.

The display module, the cartoon-type mobile personal assistant service, characterized in that to configure the command window by querying the character image and the speech bubble image of the user character in the cartoon storage module according to the emotion of the user extracted from the emotion extraction module system.
The method of claim 7, wherein

The response module determines a response emotion corresponding to the emotion of the user extracted by the emotion extraction module and determines the response text according to the response emotion,

The display module, the cartoon-type mobile personal assistant service system, characterized in that for configuring the response window by querying the character image and the speech bubble image of the personal assistant character in the cartoon storage module according to the response feelings determined in the response module. .
The method of claim 8,

The emotion extraction module recognizes the sound of the voice command as an awake state when the sound of the voice command is higher and faster than a preset sound reference, and recognizes it as a relaxed state when the sound of the voice command is higher and faster than the preset sound reference and digitizes the tension of the voice command to the degree of relaxation and awake state. Cartoon-type mobile personal assistant service system, characterized in that.
The method of claim 9,

The emotion extraction module analyzes a word of the text command and analyzes the degree of inclusion of a negative morpheme or a positive morpheme, a degree of inclusion of a negative or positive vocabulary in the text command, and a degree of displeasure of whether the ending of the text command is a verb. Cartoon mobile personal assistant service system, characterized in that to quantify the degree of enjoyment and fun.
The method of claim 10,

The emotion extracting module may include an emotion plane formed of a first axis representing a degree of discomfort and a pleasantness according to an analysis result of the text command, and a second axis representing a tension degree according to an analysis result of the voice command, Cartoon-type mobile personal assistant service system, characterized in that for classifying and extracting the user's emotions according to the position on the two-dimensional emotion plane.
The method of claim 11,

The response module sets the position on the emotion plane of the response emotion according to the position on the emotion plane of the voice command recognized by the emotion extraction module and constructs the response sentence according to the position on the emotion plane. Cartoon mobile personal assistant service system.
The method of claim 12,

The response module, Cartoon-type mobile personal assistant service system, characterized in that for configuring the response window by adjusting the expression, the shape of the speech bubble, the background color of the character of the response window in response to the position on the emotion plane.
The method of claim 12,

The response module, Cartoon-type mobile personal assistant service system, characterized in that for configuring the response sentence by adjusting the morpheme, vocabulary, and ending of the response sentence according to the position on the emotion plane of the response emotion.