WO2015156443A1 - Système de service de secrétaire particulier mobile du type dessin humoristique - Google Patents

Système de service de secrétaire particulier mobile du type dessin humoristique Download PDF

Info

Publication number
WO2015156443A1
WO2015156443A1 PCT/KR2014/003622 KR2014003622W WO2015156443A1 WO 2015156443 A1 WO2015156443 A1 WO 2015156443A1 KR 2014003622 W KR2014003622 W KR 2014003622W WO 2015156443 A1 WO2015156443 A1 WO 2015156443A1
Authority
WO
WIPO (PCT)
Prior art keywords
response
cartoon
module
emotion
window
Prior art date
Application number
PCT/KR2014/003622
Other languages
English (en)
Korean (ko)
Inventor
태정수
Original Assignee
네무스텍(주)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 네무스텍(주) filed Critical 네무스텍(주)
Publication of WO2015156443A1 publication Critical patent/WO2015156443A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/63—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for estimating an emotional state

Definitions

  • the present invention relates to a cartoon-type mobile personal assistant service system, and more particularly, the user's interest and convenience in recognizing a user's voice in a mobile device, processing a command according to the voice, and displaying the result on a display.
  • the present invention relates to a cartoon type mobile personal assistant service system that provides a cartoon form in order to enhance and effectively express emotions that are difficult to express with letters.
  • Mobile personal assistant service such as the iPhone's SIRI service, that sends a voice command to a mobile device to notify the user by voice of the results of processing or processing a search, sending an email, or scheduling an event on the mobile device.
  • a conventional personal assistant service generally recognizes a user's voice command as a text command using various voice recognition techniques and processes the user's voice command according to the recognition result.
  • Korean Laid-Open Patent Publication No. 2003-0033890 discloses a system for providing a personal assistant service using such a voice recognition technology.
  • the conventional personal assistant service converts a voice command into text through the meaning of a word included in a user's voice command and recognizes only the information as a command and responds only by voice or in the form of a simple text.
  • Such a conventional mobile personal assistant service has a problem that can be felt dry to the user and soon lose the interest of use. As a result, there is a problem that the frequency of use of the user is reduced and the desire for use of the user is also reduced.
  • the present invention has been made to solve the problems described above, by displaying the user's voice command and the response of the personal assistant service to the mobile device in a cartoon format to improve the user's interest and convenience and effectively convey emotion To provide personal assistant services.
  • Cartoon-type mobile personal assistant service system of the present invention for achieving the above object, by receiving a user's voice command from the mobile device to generate a response to the voice command of the mobile device through a virtual personal assistant
  • a cartoon type mobile personal assistant service system displayed on a display unit comprising: a voice receiving module configured to receive a voice command of a user through a microphone of a mobile device; A texting module for analyzing the voice command and converting the voice command into a textual text command; A response module for generating a response to the text command in a characterized response sentence; And a display module configured to generate a chat window on the display unit of the mobile device, generate a command window for displaying the text command in a cartoon form, and a response window for displaying the response sentence in a cartoon form, and scrollably display the chat window.
  • a voice receiving module configured to receive a voice command of a user through a microphone of a mobile device
  • a texting module for analyzing the voice command and converting the voice command into a textual
  • the cartoon-type mobile personal assistant service system of the present invention improves user's interest and improves service satisfaction by displaying commands and responses of a user and a virtual personal assistant on a display of a mobile device in a cartoon format.
  • FIG. 1 is a block diagram of a cartoon-type mobile personal assistant service system according to an embodiment of the present invention.
  • FIG. 2 is a diagram illustrating a state in which a chat window of the cartoon-type mobile personal assistant service system illustrated in FIG. 1 is displayed on a display unit of a mobile device.
  • FIG. 3 illustrates an emotion plane for explaining the cartoon-type mobile personal assistant service system shown in FIG. 1.
  • FIG. 4 illustrates another example of a command window and a response window displayed by the cartoon-type mobile personal assistant service system shown in FIG. 1.
  • FIG. 5 and 6 show another example of the response window displayed by the cartoon-type mobile personal assistant service system shown in FIG. 1, respectively.
  • the cartoon-type mobile personal assistant service system of the present embodiment includes a voice receiving module 110, a texting module 120, a response module 140, and a display module 150.
  • the voice receiving module 110 receives a voice command of a user through a microphone of the mobile device.
  • the user may speak voice commands such as "What is the weather of the day?”, “What is my schedule today?", "What is the phone number of the nearest coffee shop?”
  • the voice command received by the voice receiving module 110 is transmitted to the texting module 120 and the emotion extraction module 130.
  • the texting module 120 analyzes the voice commands and converts them into textual text commands.
  • the texting module 120 converts a user's voice command into a textual command using commonly used speech recognition technology.
  • the emotion extracting module 130 receives and analyzes a voice command from the voice receiving module 110 and extracts a user's emotion by receiving and analyzing a text command from the texting module 120.
  • the emotion extraction module 130 determines the degree of harmony of the user conversation using the text command, and determines the tension of the user using the voice command.
  • the degree of harmony is a value obtained by quantifying the degree of pleasantness and displeasure of user emotion.
  • the emotion extracting module 130 analyzes the words of the text command and analyzes the degree of inclusion of negative morphemes or positive morphemes, the degree of inclusion of negative or positive words in the text command, and the degree of discomfort of the ending of the text command. And the degree of pleasantness are quantified as the degree of harmony.
  • the emotion extraction module 130 digitizes the degree of harmony in consideration of the morpheme, the vocabulary, the presence or absence of a compound, etc.
  • the degree of tension is a numerical value of the degree of tension or excitement of the user. High tension is a state of surprise and awakening; low tension is a state of calm and relaxation.
  • the emotion extraction module 130 analyzes the sound of the voice command and digitizes the degree of tension to the degree of relaxation and awakening. The emotion extracting module 130 recognizes that the sound of the voice command is awake state when the sound of the voice command is higher and faster than the preset sound criterion, and is relaxed when the sound of the voice command is lower than the sound criterion.
  • the emotion extraction module 130 may quantify the tension in consideration of the amplitude of the sound of the voice command, that is, the amplitude of the sound.
  • the emotion extraction module 130 may quantify the degree of tension by further considering the accuracy of the pronunciation of the voice command read by the recognition rate of the voice.
  • the emotion extraction module 130 may determine the emotion of the user by expressing the harmony and tension as described above as coordinate values on the emotion plane as shown in FIG. 3.
  • the degree of unpleasantness and the level of unpleasantness is expressed by the coordinates of the first axis (x-axis), and the tension indicating the degree of excitement of the user is represented by the second axis (y-axis). It is represented by coordinates.
  • the emotion extraction module 130 may classify the type of emotion for each area on the emotion plane. For example, in the state of moderate tension, when the degree of harmony is low, it is judged by the feeling of "unhappy, misery, sadness", and when the degree of harmony is high, it is judged by the feeling of "happy, joy".
  • the response module 140 analyzes the text command characterizing the voice command in the text module 120 to process the command and to provide a textual response to the text command. That is, the response module 140 analyzes the text command to determine the meaning of the voice command and performs the command according to the meaning.
  • the response module 140 may search for information necessary by wireless communication, search for a contact stored in the mobile device of the user, and grasp a user's schedule or register a new user.
  • the response module 140 retrieves the weather of the day through wireless communication and “rains today.” Or a response that says, "I'll tell you today's weather.”
  • the display module 150 generates a chat window 200 as shown in FIG. 2 on the display unit 170 of the mobile device, and communicates the dialog between the user and the virtual personal assistant through the chat window 200 in a cartoon format. ).
  • the display module 150 displays the command window 210, the response window 220, and the result window 230 in a chat window 200 in a scrollable manner.
  • the command window 210 is a result of the text module 120 converting a user's voice command into a text command.
  • the display module 150 does not simply display the text command in the command window 210 in letters, but in a cartoon format.
  • the cartoon format means setting a frame that is a frame of the command window 210 like a cartoon format, setting a background image or a background color in the frame, displaying a character representing the user, and using a speech bubble.
  • the user's voice command draws a speech bubble next to the user character and a text command inside the speech bubble.
  • the command window 210 may further display an image of an object suitable for the voice command of the user and the conversation of the personal assistant.
  • the user's interest can be enhanced and satisfaction with the personal assistant service can be improved.
  • the emotion of the user using a cartoon format can have a transmission power more than the character and has the advantage of improving the satisfaction of the user.
  • the display module 150 displays the response window 220 in a similar manner to the command window 210 described above.
  • the response window 220 displays a response of the personal assistant to the voice command of the user in the chat window 200 in a cartoon form.
  • the frame is composed of a background color, a character image of a personal assistant, and text of a response sentence displayed inside a speech bubble. Response sentences such as "I will inform you the weather of today", "I will guide the phone number information" is displayed inside the speech bubble of the response window (220).
  • the display module 150 displays the result of inquiring the information (response information) in the result window 230. That is, when the voice command is a command for requesting inquiry of response information (eg, a phone number) stored in the mobile device or response information (eg, bus operation information) stored in an external server, the response module 140 may request response information. The result is displayed in the result window 230.
  • the response module 140 receives the response information in the HTML format or processes the response information in the HTML format and transmits the response information to the display module 150, and the display module 150 displays the response information in the result window 230.
  • the display module 150 may simply display the result window 230 in text form according to the content of the HTML format of the response information, and like the command window 210 and the response window 220, the result window 230 may be in a cartoon format. ) Can also be displayed. In the example shown in FIG. 2, the result window 230 displays a picture, a name, and a phone number of the person inquiring.
  • the cartoon-type mobile personal assistant service system of the present embodiment can be used in conjunction with an external server that provides various information such as movie timetable, bus operation information, aircraft operation information, weather information, etc.
  • the external server may provide response information in various visual ways.
  • the operator providing the cartoon-type mobile personal assistant service system of the present embodiment only needs to manage the cartoon image of the command window 210 and the result window 230, and the server manager of the external connection service of the result window 230 Since the result window 230 can be provided in an effective way according to the standard, there is an advantage of improving the operation efficiency of the overall service.
  • the response module 140 pre-examines the size of the result window 230 when inquiring the response information to the external server. Will be sent to the server.
  • the external server transmits the response information in HTML format to the mobile device in consideration of the size of the result window 230.
  • the display module 150 links and generates the result window 230 with the related application so as to be linked with the related application by the touch.
  • a weather-related result window 230 when a user touches the result window 230, a weather application connected to an external server providing weather information is executed on the mobile device.
  • the result window 230 related to the movie showing time when the user touches the result window 230, an application connected to an external server that provides the movie showing timetable is executed on the mobile device.
  • the phonebook application is executed to search for more detailed information desired by the user. In this way, if the user wants to inquire more detailed information from the cartoon-type mobile personal assistant service system of the present embodiment, it is possible to inquire the corresponding information by touching the result window 230.
  • the cartoon storage module 160 stores various images used for constructing the command window 210 and the response window 220.
  • Various images of the user character to be used in the command window 210 and various images of the personal assistant character to be used in the response window 220 are stored in the cartoon storage module 160.
  • the user character and the personal assistant character are designed with expressions corresponding to various emotions such as joy, anger, and sadness and are stored in the cartoon storage module 160.
  • Various shapes of speech bubbles to be used in the command window 210 and the response window 220 are also stored in the cartoon storage module 160. Speech balloons are also variously designed according to the emotion of the user or personal assistant and stored in the cartoon storage module 160.
  • various background colors of the command window 210 and the response window 220 corresponding to the emotion of the user or the personal assistant may also be stored in the cartoon storage module 160 in response to the emotion, and the command window 210 and Various objects such as a clock, a cup, and a book to be displayed in the response window 220 are also stored in the cartoon storage module 160.
  • the display module 150 configures the command window 210 by inquiring the character image and the speech bubble image of the appropriate user character in the cartoon storage module 160 according to the emotion of the user extracted by the emotion extraction module 130 described above.
  • the response module 140 determines a response emotion corresponding to the emotion of the user extracted by the emotion extraction module 130 and constructs a response sentence accordingly.
  • the response module 140 determines the response text according to the response feelings.
  • the display module 150 configures the response window 220 by inquiring the character image and the speech bubble image of the personal assistant character who can express the appropriate emotion in the cartoon storage module 160 according to the response emotion determined by the response module 140. .
  • the coordinates on the emotion plane of the response sentence corresponding thereto are set for each coordinate on the emotion plane of the voice command.
  • the correspondence between the coordinates on the emotion plane of the voice command and the emotion plane of the response emotion may be set in various ways. For example, when the user's emotion is "happiness, joy", the response emotion may also be set to correspond to the user's emotion by "corresponding to" happy, joy ". In addition, when the emotion of the voice command is "unhappy, sad", the response emotion may be set to comfort and alleviate the user's emotion by responding with "difference, calm”.
  • the response sentence may be configured or the background color of the response window 220 or the shape of the speech bubble may be determined according to the determined emotion.
  • the response module 140 configures the response sentence by adjusting the morpheme, vocabulary, and ending of the response sentence of the response sentence according to the position on the emotion plane of the response emotion.
  • the command window 210 is configured and displayed on the display unit 170 in a form that can be expressed.
  • 5 and 6 illustrate examples of character images, background images, and speech bubbles of personal assistants modified according to response emotions.
  • an appropriate response to the voice command of the user is displayed in the response window 220 in the form of a cartoon, so that the user's interest and satisfaction can be improved as compared to a personal assistant service that has conventionally only responded to text.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Hospice & Palliative Care (AREA)
  • Psychiatry (AREA)
  • General Health & Medical Sciences (AREA)
  • Signal Processing (AREA)
  • Child & Adolescent Psychology (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente invention concerne un système de service de secrétaire particulier mobile du type dessin humoristique qui reçoit une commande vocale d'un utilisateur sur un appareil mobile destinée à générer une réponse à la commande vocale, affichant ainsi la réponse sur une unité d'affichage de l'appareil mobile par le biais d'un secrétaire personnel virtuel comprenant: un module de réception de voix destiné à recevoir une commande vocale de l'utilisateur par le biais d'un microphone de l'appareil mobile; un module de transcription permettant de convertir la commande vocale en une commande sous format texte par analyse de la commande vocale; un module de réponse permettant de générer une réponse à la commande sous format texte sous la forme d'une phrase de réponse transcrite; et un module d'affichage permettant de générer une fenêtre de discussion sur l'unité d'affichage de l'appareil mobile et de générer respectivement une fenêtre d'instruction affichant la commande sous format texte dans un format du type dessin humoristique et une fenêtre de réponse affichant la phrase de réponse dans le format du type dessin humoristique de sorte qu'un défilement soit permis dans la fenêtre de discussion.
PCT/KR2014/003622 2014-04-11 2014-04-24 Système de service de secrétaire particulier mobile du type dessin humoristique WO2015156443A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2014-0043484 2014-04-11
KR20140043484 2014-04-11

Publications (1)

Publication Number Publication Date
WO2015156443A1 true WO2015156443A1 (fr) 2015-10-15

Family

ID=54288009

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2014/003622 WO2015156443A1 (fr) 2014-04-11 2014-04-24 Système de service de secrétaire particulier mobile du type dessin humoristique

Country Status (1)

Country Link
WO (1) WO2015156443A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108364653A (zh) * 2018-02-12 2018-08-03 王磊 语音数据处理方法及处理装置
WO2020116818A1 (fr) * 2018-12-03 2020-06-11 Samsung Electronics Co., Ltd. Dispositif électronique et son procédé de commande
CN113794927A (zh) * 2021-08-12 2021-12-14 维沃移动通信有限公司 信息显示方法、装置及电子设备

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0855130A (ja) * 1994-08-11 1996-02-27 Sharp Corp 電子秘書システム
US20050080783A1 (en) * 2000-01-05 2005-04-14 Apple Computer, Inc. One Infinite Loop Universal interface for retrieval of information in a computer system
KR20100088461A (ko) * 2009-01-30 2010-08-09 삼성전자주식회사 음성 신호를 이용한 감정 인식 장치 및 방법

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0855130A (ja) * 1994-08-11 1996-02-27 Sharp Corp 電子秘書システム
US20050080783A1 (en) * 2000-01-05 2005-04-14 Apple Computer, Inc. One Infinite Loop Universal interface for retrieval of information in a computer system
KR20100088461A (ko) * 2009-01-30 2010-08-09 삼성전자주식회사 음성 신호를 이용한 감정 인식 장치 및 방법

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
MARO&SUNNYSPOT: "TOONPAL (Cartoon Random Chatting", GOOGLE PLAY, 13 March 2014 (2014-03-13), Retrieved from the Internet <URL:https://play.google.com/store/apps/details?id=com.sunnyspot.toonpal&hl=ko> *
TOM WARREN: "Apple has Siri, and Microsoft is about to get Cortana", THE VERGE, 20 February 2014 (2014-02-20), Retrieved from the Internet <URL:http://www.theverge.com/2014/2/20/5430188/microsoft-cortana-personal-digital-assistant-windows-phone-8-1> *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108364653A (zh) * 2018-02-12 2018-08-03 王磊 语音数据处理方法及处理装置
WO2020116818A1 (fr) * 2018-12-03 2020-06-11 Samsung Electronics Co., Ltd. Dispositif électronique et son procédé de commande
US11495220B2 (en) 2018-12-03 2022-11-08 Samsung Electronics Co., Ltd. Electronic device and method of controlling thereof
US12087298B2 (en) 2018-12-03 2024-09-10 Samsung Electronics Co., Ltd. Electronic device and method of controlling thereof
CN113794927A (zh) * 2021-08-12 2021-12-14 维沃移动通信有限公司 信息显示方法、装置及电子设备

Similar Documents

Publication Publication Date Title
CN109447234B (zh) 一种模型训练方法、合成说话表情的方法和相关装置
CN109697973B (zh) 一种韵律层级标注的方法、模型训练的方法及装置
CN110418208B (zh) 一种基于人工智能的字幕确定方法和装置
KR101777807B1 (ko) 수화 번역기, 시스템 및 방법
CN103116576A (zh) 一种语音手势交互翻译装置及其控制方法
JP6392374B2 (ja) ヘッドマウントディスプレイシステム及びヘッドマウントディスプレイ装置の操作方法
CN108763552B (zh) 一种基于家教机的学习方法及家教机
EP2933607A1 (fr) Système de navigation ayant une fonction auto-adaptative de catégorie de langage et procédé de commande du système
JP2019008570A (ja) 情報処理装置、情報処理方法及びプログラム
US11120063B2 (en) Information processing apparatus and information processing method
JP2017531197A (ja) 文字データの内容を文字データ送信者の音声で出力する方法
US20150120277A1 (en) Method, Device And System For Providing Language Service
WO2021006538A1 (fr) Dispositif de transformation visuelle d&#39;avatar exprimant un message textuel en tant que v-moji et procédé de transformation de message
CN203149569U (zh) 一种语音手势交互翻译装置
CN108074574A (zh) 音频处理方法、装置及移动终端
WO2016203805A1 (fr) Dispositif, système, procédé et programme de traitement d&#39;informations
CN109308178A (zh) 一种语音画图方法及其终端设备
WO2015156443A1 (fr) Système de service de secrétaire particulier mobile du type dessin humoristique
CN109686359B (zh) 语音输出方法、终端及计算机可读存储介质
CN110555329A (zh) 一种手语翻译的方法、终端以及存储介质
CN114065168A (zh) 信息处理方法、智能终端及存储介质
KR101981091B1 (ko) 감정시각화자막 생성장치
KR101567154B1 (ko) 다중 사용자 기반의 대화 처리 방법 및 이를 수행하는 장치
CN108491471B (zh) 一种文本信息的处理方法、移动终端
CN111145734A (zh) 一种语音识别方法及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14888917

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14888917

Country of ref document: EP

Kind code of ref document: A1