KR20200028158A - Media play device, method and computer program for providing multi language voice command service - Google Patents

Abstract

A device for playing media to provide a multilanguage voice command service comprises: an input unit receiving a voice command from a user; a language recognition unit analyzing the inputted voice command and recognizing language uttered by the user; a classification unit determining a sentence structure of the voice command based on the recognized language and classifying vocabulary proficiency according to the determined sentence structure; a conversion unit converting output data for a user interface of the device for playing media based on the recognized language and the classified vocabulary proficiency; and an output unit dynamically outputting the user interface by applying the converted output data.

Description

MEDIA PLAY DEVICE, METHOD AND COMPUTER PROGRAM FOR PROVIDING MULTI LANGUAGE VOICE COMMAND SERVICE}

The present invention relates to a media playback apparatus, method and computer program that provides a multilingual voice command service.

The intelligent personal assistant is a software agent that handles the tasks required by the user and provides specialized services to the user. The intelligent personal assistant collects and provides personalized information to the user based on the artificial intelligence (AI) engine and speech recognition, and performs various functions such as scheduling, emailing, and restaurant reservation according to the user's voice commands. It has the advantage of improving convenience.

These intelligent personal assistants are mainly provided in the form of personalized personal services on smartphones, including Apple's Siri, Google's Now, and Samsung's Bixby. In this regard, Korean Patent Publication No. 2016-0071111, which is a prior art, discloses a method for providing a personal assistant service in an electronic device.

Recently, intelligent personal assistants have been applied to various service industries such as concierge services, and interaction with users has been expanded through touch / visual / speech, and through this, it has been possible to provide more diverse information. However, foreigners have difficulties in using intelligent personal assistants, and have the problem of being unable to receive services based on expanded interaction such as visual / voice.

It is intended to provide a media playback device, method, and computer program that provides a multilingual voice command service that dynamically configures a user interface in real time according to a user's language, vocabulary inclination, and accent style.

It is an object of the present invention to provide a media playback device, method, and computer program that provides a multilingual voice command service that provides a personalized speech guide to induce utterance of speech commands in a user's language according to a user's vocabulary inclination.

Multilingual voice command service that analyzes the pattern of the user's voice command and synthesizes the analysis result for the voice command to be similar to the user's intonation style, and outputs the analysis result of the user's intonation style and synthesized voice command through audio. It is intended to provide a media playback device, method and computer program providing a.

A media playback device that provides a multilingual voice command service that can improve the usability of a user's voice command service and improve the convenience of user interaction regardless of the user's language by reconfiguring the user interface in a language used by the user, It is intended to provide methods and computer programs.

However, the technical problems to be achieved by the present embodiment are not limited to the technical problems as described above, and other technical problems may exist.

As a means for achieving the above technical problem, an embodiment of the present invention, an input unit for receiving a voice command from a user, a language recognition unit for analyzing the input voice command to recognize the language spoken by the user, the The classification unit for determining the sentence structure of the voice command based on the recognized language, and classifying the vocabulary propensity according to the determined sentence configuration, the media playback device based on the recognized language and the classified vocabulary propensity It is possible to provide a media playback device including a conversion unit for converting output items for a user interface and an output unit for dynamically outputting the user interface by applying the converted output items.

According to another embodiment of the present invention, receiving a voice command from a user, analyzing the input voice command to recognize a language spoken by the user, and constructing a sentence structure of the voice command based on the recognized language Determining, classifying a vocabulary proficiency according to the determined sentence composition, converting outputs for a user interface of the media playback device based on the recognized language and the classified vocabulary proficiency, and the converted It is possible to provide a method for providing a multilingual voice command service, including dynamically outputting the user interface by applying output information.

According to another embodiment of the present invention, when a computer program is executed by a computing device, a voice command is input from a user, and the input voice command is analyzed to recognize a language spoken by the user, and the recognized user The sentence structure of the voice command is determined based on the language, the vocabulary proficiency is classified according to the determined sentence configuration, and the user interface of the media player is based on the recognized language and the classified vocabulary proficiency. It is possible to provide a computer program stored in a medium including a sequence of instructions for converting an output item and applying the converted output item to dynamically output the user interface.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present invention. In addition to the exemplary embodiments described above, there may be additional embodiments described in the drawings and detailed description of the invention.

According to any one of the above-described problem solving means of the present invention, a media playback device, method and method for providing a multilingual voice command service that dynamically configures a user interface in real time according to a language, vocabulary propensity, and accent style used by a user, and Computer programs can be provided.

According to a user's vocabulary inclination, a media playback device, method, and computer program that provides a multilingual voice command service that provides a personalized speech guide to induce speech in a user's language can be provided.

Multilingual voice command service that analyzes the pattern of the user's voice command and synthesizes the analysis result for the voice command to be similar to the user's intonation style, and outputs the analysis result of the user's intonation style and synthesized voice command through audio. It may provide a media playback device, method and computer program providing a.

A media playback device that provides a multilingual voice command service that can improve the usability of a user's voice command service and improve the convenience of user interaction regardless of the user's language by reconfiguring the user interface in a language used by the user, Methods and computer programs can be provided.

1 is a block diagram of a system for providing a multilingual voice command service according to an embodiment of the present invention.
2 is a block diagram of a media playback apparatus according to an embodiment of the present invention.
3 is an exemplary diagram for describing a process of classifying a user's vocabulary based on a user's language in a media playback apparatus according to an embodiment of the present invention.
4A and 4B are exemplary diagrams for explaining a process of converting a utterance list sentence into a language of a recognized user based on a preset mapping table for each language in the media playback apparatus according to an embodiment of the present invention.
5 is an exemplary diagram for explaining a process of inferring an accent style for a user's voice command in a media playback apparatus according to an embodiment of the present invention.
6A to 6C are exemplary views illustrating a user interface of a media playback device in which output items are converted based on a user's language according to an embodiment of the present invention.
7 is a flowchart of a method of providing a multilingual voice command service in a media playback apparatus according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains can easily practice. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention in the drawings, parts irrelevant to the description are omitted, and like reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element in between. . Also, when a part is said to “include” a certain component, it means that the component may further include other components, not to exclude other components, unless otherwise stated. However, it should be understood that the existence or addition possibilities of numbers, steps, actions, components, parts or combinations thereof are not excluded in advance.

In the present specification, the term “unit” includes a unit realized by hardware, a unit realized by software, and a unit realized by using both. Further, one unit may be realized by using two or more hardware, and two or more units may be realized by one hardware.

Some of the operations or functions described in this specification as being performed by a terminal or device may be performed instead on a server connected to the corresponding terminal or device. Similarly, some of the operations or functions described as being performed by the server may be performed in a terminal or device connected to the corresponding server.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a system for providing a multilingual voice command service according to an embodiment of the present invention. Referring to FIG. 1, the multilingual voice command service providing system 1 may include a media playback device 110, a display 115, and a multilingual conversation server 120. The media playback device 110, the display 115, and the multilingual conversation server 120 exemplarily show components that can be controlled by the multilingual voice command service providing system 1.

Each component of the multilingual voice command service providing system 1 of FIG. 1 is generally connected through a network. For example, as shown in FIG. 1, the media playback device 110 may be connected to the multilingual conversation server 120 at the same time or at a time interval.

Network refers to a connection structure capable of exchanging information between nodes such as terminals and servers, and a local area network (LAN), a wide area network (WAN), and the Internet (WWW: World) Wide Web), wired and wireless data communication networks, telephone networks, and wired and wireless television communication networks. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasound Communication, Visible Light Communication (VLC), LiFi, and the like are included, but are not limited thereto.

The media playback device 110 may receive a voice command from the user 100 and analyze the input voice command to recognize the language spoken by the user 100.

The media playback device 110 may determine the sentence structure of the voice command based on the recognized language, and classify the vocabulary propensity according to the determined sentence structure. For example, the media playback device 110 determines a part-of-speech of at least one word included in a sentence configuration of a voice command, categorizes a used vocabulary for each determined part-of-speech, and determines each part-of-speech according to the categorized use vocabulary The ratings are mapped by stars, and the vocabulary propensity of the user 100 may be determined based on the ratings mapped to each part of speech.

The media playback device 110 may generate a utterance guide sentence in a language spoken by the user 100 based on the classified vocabulary propensity. For example, the media playback device 110 may select a vocabulary based on the classified vocabulary propensity, and convert the utterance guide sentence into the language of the recognized user 100 based on the selected vocabulary. At this time, the media playback device 110 derives voice command information based on the frequency of use of the voice command spoken by the user 100, selects a speech list sentence based on the derived voice command information, and selects the spoken speech The vocabulary of the list sentence may be reconstructed based on the vocabulary proficiency of the user 100. The media playback device 110 may convert a speech list sentence in which the vocabulary is reconstructed into a language of the recognized user 100 based on a preset mapping table for each language. The media playback device 110 may convert display outputs for the user interface of the media playback device 110 based on the recognized language and classified vocabulary propensity.

The media playback device 110 may convert the text output on the display 115 of the media playback device 110 into a recognized language format by referring to the classified vocabulary propensity.

The media playback device 110 may extract language feature information for the accent and accent of the user 100 from the input voice command, and analyze the accent style for the voice command through a pattern for the extracted language feature information. . For example, the media playback device 110 converts the input voice command into a pitch symbol based on the rhythm notation convention notation, analyzes the transformed pitch symbol based on a preset speech sequence, and analyzes the pattern of the pitch symbol The intonation style for the voice command may be inferred based on the similarity with the pre-stored intonation pattern for each language.

The media playback device 110 may transmit a voice command to the multilingual conversation server 120 based on the recognized language of the user 100 and receive an analysis result of the voice command from the multilingual conversation server 120. At this time, the media playback device 110 may synthesize the analysis result of the voice command received from the multilingual conversation server 120 with the accent style of the user 100. The media playback device 110 may convert outputs for the user interface based on the inferred accent style.

When the media playback device 110 converts the output for the user interface, the media playback device 110 may dynamically output the user interface by applying the converted output. For example, the media playback device 110 may output a utterance guide sentence generated to induce a voice command in the language of the user 100 on the display 150. For another example, the media playback device 110 displays the converted text when the text output on the display 115 of the media playback device 110 is converted into a recognized language format by referring to the classified vocabulary propensity. (115). For another example, the media playback device 110 may output the analysis result of the speech style synthesized with the accent style of the user 100 through audio.

The display 115 may output a user interface screen through a display. For example, the display 115 may output a Korean user interface set as a default through the display, and output, for example, an English user interface through the display based on the recognized language of the user 100. .

The display 115 displays a user guide screen displaying a utterance guide sentence through the first area, displaying life information such as date, time, weather, temperature, and humidity through the second area, and sub-menu through the third area. And display external linkage messages and terminal setting information through the fourth area, and display language setting information and control information through the fifth area.

The multilingual conversation server 120 may receive a voice command from the media playback device 110. In addition, the multilingual conversation server 120 may analyze the received voice command and transmit the analysis results to the media playback device 110. At this time, the multilingual conversation server 120 may transmit an action ID result value for processing a voice command to the media playback device 110.

2 is a block diagram of a media playback apparatus according to an embodiment of the present invention. Referring to FIG. 2, the media playback device 110 includes an input unit 210, a language recognition unit 220, a classification unit 230, an utterance guide sentence generation unit 240, a conversion unit 250, and a communication unit 260. , A language feature analysis unit 270, a voice conversion unit 280 and an output unit 290.

The input unit 210 may receive a voice command from the user 100. For example, the input unit 210 may receive a foreign language voice command such as “GiGa Genenie, Turn the all lights off”.

The language recognition unit 220 may analyze the input voice command to recognize the language spoken by the user 100. At this time, the language recognition unit 220 automatically recognizes the language spoken by the user 100 using at least one automatic language recognition algorithm among neural networks, pattern recognition such as a Hidden Markov model, and deep learning-based natural language processing. You can. For example, the language recognition unit 220 may analyze “GiGa Genenie, Turn the all lights off” to recognize that the language spoken by the user 100 is “English”.

The classification unit 230 may determine the sentence structure of the voice command based on the recognized language, and classify the vocabulary proficiency according to the determined sentence structure. For example, the classification unit 230 determines parts of speech (eg, nouns, verbs, adjectives, adverbs, etc.) of at least one word included in the sentence composition of the voice command, and categorizes the used vocabulary for each determined part of speech. It is possible to map the ratings for each part of speech determined according to the categorized use vocabulary, and to determine the vocabulary propensity of the user 100 based on the ratings for each part of speech. The process of determining the vocabulary propensity of the user 100 will be described in detail with reference to FIG. 3.

3 is an exemplary diagram for describing a process of classifying a user's vocabulary based on a user's language in a media playback apparatus according to an embodiment of the present invention. In FIG. 3, it was assumed that the user 100 uttered the sentence "Give me linen amenities."

The classification unit 230 may determine the part-of-speech name 310 for each spoken word 300 included in the sentence configuration of the voice command. For example, the classification unit 230 determines the part of speech of 'give' as' verb '311, the part of speech of' me 'as' pronoun' 312, and the part of speech of 'linen' as' noun It can be determined as' (313), and the part of speech of 'amenities' can be determined as' noun' (314).

The classification unit 230 may categorize the vocabulary used for each part of speech. At this time, the classification unit 230 may categorize the used vocabulary for each part of speech by using the vocabulary class dictionary database. For example, the classification unit 230 may categorize the 'verb' 311 to include 'give' and the 'noun' (313, 314) to include 'linen, amenities'.

The classification unit 230 may map a grade for each part of speech determined according to the categorized use vocabulary. Each part-of-speech may be mapped to class A, for example, a class B to a noun, and a class C to an adjective. At this time, conjunctions, pronouns, prepositions, interjections, and articles can be excluded from the class mapping. For example, the classification unit 230 maps the grades of the 'verb' 311 to the 'A' 321, and the grades of the 'noun' (313, 314) to the 'B' (322, 323), respectively. Can be mapped.

The classification unit 230 may determine the vocabulary proficiency 320 of the user 100 based on the grade mapped for each part of speech. For example, in the classification unit 230, the 'verb' 311 is composed of one 'A' (321) grade, and the 'noun' (313, 314) is two 'B' (322, 323) grades When configured as, it may determine the vocabulary inclination of the user 100 as a 'B' type.

Referring back to FIG. 2, the utterance guide sentence generation unit 240 may generate a utterance guide sentence in a language spoken by the user 100 based on the vocabulary proficiency classified by the classification unit 230.

The utterance guide sentence generation unit 240 may select a vocabulary based on the classified vocabulary proficiency, and convert the utterance guide sentence into the language of the recognized user 100 based on the selected vocabulary.

The utterance guide sentence generation unit 240 may derive voice command information based on the frequency of use of the voice command uttered by the user 100, and may select the utterance list sentence based on the derived voice command information. For example, the utterance guide sentence generation unit 240 derives voice command information corresponding to the top 10% based on the frequency of use of the voice command spoken by the user 100, and speaks based on the derived voice command information List sentences can be selected. At this time, the utterance guide sentence generation unit 240 may reconstruct the vocabulary of the selected utterance list sentences based on the vocabulary use propensity of the user 100.

The utterance guide sentence generation unit 240 may convert a utterance list sentence in which the vocabulary is reconstructed into a language of the recognized user 100 based on a preset mapping table for each language. At this time, if there is no preset mapping table for each language, language translation may be performed through a translator module (not shown). Here, the translator module may be a web translator or a separately developed translator. The process of converting the speech list sentence into the language of the recognized user 100 based on the preset mapping table for each language will be described in detail with reference to FIGS. 4A and 4B.

4A and 4B are exemplary diagrams for explaining a process of converting a speech list sentence into a recognized user's language in a media playback apparatus according to an embodiment of the present invention.

4A is an exemplary diagram for explaining a process of converting a utterance list sentence into a language of a recognized user based on a preset mapping table for each language in a media playback apparatus according to an embodiment of the present invention. Referring to FIG. 4A, the utterance guide sentence generation unit 240 may convert a utterance list sentence into a language of the recognized user 100 based on a preset language-specific mapping table by searching a language conversion database.

The utterance guide sentence generation unit 240 may convert the utterance list sentence 400 into the language of the recognized user 100 based on the preset mapping table 410. For example, if the language of the recognized user 100 is English 420, the utterance guide sentence generation unit 240 sets the utterance list sentence 400 to "Turn on the TV", "Play the music", " How is the weather? "," What time is it now? " And so on.

4B is an exemplary diagram for explaining a process of converting a utterance list sentence into a language of a user recognized through a translator module in a media playback apparatus according to an embodiment of the present invention. Referring to FIG. 4B, the utterance guide sentence generation unit 240 separates the translator module 430 when text language information corresponding to the language-specific mapping table 410 in which the utterance list sentence 400 is preset does not exist. Through the utterance list sentence can be converted to the language of the user (100).

For example, if the language of the recognized user 100 is 'German' 440, but the 'German' 440 does not exist in the preset mapping table, the utterance guide sentence generation unit 240 is the utterance list sentence By transmitting the 400 to a separate translator module 430, the utterance list sentence 400 converted to 'German' 440 through the translator module 430 may be received.

Returning to FIG. 2 again, the conversion unit 250 may convert outputs of the user interface of the media playback device 110 based on the recognized language and classified vocabulary propensity. Also, the converter 250 may convert the text output on the display 115 of the media playback device 110 into a recognized language format by referring to the classified vocabulary propensity. For example, when the recognized language of the user 100 is 'English', the conversion unit 250 outputs the user interface displayed as 'Korean' based on the vocabulary propensity of the user 100 Can be converted to 'English'.

The conversion unit 250 may convert the output information for the user interface based on the inferred accent style.

The communication unit 260 may transmit a voice command to the multilingual conversation server 120 based on the recognized language of the user 100 and receive an analysis result of the voice command from the multilingual conversation server 120. At this time, the communication unit 260 may receive an action ID result value for processing a voice command from the multilingual conversation server 120.

The language feature analysis unit 270 extracts language feature information for the accent and accent of the user 100 from the input voice command, and analyzes the accent style for the voice command through a pattern for the extracted language feature information. have. For example, the language feature analysis unit 270 converts the input voice command into a simplified pitch symbol based on the ToBI, Tones and Break Indices (ToBI), and the converted pitch based on a preset speech sequence. The symbols may be analyzed, and the intonation style for the voice command may be inferred based on the similarity between the analyzed pitch symbol pattern and the language-specific intonation pattern stored in the database. The process of inferring the accent style for the voice command will be described in detail with reference to FIG. 5.

5 is an exemplary diagram for explaining a process of inferring an accent style for a user's voice command in a media playback apparatus according to an embodiment of the present invention. Referring to FIG. 5, the language feature analysis unit 270 may convert a voice command into a simplified pitch symbol based on the ruler notation convention notation. The simplified pitch symbol 510 may be composed of a high tone (H), a medium tone (M), a low tone (L), and a high and low accent (+).

For example, when the user 100 utters the voice command “TURN THE AIR CONDITIONER UP 10 DEGREE.” 500, the language feature analysis unit 270 may display the same as 'LL + ML + HHM' 520. Can be converted to a simplified pitch symbol.

Returning to 2 again, the voice conversion unit 280 may synthesize the analysis result of the voice command received from the multilingual conversation server 120 with the accent style of the user 100.

When the output unit 290 converts the output items for the user interface in the conversion unit 250, the output unit 290 may dynamically output the user interface by applying the converted output items. For example, the output unit 290 may output the utterance guide sentence generated to induce a voice command in the language of the user 100 on the display 150. In addition, when the output unit 290 converts the text output on the display 115 of the media playback device 110 by the conversion unit 250 into a recognized language format by referring to the classified vocabulary propensity, the converted text Can be output to the display 115. For another example, the output unit 290 may output the analysis result of the voice command synthesized with the accent style of the user 100 through audio.

6A to 6C are exemplary views illustrating a user interface of a media playback device in which output items are converted based on a user's language according to an embodiment of the present invention.

6A is an exemplary diagram illustrating a Korean user interface according to an embodiment of the present invention. Referring to FIG. 6A, the media playback device 110 displays the utterance guide sentence in the first area 610 through the Korean user interface screen 600, and displays life information in the second area 620, The sub-menu may be displayed in the third area 630, external interlocking messages and terminal setting information may be displayed in the fourth area 640, and language setting information and control information may be displayed in the fifth area 650.

As for the utterance guide sentence, any one of the utterance guide list sentences may be displayed on the display 115 based on the vocabulary propensity of the user 100. For example, in the utterance guide list sentence, when the vocabulary propensity of the user 100 is classified as 'A type', “Geniya, turn off all lights” is displayed on the display, and the user 100's vocabulary propensity is When classified as 'B type', "Gniya. Turn off all lights" is displayed on the display, and when the vocabulary of the user 100 is classified as 'C type', "Gniya, turn off all lights" Can be displayed on the display. For example, when the vocabulary inclination of the user 100 is classified as a 'C' type, the media playback device 110 may display a utterance guide sentence, "Gniya. Turn off all lights" on the display.

6B is an exemplary diagram illustrating an English user interface according to an embodiment of the present invention. Referring to FIG. 6B, when the language of the user 100 is recognized as 'English', the media playback device 110 may convert and display a Korean user interface screen into an English type. The media playback device 110 displays the utterance guide sentence in the first area 610 in English through the user interface screen 600 converted to the English type, and displays the living information in the second area 620 in English. , Displays the sub-menu in English in the third area 630, displays external linkage messages and terminal setting information in English in the fourth area 640, and displays language setting information and control information in the fifth area 650. Can be displayed in English.

6C is an exemplary diagram illustrating a user interface to which a user's language, vocabulary inclination, and accent style are applied based on a user's voice command according to an embodiment of the present invention. Referring to FIG. 6C, when the user 100 utters the voice command “turn on all room lights 660”, the media playback device 110 recognizes that the language used by the user 100 is 'English' Then, the vocabulary inclination of the user 100 may be classified as 'A type', and the accent pattern of the user 100 may be analyzed.

For example, since the recognized language of the user 100 is 'English', the media playback device 110 converts the user interface to English and outputs it, and displays a utterance guide sentence according to the vocabulary proficiency of the user 100 can do. At this time, the media playback device 110 synthesizes the analysis result (eg, “All room lights are now turned off (670)”) of the user 100 with the accent style of the user 100 You can output through audio.

7 is a flowchart of a method of providing a multilingual voice command service in a media playback apparatus according to an embodiment of the present invention. The method for providing a multilingual voice command service in the media playback device 110 shown in FIG. 7 is time-sequentially processed by the multilingual voice command service providing system 1 according to the embodiment shown in FIGS. 1 to 6C. Includes Therefore, even if omitted, the media playback apparatus 110 according to the embodiment shown in FIGS. 1 to 6C is also applied to a method of providing a multilingual voice command service.

In step S710, the media playback device 110 may receive a voice command from the user 100.

In step S720, the media playback device 110 may recognize the language spoken by the user 100 by analyzing the input voice command.

In step S730, the media playback device 110 may determine the sentence structure of the voice command based on the recognized language, and classify the vocabulary proficiency according to the determined sentence structure.

In step S740, the media playback device 110 may convert outputs of the user interface of the media playback device 110 based on the recognized language and classified vocabulary propensity.

In step S750, the media playback device 110 may dynamically output the user interface by applying the converted output.

In the above description, steps S710 to S750 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted if necessary, and the order between the steps may be switched.

The method of providing a multilingual voice command service in the media playback apparatus described with reference to FIGS. 1 to 7 may also be implemented in the form of a computer program stored in a medium executed by a computer or a recording medium including instructions executable by a computer. You can. In addition, the method for providing a multilingual voice command service in the media playback apparatus described with reference to FIGS. 1 to 7 may also be implemented in the form of a computer program stored in a medium executed by a computer.

Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer readable media may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above description of the present invention is for illustration only, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and it should be interpreted that all changes or modified forms derived from the meaning and scope of the claims and equivalent concepts thereof are included in the scope of the present invention. do.

110: media playback device
115: display
120: multilingual chat server
210: input
220: language recognition unit
230: classification
240: utterance guide sentence generation unit
250: conversion unit
260: communication unit
270: language feature analysis unit
280: speech converter
290: output

Claims (20)

A media playback device that provides a multilingual voice command service,
An input unit that receives a voice command from a user;
A language recognition unit that analyzes the input voice command and recognizes a language spoken by the user;
A classification unit determining a sentence configuration of the voice command based on the recognized language, and classifying a vocabulary proficiency according to the determined sentence configuration;
A conversion unit configured to convert outputs to the user interface of the media playback device based on the recognized language and the classified vocabulary proficiency; and
And an output unit that dynamically outputs the user interface by applying the converted output information.
According to claim 1,
The classification unit determines a part-of-speech of at least one word included in the sentence configuration of the voice command, categorizes a used vocabulary for each determined part-of-speech, and maps a rating for each determined part-of-speech according to the categorized used vocabulary And determining a user's vocabulary propensity based on the level mapped for each part of speech.
According to claim 1,
A utterance guide sentence generation unit that generates a utterance guide sentence in a language spoken by the user based on the classified vocabulary propensity
Further comprising,
And the output unit outputs the generated utterance guide sentence to a display.
The method of claim 3,
The speech guide sentence generator selects a vocabulary based on the classified vocabulary proficiency, and converts the speech guide sentence into the recognized user's language based on the selected vocabulary.
The method of claim 4,
The utterance guide sentence generation unit derives voice command information based on the frequency of use of the voice command spoken by the user,
Sentence list sentences are selected based on the derived voice command information,
And reconstructing the vocabulary of the selected speech list sentence based on the user's vocabulary propensity.
The method of claim 5,
The speech guide sentence generation unit converts the speech list sentence in which the vocabulary is reconstructed into a language of the recognized user based on a preset mapping table for each language.
According to claim 1,
Transmitting the voice command to a multilingual conversation server based on the recognized user's language,
And a communication unit configured to receive an analysis result of the voice command from the multilingual conversation server.
The method of claim 7,
Further comprising a voice conversion unit for synthesizing the analysis result of the voice command received from the multilingual conversation server with the accent style of the user,
The output unit is to output the analysis result of the voice command synthesized with the accent style of the user through audio, media playback device.
According to claim 1,
Further comprising a language feature analysis unit for extracting the language feature information for the accent and accent of the user from the input voice command, and analyzing the accent style for the voice command through a pattern for the extracted language feature information, Media playback device.
The method of claim 9,
The language feature analysis unit converts the input voice command into a pitch symbol based on a rhythm notation convention notation, analyzes the converted pitch symbol based on a preset speech sequence, and stores the pattern and the previously stored pattern of the analyzed pitch symbol. Infer the intonation style for the voice command based on the similarity with the intonation pattern for each language,
And the conversion unit converts the output information based on the inferred accent style.
According to claim 1,
The conversion unit converts the text output on the display of the media playback device into the recognized language format with reference to the classified vocabulary proficiency,
And the output unit outputs the converted text to the display.
A method for providing a multilingual voice command service in a media playback device,
Receiving a voice command from a user;
Analyzing the input voice command to recognize a language spoken by the user;
Determining a sentence structure of the voice command based on the recognized language, and classifying a vocabulary propensity according to the determined sentence structure;
Converting outputs for a user interface of the media player based on the recognized language and the classified vocabulary proficiency; And
And dynamically outputting the user interface by applying the converted output information.
The method of claim 12,
The classifying may include determining a part-of-speech of at least one word included in the sentence configuration of the voice command;
Categorizing the used vocabulary for each of the determined parts of speech;
Mapping a grade for each of the determined parts of speech according to the categorized use vocabulary; And
And determining a user's vocabulary propensity based on the grade mapped for each part of speech.
The method of claim 12,
Generating an utterance guide sentence in a language spoken by the user based on the classified vocabulary propensity; And
Displaying the generated utterance guide sentence on a display
The method further comprising, providing a multilingual voice command service.
The method of claim 14,
Generating the utterance guide sentence may include selecting a vocabulary based on the classified vocabulary propensity; And
A method of providing a multilingual voice command service, wherein the utterance guide sentence is converted into the recognized user's language based on the selected vocabulary.
The method of claim 15,
The generating of the spoken guide sentence may include deriving voice command information based on the frequency of use of the voice command spoken by the user;
Selecting a utterance list sentence based on the derived voice command information;
Reconfiguring the vocabulary of the selected utterance list sentence based on the user's vocabulary propensity; And
And converting the speech list sentence in which the vocabulary is reconstructed into a language of the recognized user based on a preset mapping table for each language.
The method of claim 12,
Transmitting the voice command to a multilingual conversation server based on the recognized user's language information; And
Receiving an analysis result of the voice command from the multilingual conversation server;
Synthesizing an analysis result of the voice command received from the multilingual conversation server with the accent style of the user; And
And outputting an analysis result of the speech style synthesized with the user's intonation style through audio.
The method of claim 12,
Extracting language feature information for the accent and accent of the user from the input voice command; And
And analyzing an accent style for the voice command through a pattern for the extracted language feature information.
The method of claim 18,
Converting the input voice command into a pitch symbol based on a leap notation convention notation;
Analyzing the converted pitch symbol based on a preset speech sequence; And
Inferring an accent style for the voice command based on the similarity between the analyzed pitch symbol pattern and the pre-stored accent pattern for each language; And
Transforming the output based on the inferred accent style further
Further comprising, a method for providing a multilingual voice command service.
A computer program stored on a computer readable medium comprising a sequence of instructions for providing a multilingual voice command service through a media playback device,
When the computer program is executed by a computing device,
Receiving voice commands from the user,
Recognize the language spoken by the user by analyzing the input voice command,
Determine the sentence structure of the voice command based on the recognized user's language,
Classifying vocabulary proficiency according to the determined sentence composition,
Convert outputs to the user interface of the media playback device based on the recognized language and the classified vocabulary propensity,
And a sequence of instructions for dynamically outputting the user interface by applying the converted output.

Images