KR102145370B1 - Media play device and method for controlling screen and server for analyzing screen - Google Patents

Abstract

The media playback device for controlling the screen includes an input unit for receiving a voice command uttered from a user, a voice command transmission unit for transmitting the input voice command to a voice recognition server, and text generated based on the voice command from the voice recognition server. A text information receiving unit that receives information, a screen state analysis unit that analyzes a menu screen displayed on the media playback device, a screen information transmission that transmits the received text information and screen state information on the analyzed menu screen to a screen analysis server And a receiving unit for receiving a control command generated based on the text information and the screen state information from the screen analysis server, and an execution unit for controlling the menu screen based on the received control command.

Description

Media playback device that controls the screen, method, and server that analyzes the screen {MEDIA PLAY DEVICE AND METHOD FOR CONTROLLING SCREEN AND SERVER FOR ANALYZING SCREEN}

The present invention relates to a media playback apparatus and a method for controlling a screen, and a server for analyzing the screen.

IPTV (Internet Protocol Television) is an interactive broadcasting and communication service that provides video content such as movies and broadcast programs and various multimedia contents such as Internet search through a high-speed Internet network to a television receiver. Viewers can simply search the Internet using a remote control and receive various contents and additional services provided by the Internet, such as watching movies, home shopping, home banking, online games, and MP3s, from IPTV.

Recently, IPTV provides an IPTV service through voice recognition spoken by a user instead of a remote control. In relation to this service, Korean Patent Publication No. 2011-0027362, a prior art, discloses an IPTV system and service method using a voice interface. have.

However, conventionally, when an IPTV attempts to access a menu intended by a user through voice recognition, the IPTV cannot determine which screen the user has spoken in, so it must access the relevant content or the corresponding menu through conditional text search. . For this reason, it has a disadvantage in that it takes a lot of time to access the menu or contents intended by the user.

An object of the present invention is to provide a media playback apparatus, a method, and a server that analyzes a screen that controls a screen to move to a lower menu screen suitable for the user's intention by comparing and analyzing a voice command spoken by a user with a currently displayed menu screen. An object of the present invention is to provide a media playback apparatus, a method, and a server that analyzes a screen that controls a screen that selects and displays content suitable for a user's intention by comparing and analyzing a voice command spoken by a user and a currently displayed menu screen.

In the past, the user could reach the desired menu only when the user spoke the caller + command repeatedly, but after uttering one caller, the command was continuously uttered to control the screen that allows the user to quickly reach the desired menu. It is intended to provide a server that analyzes a media playback device, a method, and a screen to be played.

After controlling the menu screen according to the user's voice command, when an additional voice command is uttered from the user, the log is extracted and managed, thereby controlling the screen that provides customized content recommendation and menu organization based on the user's history. It is intended to provide a server that analyzes devices, methods and screens. However, the technical problem to be achieved by the present embodiment is not limited to the technical problems as described above, and other technical problems may exist.

As a means for achieving the above-described technical problem, an embodiment of the present invention provides an input unit for receiving a voice command uttered from a user, a voice command transmission unit for transmitting the input voice command to a voice recognition server, and the voice recognition A text information receiving unit that receives text information generated based on the voice command from a server, a screen state analysis unit that analyzes a menu screen displayed on the media playback device, a screen state of the received text information and the analyzed menu screen A screen information transmission unit for transmitting information to a screen analysis server, a receiving unit for receiving a control command generated based on the text information and the screen state information from the screen analysis server, and the menu screen based on the received control command. It is possible to provide a media playback device including a controlling execution unit.

Another embodiment of the present invention is an input unit for receiving a voice command uttered from a user, a screen information transmission unit for transmitting the input voice command and screen status information on a menu screen displayed on the media player to a screen analysis server, It is possible to provide a media playback device including a receiver configured to receive the voice command and a control command generated based on the screen state information from the screen analysis server, and a controller configured to control the menu screen based on the received control command. have.

In another embodiment of the present invention, receiving a voice command uttered from a user, transmitting the input voice command to a voice recognition server, and receiving text information generated based on the voice command from the voice recognition server. Receiving, analyzing a menu screen displayed on the media playback device, transmitting the received text information and screen status information on the analyzed menu screen to a screen analysis server, the text information from the screen analysis server And receiving a control command generated based on the screen state information and controlling the menu screen based on the received control command.

In another embodiment of the present invention, when a media playback device receives a voice command uttered from a user, text information converted from the voice command from the media playback device and a screen state for a menu screen displayed on the media playback device It is possible to provide a menu screen analysis server including a screen status information receiving unit for receiving information and a transmission unit for generating a control command corresponding to the voice command on the menu screen and transmitting it to the media player based on the screen status information. have.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present invention. In addition to the above-described exemplary embodiments, there may be additional embodiments described in the drawings and detailed description of the invention.

According to any one of the above-described problem solving means of the present invention, a media playback device for controlling a screen to move to a lower menu screen suitable for the user's intention by comparing and analyzing a voice command uttered from a user and a currently displayed menu screen, A server that analyzes the method and screen may be provided. A media playback device, a method, and a server that analyzes a screen for controlling a screen for selecting and displaying content suitable for a user's intention by comparing and analyzing a voice command spoken by a user and a currently displayed menu screen may be provided.

In the past, the user could reach the desired menu only when the user spoke the caller + command repeatedly, but after uttering one caller, the command was continuously uttered to control the screen that allows the user to quickly reach the desired menu. It is possible to provide a server that analyzes a media playback device, a method, and a screen.

After controlling the menu screen according to the user's voice command, when an additional voice command is uttered from the user, the log is extracted and managed, thereby controlling the screen that provides customized content recommendation and menu organization based on the user's history. A server that analyzes devices, methods, and screens may be provided.

1 is a block diagram of a screen control system according to an embodiment of the present invention.
2 is a block diagram of a media playback device according to an embodiment of the present invention.
3 is an exemplary diagram showing a screen that induces a user's continuous speech for screen control according to an embodiment of the present invention.
4 is an exemplary diagram illustrating a screen for selecting numbered content through a user's voice utterance according to an embodiment of the present invention.
5 is a flow chart of a method for controlling a screen in a media playback device according to an embodiment of the present invention.
6 is a flowchart of a method for controlling a screen in a media playback device according to another embodiment of the present invention.
7 is a block diagram of a screen analysis server according to an embodiment of the present invention.
8A and 8B are exemplary diagrams illustrating a menu screen from which text is extracted from an image when text information is included in an image constituting screen data according to an embodiment of the present invention.
9 is a flowchart of a method for analyzing a screen in a screen analysis server according to an embodiment of the present invention.
10A to 10C are exemplary views for explaining a process of controlling a media playback device according to an embodiment of the present invention to move from a main screen to a screen corresponding to a newest movie category according to a user's intention.
11A to 11C are exemplary diagrams for explaining a process of controlling a media playback device according to an embodiment of the present invention to move from a latest movie category to a screen corresponding to movie content according to a user's intention.
12A and 12B are exemplary diagrams for explaining a process of controlling a screen according to continuous speech input from a user in a media playback device according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" with another part, this includes not only "directly connected" but also "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included, and one or more other features, not excluding other components, unless specifically stated to the contrary. It is to be understood that it does not preclude the presence or addition of any number, step, action, component, part, or combination thereof.

In the present specification, the term "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized by using two or more hardware, or two or more units may be realized by one piece of hardware.

In this specification, some of the operations or functions described as being performed by the terminal or device may be performed instead by a server connected to the terminal or device. Likewise, some of the operations or functions described as being performed by the server may also be performed by a terminal or device connected to the server.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a screen control system according to an embodiment of the present invention. Referring to FIG. 1, the screen control system 1 may include a media playback device 110, a display device 115, a voice recognition server 120, and a screen analysis server 130. The media playback device 110, the display device 115, the voice recognition server 120, and the screen analysis server 130 exemplarily illustrate components that can be controlled by the screen control system 1.

Each component of the screen control system 1 of FIG. 1 is generally connected through a network. For example, as shown in FIG. 1, the media playback device 110 may be connected to the voice recognition server 120 or the screen analysis server 130 at the same time or at intervals of time.

Network refers to a connection structure that enables information exchange between nodes such as terminals and servers, and is a local area network (LAN), a wide area network (WAN), and the Internet (WWW: World). Wide Web), wired and wireless data communication networks, telephone networks, wired and wireless television networks, etc. Examples of wireless data communication networks include 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Bluetooth communication, infrared communication, and ultrasound. Communication, Visible Light Communication (VLC), LiFi, etc. are included, but are not limited thereto.

According to an embodiment, the media playback device 110, the voice recognition server 120, and the screen analysis server 130 may process continuous commands through continuous standby. In the continuous standby, whether or not to execute a command is determined according to a result of processing a previous command, and after the user 100 utters a call word, it is possible to control the menu screen by waiting for input of a voice command without re-firing the call word.

The media playback device 110 may receive a voice command uttered from the user 100.

The media playback device 110 may transmit the input voice command to the voice recognition server 120 and receive text information generated based on the voice command from the voice recognition server 120.

The media playback device 110 may analyze a menu screen displayed on the media playback device 110. When the analysis of the menu screen is completed, the media playback device 110 may transmit the received text information and screen status information on the analyzed menu screen to the screen analysis server 130. The screen status information includes a menu ID, category ID, content ID, and screen component ID related to a menu screen, and each ID may be mapped to a menu name, category name, content name, and screen component name. Thereafter, the media playback device 110 may receive a control command generated based on text information and screen state information from the screen analysis server 130 and control a menu screen based on the received control command. At this time, the media playback device 110 may further transmit the screen data displayed on the media playback device 110 to the screen analysis server 130, and control commands related to the menu screen and screen data from the screen analysis server 130 If not received, the media playback device 110 may transmit the received text information to the speech recognition server 120 and may receive a search result corresponding to the text information from the speech recognition server 120.

When the media playback device 110 does not display the menu screen, the media playback device 110 transmits the received text information to the voice recognition server 120 and searches for text information from the voice recognition server 120. You can receive the results.

The display device 115 may display a menu screen for providing TV or VOD.

When the media playback device 110 receives a control command corresponding to the voice command of the user 100 from the screen analysis server 130, the display device 115 displays a menu screen moved according to the control command, or Can be displayed by playing back.

The voice recognition server 120 receives a voice file including a voice command uttered by the user 100 from the media playback device 110, extracts text information from the received voice file, and sends the extracted text information to the media playback device. Can be transmitted to (110).

The screen analysis server 130 may receive metadata related to a menu screen from a menu management server (not shown), and may store and update a mapping table for the menu screen. Here, the menu screen may be composed of a menu list, a category list, and a content list, and the menu list may be mapped to a menu ID and menu name, a category list may be a category ID and category name, and a content list may be mapped to a content ID and a content name. . Here, the content detail screen among the menu screens may include a purchase button for each quality (low/high quality), a video viewed together, a person search, a trailer/additional video, more ratings, and more plots.

When it is determined that text information corresponding to the screen component of the menu screen is included in the image constituting the screen data, the screen analysis server 130 extracts the text from the image and calculates the extracted text and the ID of the screen component. You can update the mapping table by mapping.

The screen analysis server 130 may additionally manage through a mapping table by assigning a separate ID in relation to a screen configuration menu such as a detailed content screen and settings in addition to the organization.

The screen analysis server 130 may manage a mapping table according to a product to which the media playback device 110 subscribes. For example, depending on whether the media playback device 110 subscribes to Mass/Biz as a basic product, IPTV single product/satellite-linked IPTV product, or subscribes to a channel/VOD monthly fee as an additional product, the media playback device 110 provides Since the menu screen may be different, the screen analysis server 130 may manage a mapping table for a menu screen according to a product to which the media player 110 subscribes.

The screen analysis server 130 may receive text information converted from a voice command from the media playback device 110 and screen status information on a menu screen displayed on the media playback device 110. The screen analysis server 130 may determine whether the voice command is a control command for a menu screen based on the received text information and screen state information. At this time, if the media playback device 110 receives a call word (eg, GiGA Genie) from the user 100, the screen analysis server 130 processes the media playback device 110 and the menu screen at a high speed. A view session for can be created.

The screen analysis server 130 may receive screen data from the media playback device.

The screen analysis server 130 determines whether the voice command is a control command for the menu screen based on the screen state information, generates a control command corresponding to the voice command on the menu screen, and transmits the control command to the media playback device 110. I can. For example, the screen analysis server 130 extracts at least one of a menu ID, a category ID, a content ID, and a screen component ID corresponding to text information received from the media playback device 110, and based on the extracted ID. Accordingly, a control command for a menu corresponding to the voice command on the menu screen may be transmitted to the media playback device 110.

After the screen analysis server 130 transmits a control command for a menu corresponding to the voice command to the media playback device 110, the media playback device 110 and the screen analysis server 130 perform continuous standby to perform a user ( 100), the text information corresponding to the voice command and the screen status information are continuously received, the ID corresponding to the received text information is extracted, and the control command for the menu corresponding to the voice command from the menu screen based on the extracted ID The process of transmitting the video to the media playback device 110 may be repeatedly performed. In this case, if the ID corresponding to the text information received from the media playback device 110 is not extracted, the media playback device 110 and the screen analysis server 130 may not perform continuous standby.

According to another embodiment, the media playback device 110, the voice recognition server 120, and the screen analysis server 130 may process continuous commands through continuous speech. In the continuous speech, after the user 100 utters a voice command, a session between the screen analysis server 130 and the voice recognition server 120 is created and maintained, thereby processing the continuous command to control the menu screen.

The media playback device 110 may receive a voice command uttered from the user 100 and transmit the input voice command and screen status information on a menu screen displayed on the media playback device 110 to the screen analysis server 130. have.

The media playback device 110 may receive a voice command and a control command generated based on the screen state information from the screen analysis server 130 and control a menu screen based on the received control command.

The media playback device 110 may receive and display speech guide information for the voice command based on the voice command and text information displayed on the menu screen from the screen analysis server 130. For example, the media playback device 110 may display a plurality of utterance guide information in either an overlay method or a rolling method.

The voice recognition server 120 may receive a hint derived from the voice command and screen state information from the screen analysis server 130 when the media playback device 110 allows continuous speech of the voice command.

The voice recognition server 120 may perform voice recognition based on a voice command and hint, and transmit the recognized text to the screen analysis server 130.

The screen analysis server 130 may receive a voice command from the media playback device 110 and screen status information on a menu screen displayed on the media playback device 110. At this time, when the screen analysis server 130 receives the voice command and screen status information on the menu screen from the media playback device 110, continuous speech of the voice command between the screen analysis server 130 and the voice recognition server 120 An ASR session may be created to process the data at high speed.

The screen analysis server 130 transmits a voice command received from the media playback device 110 and a hint derived from the screen state information of the media playback device 110 to the voice recognition server 120, and the voice recognition server 120 From, it is possible to receive text in order of accuracy.

The screen analysis server 130 may determine whether the voice command is a control command for a menu screen of the media player 110 based on the received recognition text and screen state information. At this time, the screen analysis server 130 determines whether the voice command is a control command for the menu screen based on the recognized text and screen state information, and generates a control command corresponding to the voice command on the menu screen to generate a media playback device ( 110).

2 is a block diagram of a media playback device according to an embodiment of the present invention. Referring to FIG. 2, the media playback device 110 includes an input unit 210, a transmission unit 220, a text information reception unit 230, a screen state analysis unit 240, a screen information transmission unit 250, and a reception unit 260. ), a control unit 270 and a display unit 280.

According to an embodiment, the media playback device 110 may process a continuous command through continuous standby. In the continuous standby, whether or not to execute a command is determined according to a result of processing a previous command, and after the user 100 utters a call word, it is possible to control the menu screen by waiting for input of a voice command without re-firing the call word.

The input unit 210 may receive a voice command uttered from the user 100. For example, the input unit 210 may receive a call word uttered from the user 100 (eg, GiGA Genie) and then receive a voice command such as "Move to the latest movie" from the user 100. have.

The transmission unit 220 may transmit the input voice command to the voice recognition server 120.

When the media player 110 does not display the menu screen, the transmission unit 220 may transmit the converted text information to the voice recognition server 120.

When the control command related to the menu screen and screen data is not received from the screen analysis server 130, the transmission unit 220 may transmit the received text information to the voice recognition server 120.

The text information receiving unit 230 may receive text information generated based on a voice command from the voice recognition server 120. Here, the text information may be generated from a voice command by the voice recognition server 120 through Speech To Text (STT).

The screen state analysis unit 240 may analyze a menu screen displayed on the media playback device 110. For example, the screen state analysis unit 240 may analyze whether a menu screen currently displayed in the media playback device 110 is a main menu screen, a category screen, or a content screen.

The screen information transmission unit 250 may transmit the received text information and screen state information on the analyzed menu screen to the screen analysis server 130. The screen status information includes a menu ID, category ID, content ID, and screen component ID related to a menu screen, and each ID may be mapped to a menu name, category name, content name, and screen component name.

The screen information transmission unit 250 may transmit screen data displayed on the media playback device 110 to the screen analysis server 130. Screen data may include, for example, html, javascript, css, image URL, hash tag, data, and the like.

The receiver 260 may receive a control command generated based on text information and screen state information from the screen analysis server 130. The control command may include, for example, moving a menu, selecting a menu or content, executing content, or the like.

The receiving unit 260 may receive a search result corresponding to text information from the voice recognition server 120 when the media playback device 110 does not display the menu screen.

When the control command related to the menu screen and screen data is not received from the screen analysis server 130, the receiving unit 260 may receive a search result corresponding to text information from the voice recognition server 120.

The controller 270 may control the menu screen based on the received control command.

According to another embodiment, the media playback device 110 may receive continuous speech from the user 100 and process a continuous command. In the continuous speech, after the user 100 utters a voice command, a session is created and maintained between the screen analysis server 130 and the voice recognition server 120, so that the menu screen can be controlled by processing the continuous command.

The input unit 210 may receive a voice command uttered from the user 100.

The screen information transmission unit 250 may transmit an input voice command and screen state information on a menu screen displayed on the media playback device 110 to the screen analysis server 130.

At this time, if the screen information transmission unit 250 transmits the voice command and screen status information on the menu screen displayed on the media playback device 110 to the screen analysis server 130, the screen analysis server 130 and the voice recognition server Between 120 sessions may be created.

The receiver 260 may receive a voice command and a control command generated based on the screen state information from the screen analysis server 130. In this case, the control command may be generated based on a voice command transmitted from the screen analysis server 130 to the voice recognition server 120 through a session and a hint derived from screen state information.

The receiver 260 may receive speech guide information for the voice command from the screen analysis server 130 based on the voice command and text information displayed on the menu screen.

The controller 270 may control the menu screen based on the received control command.

The display unit 280 may display utterance guide information. For example, the display unit 280 may display a plurality of ignition guide information in either an overlay method or a rolling method.

3 is an exemplary diagram showing a screen that induces a user's continuous speech for screen control according to an embodiment of the present invention. Referring to FIG. 3, the media playback device 110 may display speech guide information to induce the user 100 to continuously speak.

For example, when the user 100 utters the call word "giga genie", the media playback device 110 may indicate that the user 100's voice is being input through the microphone 310 display. In addition, when the page word spoken by the user 100 is recognized, the media playback device 110 may display a message 320 such as "Yes, I am listening".

The media playback device 110 may allow the user 100 to access a final menu or content desired by the user 100 by displaying speech guide information that induces continuous speech of a command. The categories include Today's Recommendations, Movies/Series, TV Replays, Kids Land, Animation, Music/Education/Documentation, TV Apps for a certain amount, and burning from TV replays to sub-menus, entry to Cannes Film Festival, legal dramas, In the case of including popular animations, VIP benefit zones, popular TOP 100, today's events, and new customer star zones, the media playback device 110 displays the user 100 as "Review TV" -> "VIP Benefit Zone". The utterance guide information 300 that induces continuous utterance of may be displayed.

4 is an exemplary diagram illustrating a screen for selecting numbered content through a user's voice utterance according to an embodiment of the present invention. Referring to FIG. 4, in a state that the menu screen displayed on the media playback device 110 is a'movie/series' screen, a'popular TOP 10' 400 corresponding to a sub-menu of'movie/series' from the user 100 ) Is received as a voice command, the media playback device 110 may display a menu screen for the'popular TOP 10' 400. In this case, the media playback device 110 may number and display a plurality of contents 410 corresponding to the'popular TOP 10' 400, respectively.

For example, when the media player 110 receives a voice command "Play it once" from the user 100 while the current'popular TOP 10' 400 menu screen is displayed, the media player 110 ) May play a'champion' corresponding to the content of'No. 1'411 or may move the menu screen to the detail screen.

As described above, the reason why numbers or alphabets are given to each of the plurality of contents is that when the contents name of the plurality of contents is long, the media playback device 110 cannot display all the contents names on the screen, so that the user moves a menu or selects the contents. This is to provide convenience for

5 is a flow chart of a method for controlling a screen in a media playback device according to an embodiment of the present invention. A method of controlling a screen in the media playback device 110 illustrated in FIG. 5 includes steps processed in a time series by the screen control system 1 according to the exemplary embodiment illustrated in FIGS. 1 to 4. Accordingly, even if the contents are omitted below, it is also applied to the method of controlling the screen in the media playback device 110 according to the exemplary embodiment illustrated in FIGS.

In operation S510, the media playback device 110 may receive an input of a voice command uttered from the user 100.

In step S520, the media playback device 110 may transmit the input voice command to the voice recognition server 120.

In operation S530, the media playback device 110 may receive text information generated based on a voice command from the voice recognition server 120.

In operation S540, the media playback device 110 may analyze a menu screen displayed on the media playback device 110.

In operation S550, the media playback device 110 may transmit the received text information and screen state information on the analyzed menu screen to the screen analysis server 130.

In operation S560, the media playback device 110 may receive a control command generated based on text information and screen state information from the screen analysis server 130.

In operation S570, the media playback device 110 may control the menu screen based on the received control command.

In the above description, steps S510 to S570 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, or the order of steps may be switched.

6 is a flowchart of a method for controlling a screen in a media playback device according to another embodiment of the present invention. A method of controlling a screen in the media playback apparatus 110 shown in FIG. 6 includes steps processed in a time series by the screen control system 1 according to the exemplary embodiment shown in FIGS. 1 to 5. Accordingly, even if the contents are omitted below, it is also applied to a method of controlling a screen in the media playback device 110 according to the exemplary embodiment illustrated in FIGS. 1 to 5.

In step S610, the media playback device 110 may receive a voice command uttered from the user 100.

In operation S620, the media playback device 110 may transmit the input voice command and screen status information on the menu screen displayed on the media playback device 110 to the screen analysis server 130.

In operation S630, the media playback device 110 may receive a voice command and a control command generated based on the screen state information from the screen analysis server 130.

In step S640, the media playback device 110 may control the menu screen based on the received control command.

In the above description, steps S610 to S640 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, or the order of steps may be switched.

7 is a block diagram of a screen analysis server according to an embodiment of the present invention. Referring to FIG. 7, the screen analysis server 130 includes a screen state information receiving unit 710, an extraction unit 720, a determination unit 730, a transmission unit 740, a text extraction unit 750, and a mapping unit 760. ) And a management unit 770 may be included.

According to an embodiment, the screen analysis server 130 may process a continuous command through continuous standby with the media playback device 110. In the continuous standby, whether or not to execute a command is determined according to a result of processing a previous command, and after the user 100 utters a call word, it is possible to control the menu screen by waiting for input of a voice command without re-firing the call word.

When the media playback device 110 receives a voice command uttered from the user 100, the screen state information receiver 710 receives text information converted from the voice command from the media playback device 110 and a menu displayed on the media playback device. You can receive screen status information about the screen.

The screen state information receiving unit 710 may receive screen data from the media playback device 110. Screen data may include, for example, html, javascript, css, image URL, hash tag, data, and the like.

The extraction unit 720 extracts at least one of a menu ID, a category ID, a content ID, and a screen component ID corresponding to text information on a voice command from among screen status information on a menu screen received from the media playback device 110 can do.

The determination unit 730 may determine whether the voice command is a control command for a menu screen based on the received text information and screen state information. Specifically, the determination unit 730 may determine whether the voice command is a control command for the menu screen by analyzing each ID extracted corresponding to the text information among the screen state information. In this case, when it is determined that the voice command is a control command for the menu screen, the determination unit 730 may generate a control command corresponding to the voice command.

The transmission unit 740 may transmit a control command corresponding to the voice command on the menu screen to the media playback device 110 based on the screen state information. The control command may include, for example, moving a menu, selecting a menu or content, executing content, or the like.

When the media playback device 110 receives a continuous speech command from the user 100, the transmission unit 740 transmits a control command for the menu on the menu screen based on the continuous speech command. ). Here, the menu screen is composed of a menu list, a category list, and a content list, the menu list may be a menu ID and a menu name, the category list may be a category ID and a category name, and the content list may be mapped to a content ID and a content name, respectively. .

The transmission unit 740 may transmit a control command for a menu corresponding to the voice command on the menu screen to the media playback device 110 on the basis of the ID extracted by the extraction unit 720.

When it is determined that text information corresponding to a screen component of a menu screen is included in an image constituting the screen data, the text extracting unit 750 may extract text from the image.

The mapping unit 760 may map the extracted text and the ID of the screen component.

When an additional command is input from the user 100 after the menu screen is moved to a lower menu based on a control command from the media playback device 110, the management unit 770 may collect and manage logs for the additional command. . In addition, when an additional menu movement is received from the user 100 through a remote control, the management unit 770 may collect and manage a log of movement of the additional menu input through the remote control. Here, the additional command or the additional menu movement may occur as, for example, when duplicate text exists in the menu screen, the menu movement is processed differently according to the user's priority, or the corresponding text exists on the menu screen. In spite of this, it may occur as the user wants general search results.

The management unit 770 may determine the collected log as an utterance command case of a type not intended by the user and separately manage the collected log through an exception command set for each user so that the same type of command processing does not occur.

Based on the logs collected through this process, customized content based on user history can be recommended and applied to menu organization.

According to another embodiment, the screen analysis server 130 may process continuous commands through continuous speech. In the continuous speech, after the user 100 utters the voice command, a voice session between the screen analysis server 130 and the voice recognition server 120 is created and maintained, thereby processing the continuous command to control the menu screen. In this case, the screen analysis server 130 may further include a screen state analysis unit (not shown).

The screen status information receiver 710 may receive a voice command uttered by the user 100 from the media playback device 110 and screen status information on a menu screen displayed on the media playback device 110. At this time, when the screen status information receiving unit 710 receives the voice command and the screen status information for the menu screen from the media playback device 110, the voice command is continued between the screen analysis server 130 and the voice recognition server 120 An ASR session can be created to process the speech at high speed.

The screen state information receiver 710 may receive a voice command from the voice recognition server 120 and a predetermined number of recognized texts recognized from the screen state information according to accuracy.

The extraction unit 720 may extract at least one of a menu ID, a category ID, a content ID, and a screen component ID corresponding to the recognized text from the screen state information.

The determination unit 730 may determine whether the voice command is a control command for a menu screen based on the received recognition text and screen state information. Specifically, the determination unit 730 may determine whether the voice command is a control command for a menu screen by analyzing each ID extracted corresponding to the recognized text among the screen state information. In this case, when it is determined that the voice command is a control command for the menu screen, the determination unit 730 may generate a control command corresponding to the voice command.

The transmission unit 740 may transmit a voice command and a hint derived from screen state information to the voice recognition server 120 through an ASR session. In this case, the hint may be a text set exposed on the current menu screen.

The transmission unit 740 may transmit a control command generated based on a voice command and screen state information to the media playback device 110. For example, the transmission unit 740 may transmit a control command for a sub-menu corresponding to a voice command on the menu screen to the media playback device 110 based on the extracted ID.

8A and 8B are exemplary diagrams illustrating a menu screen from which text is extracted from an image when text information is included in an image constituting screen data according to an embodiment of the present invention. 8A and 8B, when it is determined that text information corresponding to a screen component of a menu screen is included in the images 800 and 810 constituting the screen data, the screen analysis server 130 , 810). In this case, the images 800 and 810 may be poster images, and when text information is included in the images 800 and 810 constituting the screen data, it means that the content ID exists but the content name does not exist. I can.

The screen analysis server 130 requests the original image from an external server (not shown) that stores images 800 and 810 based on the image URL information, and OCR for the original image received from the external server (not shown). Optical Character Reader) function can be used to extract text information from the original image, map the extracted text to the ID of the screen component, and store it.

Thereafter, when the media playback device 110 receives a voice command "The Huntsman" from the user 100, the media playback device 110 plays the "The Huntsman" 801 or displays a menu on the detail screen. You can move the screen.

9 is a flowchart of a method for analyzing a screen in a screen analysis server according to an embodiment of the present invention. A method of analyzing a screen in the screen analysis server 130 illustrated in FIG. 9 includes steps processed in time series by the screen control system 1 according to the exemplary embodiment illustrated in FIGS. 1 to 8B. Therefore, even if the contents are omitted below, it is also applied to the method of analyzing the screen in the screen analysis server 130 according to the exemplary embodiment illustrated in FIGS. 1 to 8B.

In step S910, when the screen analysis server 130 receives a voice command uttered from the user 100 in the media playback device 110, the text information converted from the voice command from the media playback device 110 and the media playback device ( Screen status information on the menu screen displayed in 110) may be received.

In step S920, the screen analysis server 130 may determine whether the voice command is a control command for a menu screen based on the received text information and screen state information.

In step S930, the screen analysis server 130 may transmit a control command corresponding to the voice command on the menu screen to the media playback device 110 based on the screen state information.

In the above description, steps S910 to S930 may be further divided into additional steps or may be combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, or the order of steps may be switched.

10A to 10C are exemplary views for explaining a process of controlling a media playback device according to an embodiment of the present invention to move from a main screen to a screen corresponding to a newest movie category according to a user's intention.

10A is an exemplary diagram for explaining a process of receiving a voice command for moving to a menu intended by a user in a media playback device according to an embodiment of the present invention. Referring to FIG. 10A, the media playback device 110 displays the main menu screen 1000 and the user 100 displays the "Show Movie Series 1010 menu" and "Latest Movie" along with the caller "Gi Genie". As shown in "1020, a continuous voice command may be input. In this case, the final menu intended by the user 100 may be a'Latest Movie' 1020 category.

10B is a diagram showing a search result according to a voice command in a conventional media player. Referring to FIGS. 10A and 10B, in a conventional media player, a search result 1030 for a voice command of'latest movie' received from a user is searched with a message "I will search for a recently obtained movie." The number of results and the searched movies are displayed in a line. As described above, the search result according to the conventional voice command has a disadvantage in that the user causes a selection through an additional voice command (eg, selection of all views).

10C is an exemplary diagram illustrating a search result according to a voice command in the media playback device according to an embodiment of the present invention as the result of FIG. 10A. Referring to FIGS. 10A and 10C, the media playback device 110 allows the user 100 to recognize a plurality of contents 1040 included in the'latest movie' 1020 at a glance. 1040) can be arranged in a checkerboard format. In this case, the user 100 may control the scroll through a voice command or a remote control to select the content desired by the user 100.

11A to 11C are exemplary diagrams for explaining a process of controlling a media playback device according to an embodiment of the present invention to move from a latest movie category to a screen corresponding to movie content according to a user's intention.

11A is an exemplary diagram for explaining a process of receiving a voice command for moving to a content intended by a user in a media playback device according to an embodiment of the present invention. Referring to FIG. 11A, the media playback device 110 displays the'Movie/Series' screen 1100, and the user 100 displays the "Show the latest movie menu" with the caller "Gi Genie" (1110) and Like the "Dongju" 1120, a continuous voice command may be input. In this case, the final menu intended by the user 100 may be a content called'dongju' 1120.

11B is a diagram illustrating a search result according to a voice command in a conventional media player. Referring to FIGS. 11A and 11B, a conventional media player performs a full search of a menu for a voice command “Dongju” received from a user, and a message “This is a Dongju search result” as a search result 1120. And the number of search results and the searched movies are displayed in a line. Since the conventional media player cannot determine the situation on which screen the user uttered the voice command on, the corresponding content or menu can be accessed only through a full search of the menu through text search.

11C is an exemplary diagram illustrating a search result according to a voice command in the media playback device according to an embodiment of the present invention as the result of FIG. 11A. 11A and 11C, the media player 110 moves to the'Latest Movie' 1110 menu corresponding to the voice command uttered by the user 100 among the'Movie/Series' 1000 menus. , In order for the user 100 to recognize the plurality of contents 1140 included in the'latest movie' 1110 at a glance, the plurality of contents 1140 may be arranged in a checkerboard format. In this case, the media playback device 110 may display a thick frame on the'dongju' 1150 among the plurality of contents 1140 displayed on the screen.

12A and 12B are exemplary diagrams for explaining a process of controlling a screen according to continuous speech input from a user in a media playback device according to an embodiment of the present invention.

12A is a diagram illustrating a process of moving from a main menu to a content page based on a voice command input from a user in a conventional media player. Referring to FIG. 12A, a user may utter a call word “giga genie” on the main menu screen 1200. At this time, the media player displays a message saying "Yes, I'm listening" and prepares to receive a voice command from the user through the microphone icon.

When the user utters the voice command “Find Dongju”, the media playback device performs a full search for the menu based on the text “Dongju” and arranges a plurality of contents corresponding to the search result 1210 in a line. Indicate.

Thereafter, when the user wants to execute the content “Dongju”, the user re-initiates the call word “Gi Genie”, and then the number numbered in the content desired by the user among the plurality of contents included in the search result 1210 "No. 1" (1220) corresponding to is ignited.

The media playback device 110 moves to and displays a detail page of the content “Dongju” corresponding to “No. 1” 1120.

As such, conventionally, in order to move from the main menu to the content, the user uttered a'call word + command' (Giga Genie + Find Dongju), and a command such as'call word + command' (Giga Genie + execute 1 time). The caller must be uttered before the utterance of

12B is an exemplary diagram illustrating a process of moving from a main menu to a content page based on a voice command input from a user according to an embodiment of the present invention. Referring to FIG. 12B, the user 100 may utter a call word “giga genie” on the main menu screen 1240. At this time, the media playback device 110 displays a message "Yes, I'm listening", and prepares to receive a voice command from the user 100 through the microphone icon.

When the user 100 utters a voice command “movie/series movement”, the media playback device 110 moves to the “movie/series” menu screen 1250 and displays the “movie/series” menu 1250. When the user 100 among the sub-menus utters a voice command “Move to the latest movie”, the media playback device 110 moves to the “Latest Movie” menu screen 1260 and displays the “Latest Movie” menu 1260. When the user 100 utters a voice command "execution of play" among a plurality of included contents, the media playback device 110 may select and execute the "playing" 1270 of the plurality of contents.

The method of controlling a screen in the media playback device described with reference to FIGS. 1 to 12B and the method of analyzing the screen in a screen analysis server include a computer program stored in a medium executed by a computer or a computer-executable instruction. It can also be implemented in the form of a medium. In addition, the method of controlling the screen in the media playback device described with reference to FIGS. 1 to 12B and the method of analyzing the screen in the screen analysis server may be implemented in the form of a computer program stored in a medium executed by a computer.

Computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Further, the computer-readable medium may include a computer storage medium. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that other specific forms can be easily modified without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative and non-limiting in all respects. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

110: media playback device
115: display device
120: speech recognition server
130: screen analysis server
210: input unit
220: voice command transmission unit
230: text information receiver
240: screen state analysis unit
250: screen information transmission unit
260: receiver
270: executive
280: display
710: screen status information receiving unit
720: extraction unit
730: judgment unit
740: transmission unit
750: text extraction unit
760: mapping unit
770: administration

Claims (17)

In the media playback device for controlling the screen,
An input unit receiving a voice command spoken by a user;
A voice command transmission unit for transmitting the input voice command to a voice recognition server;
A text information receiver configured to receive text information generated based on the voice command from the voice recognition server;
A screen state analysis unit that analyzes a menu screen displayed on the media playback device;
A screen information transmission unit that transmits the received text information and screen state information on the analyzed menu screen to a screen analysis server;
A receiver configured to receive a control command generated based on the text information and the screen state information from the screen analysis server; And
Including an execution unit for controlling the menu screen based on the received control command,
The screen status information includes at least one of a menu ID, a category ID, a content ID, and a screen component ID related to the menu screen,
The control command is generated based on at least one ID extracted from the screen state information corresponding to the text information.
The method of claim 1,
When the media playback device does not display the menu screen,
The transmission unit transmits the received text information to the speech recognition server,
Wherein the receiving unit receives a search result corresponding to the text information from the speech recognition server.
The method of claim 1,
Each of the IDs is mapped to a menu name, a category name, a content name, and a screen component name.
The method of claim 3,
The screen information transmission unit further transmits the screen data displayed on the media playback device to the screen analysis server.
The method of claim 4,
When a control command related to the menu screen and screen data is not received from the screen analysis server,
The transmission unit transmits the received text information to the speech recognition server,
Wherein the receiving unit receives a search result corresponding to the text information from the speech recognition server.
In the media playback device for controlling the screen,
An input unit receiving a voice command spoken by a user;
A screen information transmission unit for transmitting the input voice command and screen state information on a menu screen displayed on the media playback device to a screen analysis server;
A receiver configured to receive a control command generated based on the voice command and the screen state information from the screen analysis server; And
Including a control unit for controlling the menu screen based on the received control command,
The screen status information includes at least one of a menu ID, a category ID, a content ID, and a screen component ID related to the menu screen,
The control command is generated based on at least one ID extracted from the screen state information corresponding to text information generated based on the voice command.
The method of claim 6,
Wherein the receiving unit receives speech guide information for the voice command based on the voice command and text information displayed on the menu screen from the screen analysis server.
The method of claim 7,
Further comprising a display for displaying the utterance guide information,
Wherein the display unit displays the utterance guide information in either an overlay method or a rolling method.
The method of claim 7,
A session is created between the screen analysis server and the voice recognition server,
Wherein the control command is generated based on the voice command transmitted from the screen analysis server to the voice recognition server through the session and a hint derived from the screen state information.
In a method of controlling a screen in a media playback device,
Receiving a voice command spoken from a user;
Transmitting the input voice command to a voice recognition server;
Receiving text information generated based on the voice command from the voice recognition server;
Analyzing a menu screen displayed on the media playback device;
Transmitting the received text information and screen status information on the analyzed menu screen to a screen analysis server;
Receiving a control command generated based on the text information and the screen state information from the screen analysis server; And
Including the step of controlling the menu screen based on the received control command,
The screen status information includes at least one of a menu ID, a category ID, a content ID, and a screen component ID related to the menu screen,
The control command is generated based on at least one ID extracted from the screen state information corresponding to the text information.
In the server that analyzes the screen,
A screen status information receiver configured to receive text information converted from the voice command from the media playback device and screen status information on a menu screen displayed on the media playback device when the media playback device receives a voice command uttered from a user; And
A transmission unit that generates a control command corresponding to the voice command on the menu screen based on the screen state information and transmits it to the media player
Including,
The screen status information includes at least one of a menu ID, a category ID, a content ID, and a screen component ID related to the menu screen,
Further comprising an extraction unit for extracting at least one of a menu ID, a category ID, a content ID, and a screen component ID corresponding to the text information from the screen state information,
The control command is generated based on at least one ID extracted from the screen state information corresponding to the text information.
The method of claim 11,
When the media playback device receives a continuous speech command from the user, the transmission unit transmits a control command for a submenu of the menu screen to the media playback device based on the continuous speech command Screen analysis server.
The method of claim 11,
The menu screen is composed of a menu list, a category list and a content list,
The menu list is a menu ID and a menu name, the category list is a category ID and a category name, and the content list is a content ID and a content name respectively mapped.
The method of claim 11,
The screen state information receiving unit further receives screen data from the media playback device, the menu screen analysis server.
The method of claim 14,
A text extracting unit for extracting text from the image when it is determined that text information corresponding to a screen component of the menu screen is included in the image constituting the screen data;
And a mapping unit for mapping the extracted text and the ID of the screen component.
The method of claim 15,
And the transmission unit transmits a control command for a sub-menu corresponding to the voice command on the menu screen to the media player based on the extracted ID.
The method of claim 12,
When the media playback device receives an additional command from the user after the menu screen is moved to the sub-menu based on the control command, the menu further comprises a management unit that collects and manages a log of the additional command Screen analysis server.

Images