KR102237832B1 - Display apparatus for performing a voice control and method thereof - Google Patents

Abstract

Disclosed is a voice control method. The present method includes: waiting for input of the user's voice when the voice input mode is started, converting the user voice to text, a control step of performing a control operation corresponding to the text, and a user utterance based on the result of the control operation. A determining step of determining whether the user utterance is complete, and if it is determined that the user utterance is incomplete, waiting for input of a subsequent user's voice for a preset waiting time, and releasing the voice input mode when it is determined that the user utterance is complete. Accordingly, efficient voice control can be performed.

Description

Display device performing voice control and its voice control method {DISPLAY APPARATUS FOR PERFORMING A VOICE CONTROL AND METHOD THEREOF}

The present invention relates to a display device and a voice control method thereof, and more particularly, to a display device for releasing a voice control mode by detecting a user's utterance completion time, and a voice control method thereof.

With the development of electronic technology, various types of electronic devices are being developed and distributed. In addition, functions provided by electronic devices are also gradually diversifying.

Accordingly, efforts to develop interaction technologies that enable users to use electronic devices more conveniently have been steadily continued. One of them may be a voice control technology that controls using a user's voice.

Recently, various electronic devices such as TVs, mobile phones, and navigation devices have been implemented to recognize a user's voice and perform a control operation corresponding to the voice.

However, there were some difficulties in applying the voice control technology to a product and using it in real life. For example, while a user inputs a voice, ambient noise is input, or conversation contents in daily life are input, and thus, there is a possibility that a malfunction may be performed.

Accordingly, there has been a need for a technology that can more efficiently and accurately perform a technology for controlling through voice input.

The present invention is in accordance with this necessity, and an object of the present invention is to provide a display device and a voice control method for canceling a voice input mode according to a result of the determination by determining whether a user's utterance has been completed.

According to an embodiment of the present invention for achieving the above object, the voice control method includes the steps of waiting for input of a user's voice when a voice input mode is started, and when the user voice is input, the user's voice is converted into text. Converting, a control step of performing a control operation corresponding to the text, a determination step of determining whether the user utterance is complete based on the execution result of the control operation, and when the user utterance is determined to be incomplete, during a preset waiting time A step of waiting for input of a subsequent user's voice, and releasing the voice input mode when it is determined that the user's utterance has been completed.

Here, in the determining step, if the control operation is an operation in which information selected in at least one search process performed by the user voice or the subsequent user voice is displayed, it may be determined that the user utterance is complete. .

Alternatively, in the determining step, if the control operation is a multimedia content reproduction operation, it may be determined that the user utterance is completed.

In addition, the voice control method may further include displaying a graphic object corresponding to the voice input mode while the voice input mode is started and maintained.

Meanwhile, a display device according to an embodiment of the present invention includes an input unit receiving a user voice when a voice input mode is started, and a control unit performing a control operation corresponding to the user voice input through the input unit.

Here, the control unit determines whether the user utterance is complete based on a result of the control operation, and when it is determined that the user utterance is incomplete, waits for input of a subsequent user's voice for a preset waiting time, and the user utterance is When it is determined that it is complete, the voice input mode may be canceled.

In addition, while the voice input mode is started and maintained, a display unit for displaying a graphic object corresponding to the voice input mode may be further included.

Meanwhile, when the user voice is input, the controller may display a text corresponding to the user voice on one side of the graphic object.

In addition, when the voice input mode is released, the controller may change the shape of the graphic object into a shape indicating the release state of the voice input mode.

In addition, the control unit performs a search at least once or more according to the user's voice or the subsequent user's voice to display the search result on the display unit, and one of the search results is selected to display information according to the selected search result. If so, it can be determined that the user's utterance is complete.

Alternatively, the control unit may determine that the user utterance has been completed if the control operation is a multimedia content reproduction operation.

Meanwhile, the display device may further include a communication unit that communicates with a voice recognition device for converting the user's voice into text. In this case, the control unit transmits the user voice to the voice recognition device through the communication unit, and when the voice recognition device converts the user voice to the text, the control operation corresponding to the text is received by receiving the text. You can do it.

Alternatively, the display device may further include a communication unit for performing communication with a server device for performing a search according to the text. In this case, the control unit transmits the text to the server device, receives a search result according to the user's voice from the server device, and displays it on the display unit, and when a utterance completion signal is received from the server device, the user utterance It is determined that is completed, and the voice input mode can be canceled.

Alternatively, the display device may further include a communication unit for performing communication with a voice recognition device for converting the user's voice into the text and a server device for performing a search according to the text.

In this case, the control unit transmits the user's voice to the speech recognition device, receives the text from the speech recognition device, and transmits the text to the server device through the communication unit to receive the search result from the server device. It is received and displayed on the display unit, and when an utterance completion signal is received from the server device, it is determined that the user utterance is completed, and the voice input mode may be canceled.

Meanwhile, according to another embodiment of the present invention, the server device includes a server communication unit that receives a text corresponding to a user's voice from a display device operating in a voice input mode, a database storing content information, and a database storing the content information. And a server controller configured to perform a search, provide the searched content information to the display device, and transmit a utterance completion signal to the display device when it is determined that the user's utterance is complete based on the search result.

According to the various embodiments of the present invention as described above, the voice control system may determine whether the user's utterance has been completed, and accordingly, automatically cancel the voice input mode. Accordingly, there is no need to repeatedly press the voice input mode selection and release button repeatedly, and the risk of malfunction due to ambient noise or daily conversation can be greatly reduced.

1 is a block diagram showing a configuration of a display device according to an embodiment of the present invention;
2 is a flowchart illustrating a voice control method according to an embodiment of the present invention;
3 is a diagram for explaining a voice control process of a display device;
4 and 5 are diagrams for explaining a method of determining whether a user's utterance is complete;
6 is a diagram showing the configuration of a voice control system according to an embodiment of the present invention;
7 is a block diagram showing a configuration of a display device according to another embodiment of the present invention;
8 is a block diagram showing the configuration of a server device according to an embodiment of the present invention;
9 is a flowchart illustrating a method of supporting voice control of a server device according to an embodiment of the present invention.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a configuration of a display device according to an embodiment of the present invention. Referring to FIG. 1, the display device 100 includes an input unit 110 and a control unit 120.

The input unit 110 is a component for receiving a user's voice. The input unit 110 may directly receive a user's voice including a microphone by itself, or may indirectly receive a user's voice input from a microphone installed on a remote control or the like from the remote control.

The control unit 120 performs a control operation corresponding to the user's voice input through the input unit 110. Specifically, when the user voice input through the input unit 110 is a predefined voice command, the controller 120 may perform an operation corresponding to the voice command. For example, when the display apparatus 100 is a TV, voice commands may be designated and stored for various operations such as turn-on, turn-off, volume up, volume down, channel up, channel down, and the like. When such a voice command is input, the controller 120 performs an operation matching the voice command.

On the other hand, if the user's voice has not been defined by pronouncing an arbitrary text, the controller 120 performs an operation corresponding to the text. That is, the display apparatus 100 may analyze a user's voice and provide an interactive service in response to the user's voice. For example, when the user pronounces "movie actor ABCD", the display apparatus 100 may search for various pieces of information including the text "movie actor ABCD" and display the search result.

Although the control unit 120 may directly process the user's voice analysis and text conversion, depending on the embodiment, differently, by transmitting the user voice to an external server device (not shown), the server device transmits the user's voice. When converted to text, the converted text may be provided. The server device that converts speech into text may be referred to as a speech recognition device for convenience of description. An embodiment of converting speech into text in conjunction with a speech recognition device will be described in detail later.

The controller 120 may initiate a voice input mode according to a user's selection. When the voice input mode is started, the controller 120 may activate the input unit 110 to receive a user's voice. In this state, after the user's voice is input and the control operation is performed, the controller 120 determines whether or not the user's speech is completed according to the execution result. The completion of the user's utterance means a state in which the user is predicted to no longer use the voice control.

That is, the control unit 120 may determine whether the utterance is complete according to the characteristics of the control operation performed according to the user's voice. For example, in general, the user speaks a voice to search for certain information, or speaks a voice to execute a certain function. Accordingly, the user finally finds and confirms the desired information, or does not speak the voice for a certain period of time after executing the desired function. In consideration of this point, when the user selects one piece of information in the process of searching for information and the information is displayed, the controller 120 may determine that the user's utterance is complete. Alternatively, even when a specific application or multimedia content is executed and the execution screen is displayed, the control unit 120 may determine that the user utterance is complete.

As a result of this determination, when it is determined that the user's speech is completed, the controller 120 immediately releases the voice input mode without waiting for a waiting time. Accordingly, the possibility of malfunction due to various noises generated in the surrounding area during an unnecessary waiting time can be prevented in advance.

On the other hand, if it is determined that the user's utterance is incomplete, the controller 120 waits for input of a subsequent user's voice for a preset waiting time. When a subsequent user's voice is input again within the waiting time, the controller 120 performs a control operation according to the subsequent user's voice again, and determines once again whether the user's utterance is completed due to the subsequent user's voice. If it is determined that the voice of the subsequent user is also incomplete, the controller 120 initializes the waiting time again and waits for another voice input of the subsequent user during the waiting time. That is, after initially selecting the voice input mode only once, the controller 120 can continue to maintain the voice input mode even if the user does not select the voice input mode again until the user completes the utterance. Accordingly, it is possible to solve the hassle of a user having to select a voice input mode every time for voice control.

In FIG. 1, it has been described that the display device independently performs voice control, but voice control may be performed in conjunction with an external server device.

2 is a flowchart illustrating a voice control method according to an embodiment of the present invention. Referring to FIG. 2, when the voice input mode is started (S210), the display apparatus 100 waits for a user voice input (S220). As described above, the user's voice may be input through a microphone provided in the main body of the display apparatus 100, or may be input through a microphone provided in a remote control or other external device and transmitted to the display apparatus 100.

When the user's voice is input, the display device 100 (S230). The user's voice is converted into text (S240). Text conversion may be performed by the display device 100 itself, but according to another embodiment, the display device 100 transmits a user's voice to an external server device, that is, a voice recognition device. When the user's voice is converted to text, the text can be received.

The display apparatus 100 performs a control operation corresponding to the converted text (S250). For example, if the user utters a keyword, the display apparatus 100 may search for various pieces of information including the keyword and display the search result. Alternatively, when the user utters a keyword and a command together, an application corresponding to the command may be executed, and the keyword may be provided as an input of the application. For example, if a command (for example, play, play, etc.) that commands to play the content with a specific content name is fired, the content playback application is executed, and the corresponding content name is provided as an input of the content playback application. do. Accordingly, content corresponding to the content name is played.

After performing the control operation, the display apparatus 100 may determine whether the user has completed the utterance based on the content of the control operation (S260). This determination may be performed by the display device 100 itself, or may be determined by an external server device and notified to the display device 100.

A method of determining whether the user utterance is complete may be implemented in various ways. For example, in the determining step, when the control operation is an operation of displaying information selected in at least one search process performed by a user voice or a subsequent user voice, it may be determined that the user utterance is complete.

Alternatively, in the determination step, when the control operation performed by the user's voice or the subsequent user's voice is the multimedia content playback operation, it may be determined that the user's utterance is complete.

An example of a specific determination method will be described with reference to the drawings in a later part.

When the display apparatus 100 determines that the speech is completed, the voice input mode is canceled (S290). When the voice input mode is released, control by the user's voice is not performed unless the user selects the voice input mode again.

On the other hand, if it is determined that the user's utterance is incomplete, the display apparatus 100 resets the waiting time (S280), maintains the voice input mode for the waiting time, and waits for the user's voice input (S220). The waiting time may be arbitrarily set by a manufacturer or a user of the display device 100. For example, it can be set to about 30 seconds.

Even when the waiting time ends without the user's voice being input (S270), the display apparatus 100 may cancel the voice input mode.

Meanwhile, although not shown in FIG. 2, while the voice input mode is started and maintained, the step of displaying a graphic object corresponding to the voice input mode may be further included. The shape of the graphic object and its role will be described in detail in a part to be described later.

3 is a diagram for describing an operation of a display device controlled using a user's voice. Referring to FIG. 3, the display device 100 may further include a display unit 130.

Various contents selected by the user may be displayed on the display unit 130. In addition, when the voice input mode is started, a graphic object 320 corresponding to the voice input mode is displayed on the display unit 130. The graphic object 320 may be continuously displayed while the voice input mode is maintained. In FIG. 3, the graphic object 320 in the shape of a microphone is displayed on the edge of the screen, but the shape and position of the graphic object 320 are not limited thereto.

When a user's voice is input, the controller 120 displays a text corresponding to the user's voice on the display unit 130. Referring to FIG. 3, a text display area 310 may be provided on one side of the graphic object 320.

Accordingly, text corresponding to the user's voice may be displayed in the text display area 310. The user may check the text displayed in the text display area 310 to check whether or not the voice uttered by the user is normally recognized.

A result display area 330 for displaying a result of a control operation corresponding to a user's voice is provided below the text display area 310. The controller 120 displays the result of the control operation corresponding to the user's voice in the result display area 330.

In FIG. 3, an example in which the user 10 utters "AAA several fields" is shown. In this case, the controller 120 indicates a state in which the search has been performed using keywords "AAA" and "What time". The controller 120 displays the search result 330 in the result display area 330.

The controller 120 determines whether or not the user's utterance has been completed based on the search result 330. When it is determined that the utterance is complete, the controller 120 releases the voice input mode.

4 and 5 are diagrams for explaining various methods of determining whether a user's utterance is complete. 4 shows a state in which the user has uttered the keyword "AAA". The controller 120 may perform a search for "AAA" and display the search result on the result display area 330. 4 shows a state in which a number of search results are displayed. Since the plurality of search results are searched and displayed in a list format, the controller 120 may determine that the user utterance is incomplete and reset the waiting time back to an initial value.

In this state, the user can select one of the search results by inputting a subsequent user's voice. In FIG. 4, the user may directly utter an index (ie, No. 1, No. 2, No. 3, etc.) in which the search results are sorted, or may utter a name of the search result. When a user voice such as "No. 1" or "Review AAA news" is input, the controller 120 performs an additional search based on the input user voice. Accordingly, the controller 120 displays "Review AAA News" on the text display area 310 and displays the search result for "Review AAA News" on the result display area 330. Even in this case, the controller 120 may determine that the user utterance is incomplete because a plurality of search results are searched and displayed in a list form. The controller 120 resets the waiting time to an initial value again, and waits for a subsequent user's voice.

As shown in FIG. 4, when the user again utters "3 times" or "AAA news 9/3 date", the controller 120 plays multimedia content corresponding to the item selected by the user's voice. 4 shows a state in which a multimedia content called AAA News as of September 3 is played and the playback screen 410 is displayed. In this way, when the control operation executed by the user's voice or the subsequent user's voice is the multimedia content reproduction operation, the control unit 120 determines that the user's utterance has been completed.

Accordingly, the controller 120 releases the voice input mode, and transforms the shape of the graphic object into a shape (S340) indicating the release state of the voice input mode. The graphic object may be deleted after a certain period of time after the voice input mode is released.

In this way, when the content desired by the user is output, the voice input mode is immediately released, and the UI for the voice input mode is deleted. Accordingly, even if the content is selected and output in the related art, the content is covered by the UI during the waiting time, and the inconvenience of disturbing the viewing can be solved.

5 is a diagram for explaining another method of determining whether or not an utterance is complete. In FIG. 5, a case where the user utters "AAA several fields" is described as an example.

The controller 120 performs a search based on the user's voice and displays the search result in the result display area 330 in the form of a list. The user may input a subsequent user's voice and select a search result to perform an additional search within the search result. 5 shows a state in which item 1 and item 2 are sequentially selected.

When information of the lowest level is selected on the information tree, the controller 120 may display the corresponding information. 5 shows a case where the item "AAA 10 o'clock news information" is information of the lowest level. When "AAA 10 o'clock news information" is selected by a subsequent user's voice, the controller 120 displays a detailed information screen 510 corresponding to the selected item.

If the information of the lowest level is selected, the controller 120 determines that the user's utterance has been completed. That is, the control unit 120 performs at least one search according to the user's voice or the subsequent user's voice, and when one of the search results is selected and information according to the selected search result is displayed, it can be determined that the user utterance is complete. have. Accordingly, the controller 120 releases the voice input mode.

When the voice input mode is released, the control unit 120 transforms the graphic object 320 into a shape 340 indicating a voice input mode release state.

As another example, the controller 120 may determine that the user utterance is complete even when an application is selected and the application screen is displayed, and the lowest function is selected and executed in the application screen.

4 and 5 illustrate that the search results are displayed in the form of a list arranged in the vertical direction, the search results may be provided in various ways according to the shape and size of the display unit 130. For example, they may be arranged in a horizontal direction. In addition, the search result may be displayed in various forms such as a captured image, a thumbnail image, an icon, and the like, not in a text form.

As described above, while performing a control operation according to the user's voice, the controller 120 determines whether the user will not continue speaking anymore or whether a subsequent utterance is expected based on the content of the user's voice and the content of the control operation. It can be judged automatically. The controller 120 may automatically release the voice input mode according to the determination result, thereby preventing a risk of malfunction. In addition, as described above, inconvenience of obscuring the screen due to the UI in the voice input mode can be prevented, and unnecessary waste of resources can be reduced.

Meanwhile, in the above-described embodiments, it has been described that the display device converts the user's voice into text, performs a search according to the text, and performs all of the user's intent recognition, but some of these operations are performed by an external server device. can do. That is, according to another embodiment of the present invention, in a voice control system including at least one server device and a display device, the voice control method as described above may be implemented.

6 shows an example of a configuration of a voice control system according to an embodiment of the present invention. Referring to FIG. 6, the voice control system 1000 includes a voice recognition device 700, a server device 800, and a display device 100.

The display device 100 may include a chat client module (not shown) capable of interworking with the voice recognition device 700 and the server device 800. When the voice input mode is started, the controller 120 may execute a conversation client module to perform a control operation corresponding to the voice input. Specifically, when a user's voice is input, the controller 120 may transmit it to the voice recognition apparatus 700.

The voice recognition device 700 refers to a type of server device that converts and provides a user voice transmitted through the display device 100 into text.

The speech recognition apparatus 700 recognizes speech using at least one of various recognition algorithms such as a dynamic time warping method, a hidden Markov model, and a neural network, and recognizes the recognized speech. Can be converted to text. For example, in the case of using the hidden Markov model, the speech recognition apparatus 700 models a temporal change and a spectral change of a user's voice, respectively, and detects similar vocabulary in a pre-stored language database. Accordingly, the detected vocabulary can be output as text.

When text is input from the speech recognition device 700, the display device 100 performs a control operation corresponding to the input text. If there is a voice command that matches text among preset voice commands, the display apparatus 100 performs an operation corresponding to the voice command. On the other hand, if the voice command and the text do not match, the text is provided to the server device 800.

The server device 800 searches for information corresponding to the provided text from its own database or other server devices. The server device 800 feeds back the search result to the display device 100.

The display device 100 displays a search result. As described above, the display apparatus 100 may determine the user's utterance intention based on the search result. Accordingly, when the speech is completed, the voice input mode can be automatically canceled.

Meanwhile, according to another embodiment of the present invention, whether or not the user's utterance is completed may be determined by the server device 800. That is, if the server device 800 provides a search result by searching according to the user's voice or subsequent voice, the user selects lower level information and checks the screen according to the information, or if the content is played by selecting the content. , It can be determined that the user's utterance has been completed. Accordingly, the server device 800 may transmit an utterance completion signal to the display device 100.

The display device 100 may cancel the voice input mode when the utterance completion signal is received from the server device 800.

7 is a block diagram illustrating an example of a detailed configuration of a display device used in the voice input system as shown in FIG. 6. Referring to FIG. 7, the display device 100 includes an input unit 110, a control unit 120, a display unit 130, a broadcast reception unit 140, a content processing unit 150, a communication unit 160, and a storage unit 170. Includes.

The input unit 110 is a component for receiving a user's voice as described above. The control unit 120 performs a control operation corresponding to the user's voice input through the input unit 110. Since the operation of the input unit 110 and the control unit 120 has been described in detail in the above-described part, a description of the overlapping part will be omitted.

The broadcast receiver 140 is a component for selecting a broadcast channel and receiving content through the broadcast channel. When the user selects a TV channel, the controller 120 controls the broadcast receiver 140 to select the selected TV channel. When a broadcast signal is received through a TV channel selected by the broadcast receiving unit 140, the content processing unit 150 processes the received broadcast signal and outputs the processed broadcast signal through the display unit 130 and a speaker (not shown). For example, if the ATSC standard is adopted, the broadcast reception unit 140 may include an antenna, an RF down converter, a demodulator, an equalization unit, and the like.

The content processing unit 150 is a component for processing various types of content received through the communication unit 160 as well as a broadcast signal received through the broadcast receiving unit 140. The content processing unit 150 may include various signal processing units such as a demultiplexer, a video decoder, an audio decoder, and a scaler.

The communication unit 160 is a component for performing communication with various external devices. Specifically, the communication unit 160 may communicate with an external device according to various communication methods such as LAN, Wi-Fi, 3G, 4G, Bluetooth, Zigbee, NFC, and the like.

Specifically, the communication unit 160 is a voice recognition device (not shown) that converts a user's voice into text, a server device that performs a search based on a text corresponding to the user's voice, and a server device that determines whether a user's conversation has been completed. Communication with (not shown) can be performed.

* The storage unit 170 is a component in which various programs or data used for the operation of the display device 100 are stored. As described above, the chat client module may be stored in the storage unit 170 in order to interwork with external server devices. Alternatively, when the display device 100 itself performs a text conversion task, a search task, or a task for determining whether or not the utterance is complete, various programs such as a speech recognition module, a search module, and a utterance completion determination module are stored in the storage unit 170. It can also be stored in.

The control unit 120 may perform the voice control method according to various embodiments described above by using a program stored in the storage unit 170.

That is, when a voice input mode is selected through a button provided on the main body of the display device 100 or a button provided on a remote control, or a predefined trigger motion or a predefined trigger voice is input, the controller 120 You can switch to the voice input mode by executing the conversation client module. Accordingly, as described above, the control operation can be performed according to the user's voice.

When the voice input mode is started, the controller 120 may configure a graphic object and display it on the display 130 as described above. Specifically, the control unit 120 may calculate a location in which a graphic object is to be generated in the screen of the display unit 110 using a GPU (Graphic Processing Unit). The GPU calculates attribute values such as coordinate values, shape, size, and color in which the graphic object is to be displayed at a preset default value. The GPU renders a graphic object in the screen of the display unit 150 based on the calculated attribute value. When the voice input mode is released, the controller 120 may change the shape of the graphic object or delete the graphic object as described above.

In addition, when a user voice is input, the controller 120 transmits the user voice to the voice recognition apparatus 700 using the communication unit 160 and receives a text corresponding to the user voice.

The controller 120 provides the received text to the server device 800. Accordingly, when the server device 800 transmits a text-based search result, the control unit 120 displays the search result on the display unit 130.

In this state, when a utterance completion signal is received from the server device 800, the controller 120 may determine that the user utterance has been completed and may cancel the voice input mode.

Meanwhile, in FIG. 6, it is illustrated and described as including both the voice recognition device 700 and the server device 800, but some devices may be omitted according to embodiments.

For example, according to an embodiment in which the speech recognition apparatus 700 is omitted, the control unit 120 may perform text conversion using a text conversion module stored in the storage unit 170. That is, when the user's voice is input, the controller 120 does not transmit the user's voice to the voice recognition apparatus 700, but executes the text conversion module to convert the user's voice into text by itself. Accordingly, the converted text is transmitted to the server device 800, and the search result is received and displayed.

Alternatively, according to an embodiment in which the server device 800 is omitted, the controller 120 may directly perform a search using the text converted by the speech recognition device 700. That is, the control unit 120 executes a search module stored in the storage unit 170 and performs a search by inputting text into the search module. The controller 120 displays the search result. In addition, the controller 120 may determine whether or not the user has completed the utterance after each search by executing the utterance completion determination module. Accordingly, when it is determined that the utterance is complete, the controller 120 may cancel the voice input mode.

As described above, the voice control system may be implemented in various forms, and the configuration and operation of the display device 100 may be variously modified according to the form of the voice control system.

8 is a block diagram showing the configuration of a server device according to an embodiment of the present invention. Referring to FIG. 8, the server device 800 includes a server communication unit 810, a server control unit 820, and a database 830.

The server communication unit 810 is a component for performing communication with the display device 100. The server communication unit 810 may receive a text corresponding to a user's voice from a display device operating in a voice input mode.

The database 830 may store various types of content information. Specifically, various information such as EPG information, latest movie information, cable broadcasting schedule, web site information, and the like may be stored.

The server control unit 820 searches the database for information including text transmitted through the server communication unit 810. Accordingly, the searched content information is provided to the display device 100. When the subsequent text is transmitted according to the voice of the subsequent user, the server controller 820 may perform a search based on the subsequent text again and transmit the search result additionally.

When the content information is selected and confirmed by the display apparatus 100, the server controller 820 may determine that there is no additional search and transmit a utterance completion signal to the display apparatus 100. Alternatively, the server controller 820 may transmit a speech completion signal to the display device 100 when it is determined that information of the lowest level or information on the content is provided to the display device 100 by a subsequent search.

In this way, the server device 800 may perform a search according to the user utterance and analysis of the intention to complete the user utterance by interworking with the display device 100. As a result, it is possible to perform the above-described voice control method without the display apparatus 100 having an excessive computational burden.

9 is a flowchart illustrating a method of supporting voice control of a server device according to an embodiment of the present invention. Referring to FIG. 9, when text is received from the display device 100 (S910), the server device 800 performs a search based on the text (S920).

The server device 800 transmits the search result to the display device 100 (S930). The server device 800 may repeatedly perform the above-described steps whenever text is received.

In this state, when it is determined that the user's utterance is complete (S940), the server device 800 transmits a utterance completion signal to the display device 100 (S950). Accordingly, the display apparatus 100 may automatically release the voice input mode.

On the other hand, if it is determined that the utterance is incomplete, text transmission according to the voice of the subsequent user is waited (S960). That is, the voice input mode may be maintained in the display device 100.

According to the various embodiments as described above, when the user is predicted to have no further speech intention while the user is using the speech input mode, the speech input mode may be automatically canceled. Accordingly, it is possible to prevent obstruction of viewing due to unnecessary screen blocking, reduce the risk of malfunction due to misrecognition of voice, and reduce unnecessary waste of system resources.

Various voice control methods or voice control support methods as described above may be implemented as program codes and provided in a state stored in a non-transitory readable medium. A server device or a display device in which such a non-transitory readable medium is mounted may execute the program code to execute the method according to the various embodiments described above.

The non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, not a medium that stores data for a short moment, such as a register, cache, and memory. Specifically, it may be a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, or the like.

In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be understood individually from the technical spirit or prospect of the present invention.

100: display device 110: input unit
120: control unit 130: display unit

Claims (21)

In the electronic device,
A display unit;
Input unit; And
When a first user voice input is received through the input unit within a first time period, performing a function corresponding to voice recognition to obtain information corresponding to the first user voice input,
When the information corresponding to one result is obtained, controlling the display unit to display the one result,
When the information corresponding to a plurality of results is obtained, the display unit is controlled to display the plurality of results, and after the plurality of results are displayed, the plurality of And a controller configured to control the display unit to display a screen corresponding to the second user voice input when a second user voice input for selecting one of the results is received. The method of claim 1,
The first time interval is a waiting time, and the second time interval is the same as the first time interval. The method of claim 1,
The control unit,
When the information corresponding to the one result is obtained, the speech recognition operation is terminated. The method of claim 1,
The control unit,
An electronic device that controls the display unit to display a graphic object representing the speech recognition operation. The method of claim 4,
The control unit,
When the information corresponding to the one result is obtained, changing a shape of a graphic object representing the speech recognition operation. The method of claim 1,
The control unit,
When the information corresponding to the plurality of results is obtained, controlling the display unit to display the plurality of results and a graphic object representing the speech recognition operation. The method of claim 6,
The control unit,
When a second user voice input for selecting one of the plurality of results is received within the second time period, the electronic device changes a shape of a graphic object representing the voice recognition operation. The method of claim 1,
The control unit,
When a second user voice input for selecting one of the plurality of results is received within the second time period, the voice recognition operation is terminated. The method of claim 1,
The control unit,
When the first user voice input is received, the electronic device converts the first user voice input into text and controls the display to display the text corresponding to the first user voice input. The method of claim 1,
The communication unit; further includes,
The control unit,
When the first user voice input is received within the first time period through the input unit, a text corresponding to the first user voice input is transmitted to a server through the communication unit, and the text corresponding to the text is transmitted from the server. An electronic device that obtains information. In the control method of an electronic device,
When a first user voice input is received within a first time period, performing a function corresponding to voice recognition to obtain information corresponding to the first user voice input; And
When the information corresponding to one result is obtained, the one result is displayed on the display unit,
When the information corresponding to a plurality of results is obtained, the plurality of results are displayed on the display, and after the plurality of results are displayed, the plurality of results within a second time period for receiving a subsequent user's voice input And when a second user voice input for selecting one of them is received, displaying a screen corresponding to the second user voice input on the display unit. The method of claim 11,
The first time interval is a waiting time, and the second time interval is the same as the first time interval. The method of claim 11,
When the information corresponding to the one result is obtained, terminating the speech recognition operation. The method of claim 11,
The method of controlling an electronic device further comprising: displaying a graphic object representing the speech recognition operation on the display unit. The method of claim 14,
When the information corresponding to the one result is obtained, changing a shape of a graphic object representing the speech recognition operation. The method of claim 11,
When the information corresponding to the plurality of results is obtained, displaying the plurality of results and a graphic object representing the speech recognition operation on the display unit; The method of claim 16,
When a second user voice input for selecting one of the plurality of results is received within the second time period, changing a shape of a graphic object representing the voice recognition operation; further comprising, control of an electronic device Way. The method of claim 11,
When a second user voice input for selecting one of the plurality of results is received within the second time period, terminating the voice recognition operation. The method of claim 11,
When the first user voice input is received, converting the first user voice input into text and displaying the text corresponding to the first user voice input on the display unit; Control method. The method of claim 11,
When the first user voice input is received within the first time period, the text corresponding to the first user voice input is transmitted to a server through a communication unit, and the information corresponding to the text is transmitted from the server to the communication unit. Acquiring through; further comprising, the control method of the electronic device. A non-transitory computer-readable recording medium recording a program that can be executed by a control unit to execute a control method of an electronic device, the control method comprising:
When a first user voice input is received within a first time period, performing a function corresponding to voice recognition to obtain information corresponding to the first user voice input; And
When the information corresponding to one result is obtained, the one result is displayed on the display unit,
When the information corresponding to a plurality of results is obtained, the plurality of results are displayed on the display, and after the plurality of results are displayed, the plurality of results within a second time period for receiving a subsequent user's voice input And when a second user voice input for selecting one is received, displaying a screen corresponding to the second user voice input on the display unit.

Images