KR20230024712A - Display apparatus and method for searching in consideration of utterance intent - Google Patents

Abstract

Disclosed are a display device and method for performing a search in consideration of utterance intention. The disclosed display device searches content in consideration of the utterance intention of a user with information extracted from content information being executed and a voice command, or searches content in consideration of the utterance intention of the user with information extracted from broadcast information and the voice command of broadcasts currently being output.

Description

Display device and method for searching in consideration of utterance intention

It relates to a display device and method for searching for content in consideration of speech intention.

The search function is a function that must be provided in order to search for a vast amount of data, and is a key function that users also use thinking that it must exist in a service that provides content.

Search technology has been developed by utilizing various methods such as content index, progressive search function, similarity, and relevance. Speech search has also made great progress with the introduction of a more accurate speech-to-text (STT) technique by utilizing techniques such as LSTM (Long Short-Term Memory) among deep learning techniques.

In the case of searching through voice, although it is a different search than keyboard search, it has been searched by simply passing the analyzed keywords. That is, since it is a form of simply searching for words included in a voice command uttered by a user without considering the user's utterance intention, there is room for development.

According to various embodiments disclosed in this document, content may be searched in consideration of a user's utterance intention using content information being executed and information extracted from a voice command.

Further, according to various embodiments, content may be searched in consideration of a user's speech intention using information extracted from broadcast information of a broadcast currently being output and a voice command.

A search method considering an utterance intention according to an embodiment includes receiving a voice command; checking whether a content app included in the content app list is in an running state; extracting at least one of information about a content app, title, period, content type, and action to be executed in the voice command; if the content app included in the content app list is in an running state, checking information about the running content app; and an operation of searching for content by combining information about the running content app and information extracted from the voice command.

According to an exemplary embodiment, a display apparatus performing a search in consideration of an utterance intention includes a microphone for receiving a voice command; and checking whether a content app included in the content app list is in an running state, extracting at least one of information about a content app to be executed by the voice command, a title, a period, a content type, and an action, and extracting at least one of information about the content app included in the content app list. and a processor that checks information about the running content app when the app is running, and searches for content by combining information about the running content app with information extracted from the voice command.

According to an embodiment, a search system in consideration of speech intention includes a voice search server that extracts at least one of information about a content app to be executed from voice, a title, a period, a content type, and an action; A content search server that collects content information provided by each content providing server corresponding to the content app, stores it in a content database, and manages the content information; and when receiving a voice command, transmits the voice command to the voice search server to receive extracted information, checks whether a content app included in the content app list is running, and executes the content app included in the content app list. state, a display device that checks information about the running content app and searches content through the content search server by combining the information about the running content app and the information extracted from the voice command.

According to various embodiments, the user's utterance intention can be detected by searching for content by combining information extracted from content information being executed and information extracted from a voice command, or by searching content by combining broadcast information of a broadcast being output and information extracted from a voice command. We can provide you with a search that takes into consideration.

1 illustrates a schematic configuration of a display system for searching in consideration of an utterance intention according to an exemplary embodiment.
2 illustrates a process of searching for content in consideration of an utterance intention in a display device according to an exemplary embodiment.
3 illustrates a process of searching for content by combining information about a content app running on a display device and information extracted from a voice command according to an embodiment.
4 illustrates a process of searching for content by combining broadcasting information and information extracted from a voice command in a display device according to an exemplary embodiment.
5 illustrates a process of searching for content using information extracted from a voice command in a display device according to an exemplary embodiment.
6 is a block configuration diagram of a display device according to an exemplary embodiment.
7 is a detailed block configuration diagram of a display device according to an exemplary embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same reference numerals are given to the same components regardless of reference numerals, and overlapping descriptions thereof will be omitted.

Hereinafter, according to an embodiment of this document A display apparatus and method for searching in consideration of utterance intention will be described in detail with reference to FIGS. 1 to 7 attached.

1 illustrates a schematic configuration of a display system for searching in consideration of an utterance intention according to an exemplary embodiment.

Referring to FIG. 1 , a display system may include a display device 110 , a voice search server 120 and a content search server 130 .

The voice search server 120 may extract at least one of information about a content app to be executed, a title, a period, a content type, and an action from the voice received from the display device 110 .

The content search server 130 may collect content information provided from each of the content providing servers 131 , 132 , and 139 corresponding to the content app, and store and manage the content information as a content database. Collected content information may include information capable of classifying content, such as provided content app information, title, initial start date, content type, character, production company, and supervisor.

When receiving a voice command, the display device 110 transmits the voice command to the voice search server 120 to receive extracted information, checks whether a content app included in the content app list is running, and includes it in the content app list. When the currently running content app is in an running state, information on the running content app may be checked, and content may be searched through the content search server 130 by combining information on the running content app with information extracted from a voice command. .

When the display device 110 searches by combining the information about the running content app and the information extracted from the voice command, duplicate types of information exist in both the information about the running content app and the information extracted from the voice command. Then, it is possible to search by giving a higher weight to the information extracted from the voice command.

For example, if information about a content app to be executed exists in the information extracted from the voice command, the display device 110 searches for content in the content app to be executed with the highest priority, and then searches for content in the running content app with the next priority. And finally, content can be searched from other content apps included in the list of content apps. At this time, when the display device 110 searches for content in the content apps included in the content app list, it determines the priority of the content apps by considering the execution frequency of the content apps included in the content app list or the user's preference; According to the determined priority, search results may be sorted and provided.

In addition, when the content app included in the content app list is not running and is in a broadcast output state, the display device 110 obtains currently output broadcast information, and searches for content by combining broadcast information and information extracted from a voice command. Content may be searched through the server 130 .

In this case, the information about the broadcast may be the type of currently output broadcast (eg, drama, entertainment, movie, documentary), character, production company, broadcasting time information, and the like.

When the display device 110 searches by combining the information about the broadcast and the information extracted from the voice command, duplicate types of information exist in both the information about the broadcast and the information extracted from the voice command, or are related to the other one. If information (higher or lower category information) exists, a search may be performed by giving a higher weight to information extracted from a voice command.

For example, when a voice command “Find a TV program aired last year” is received in a situation where a broadcast whose content type is “drama” is being output, the display device 110 displays information extracted from the voice command. By giving a relatively low weight to the content type “drama” included in the broadcast information, which is a subcategory of the included “TV program”, search for TV programs aired last year through the content app included in the content app list, and , When outputting search results, the TV programs of the drama may be sorted and outputted so that they precede.

Although the voice search server 120 and content search server 130 are located outside in FIG. 1 , they may be included inside the display device 110 .

In this document, if information extracted from a voice command is defined as a user context, information on a currently running content app or broadcasting information may be defined as a background context.

The display device 110 searches for content by combining the user context and the background context, and may set information included in the user context to have a higher priority than information included in the background context. The display device 110 may perform a search in consideration of set priorities or arrange output results in consideration of set priorities.

A method of searching for content in consideration of the user's utterance intention in the display device 110 will be described later in detail with reference to FIGS. 2 through 5 .

Hereinafter, the method according to the present document configured as described above will be described with reference to the drawings below.

2 illustrates a process of searching for content in consideration of an utterance intention in a display device according to an exemplary embodiment.

Referring to FIG. 2 , when receiving a voice command (210), the display device 110 may check whether a content app is running (212).

Then, the display device 110 extracts information from the voice command (214). In this case, the extracted information may be at least one of information about a content app to be executed, a title, a period, a content type, and an action. At this time, the display device 110 may transmit the voice command received in operation 214 to the voice search server 120 and receive the extracted information from the voice search server 120 .

Meanwhile, operations 212 and 214 may be performed in reverse order or simultaneously.

Then, when the content app is running (216), the display device 110 checks information on the running content app (218), and searches for content by combining the information on the running content app with the extracted information. (220), and the result of searching for the content is output (230). Operation 220 will be described later in detail with reference to FIG. 3 .

As a result of checking operation 216, if the content app is not running, the display device 110 checks whether it is in a broadcast output state (222).

As a result of checking in operation 222, if the broadcast is output, the display device 110 obtains currently output broadcast information (224), searches for content by combining the broadcast information and the extracted information (226), and retrieves the content. One result is output (230). Operation 226 will be described later in detail with reference to FIG. 4 .

As a result of checking in operation 222, if the broadcast is not output, the display device 110 searches for content using the extracted information (228) and outputs the content search result (230). Operation 228 will be described later in detail with reference to FIG. 5 .

When displaying the result of searching for content in operation 230 , the display device 110 may check the priority of the content app and sort and output the searched result according to the checked priority. In this case, the priority of the content apps may be determined by considering the execution frequency of the content apps included in the content app list or the user's preference.

Meanwhile, in operation 230, the display device 110 may execute the searched content if the information about the action extracted from the voice command is execution and the searched result is one.

Meanwhile, in operation 230, the display device 110 may output a content list that is the search result if there are a plurality of search results even if the action information extracted from the voice command is execution.

3 illustrates a process of searching for content by combining information about a content app running on a display device and information extracted from a voice command according to an embodiment.

Referring to FIG. 3 , the display device 110 checks whether the extracted information includes information about a content app to be executed (310).

As a result of checking operation 310, if the extracted information includes information about a content app to be executed, the display device 110 searches for content provided through the content app to be executed using the extracted information (312).

As a result of searching for content in operation 312, if the searched content does not exist (314), the display device 110 searches for content provided through the running content app using the remaining information except for the content app to be executed from the extracted information ( 316).

Meanwhile, as a result of checking operation 310, if the information about the content app to be executed is included in the extracted information, the display device 110 performs operation 316.

As a result of searching for content in operation 316, if the searched content does not exist (318), the display device 110 uses the remaining information except for the content app to be executed from the extracted information to provide through other content apps included in the content app list. Content is retrieved (320).

Referring to FIG. 3 , the display device 110 receives a voice command to search for content named “A” through the second content app in a situation where the first content app is running, and the second content requested is requested. If the content named “A” is found in the app and there is no content named “A” in the second content app, the content named “A” is searched in the first content app that is being executed, and the content named “A” is also found in the first content app. If there is no content named "A", the content named "A" may be searched through another content app included in the content app list.

In addition, when a voice command to search for content named "B" is received in a situation where the first content app is running, the display device 110 searches for content named "B" in the first content app that is being executed. In addition, if there is no content named “B” in the first content app, the content named “B” may be searched through another content app included in the content app list.

Meanwhile, when the display device 110 searches for content in operations 312, 316, and 320, the content search server collects content information provided by each content providing server corresponding to the content app, stores it in a content database, and manages the content information. Contents can be searched through (130).

4 illustrates a process of searching for content by combining broadcasting information and information extracted from a voice command in a display device according to an exemplary embodiment.

Referring to FIG. 4 , the display device 110 checks whether the extracted information includes information about a content app to be executed (410).

As a result of checking operation 410, if the extracted information includes information about a content app to be executed, the display device 110 searches for content provided through the content app to be executed using the extracted information and information about broadcasting (412). .

As a result of searching for content in operation 412, if the searched content does not exist (414), the display device 110 searches for content provided through a content app included in the content app list using the extracted information and information about broadcasting. (416).

Referring to FIG. 4 , in a situation where the display device 110 is outputting a broadcast whose content type is “drama,” when a voice command to search for a TV program aired last year is received through the second content app, a request is made. Search for content with a period of "last year" and a content type of "TV program" in the received second content app, and if there is no search result in the second content app, the period is "last year" through another content app included in the list of content apps. " and the content type is "TV program". Thereafter, when providing search results, the display device 110 may arrange TV programs corresponding to "drama" corresponding to broadcasting information among TV programs in the search results to be displayed at the top and provided to the user.

On the other hand, when the display device 110 searches for content in operations 412 and 416, the content search server 130 collects content information provided from each content providing server corresponding to the content app, stores it in a content database, and manages the content information. Content can be searched through.

5 illustrates a process of searching for content using information extracted from a voice command in a display device according to an exemplary embodiment.

Referring to FIG. 5 , the display device 110 checks whether the extracted information includes information about a content app to be executed (510).

As a result of checking operation 510, if the extracted information includes information about a content app to be executed, the display device 110 searches for content provided through the content app to be executed using the extracted information (512).

As a result of searching for content in operation 512, if the searched content does not exist (514), the display device 110 searches for content provided through a content app included in the content app list using the extracted information (516).

On the other hand, when the display device 110 searches for content in operations 512 and 516, the content search server 130 collects content information provided from each content providing server corresponding to the content app, stores it in a content database, and manages the content information. Content can be searched through.

A display device for searching in consideration of utterance intention may be configured as shown in FIG. 6 or 7 below.

6 is a block configuration diagram of a display device according to an exemplary embodiment.

7 is a detailed block configuration diagram of a display device according to an exemplary embodiment.

As shown in FIG. 6 , the display device 110 according to an embodiment may include a memory 620, a processor 630, a communication unit 650, and a sensing unit 691. However, not all illustrated components are essential components. The display device 110 may be implemented with more components than those illustrated, or the display device 110 may be implemented with fewer components.

For example, as shown in FIG. 7 , the display device 110 according to an embodiment includes a memory 620, a processor 630, a communication unit 650, a sensing unit 691, and a display 610. , a tuner unit 640, a sensing unit 660, an input/output unit 670, a video processing unit 680, an audio processing unit 615, an audio output unit 626, and a power supply unit 690 may be further included. .

Hereinafter, we look at the above components.

The processor 630 controls overall operation of the display device 110 and signal flow between internal components of the display device 110 and processes data. The processor 630 may execute an operation system (OS) and various applications stored in the memory 620 when there is a user's input or when a pre-set and stored condition is satisfied.

The processor 630 stores signals or data input from the outside of the display device 110 or controls the RAM used as a storage area corresponding to various tasks performed in the display device 110 and the display device 110. It may include a ROM and a processor in which a control program for the control program is stored.

The processor 630 may include a graphic processing unit (not shown) for graphic processing corresponding to video. The processor 630 may be implemented as a system on chip (SoC) in which a core (not shown) and a GPU (not shown) are integrated. The processor 630 may include a single core, a dual core, a triple core, a quad core, and multiple cores thereof.

In addition, when receiving a voice command through the microphone 661, the processor 630 transmits the voice command to the voice search server 120 to receive extracted information, and checks whether a content app included in the content app list is running. and if the content app included in the list of content apps is in an running state, information on the running content app is checked, and the content search server 130 is configured by combining the information on the running content app with the information extracted from the voice command. You can search through the content.

In addition, the processor 630 obtains currently outputted broadcasting information when a content app included in the content app list is not running and is in a broadcasting output state, and combines broadcasting information with information extracted from a voice command to a content search server. Contents can be searched through (130).

Also, the processor 630 may include a plurality of processors. For example, the processor may be implemented as a main processor (not shown) and a sub processor (not shown) operating in a sleep mode.

According to an embodiment, the processor 630 executes one or more instructions stored in the memory 620, and through the sensing unit 691 including the at least one sensor, at least one sensor corresponding to the at least one sensor. A sensing value can be detected.

In addition, according to an embodiment, the processor 630 executes one or more instructions stored in the memory 620, and when it is determined that at least one detected sensing value is equal to or greater than a preset threshold value, the mobile device 200 It may be determined that is touched on the display device 110 .

Also, according to an embodiment, the processor 630 compares at least one detected sensing value with a sensing value of the mobile device received from the mobile device 200 by executing one or more instructions stored in the memory 620. By doing so, it can be determined that the mobile device 200 has touched the display device 110 .

Also, according to an embodiment, the processor 630 requests identification information of the mobile device 200 and receives the identification information of the mobile device 200 by executing one or more instructions stored in the memory 620. can Also, the processor 630 may confirm that the mobile device 200 is a device pre-registered in the display device 110 based on the identification information of the mobile device 200 .

Also, according to an embodiment, the processor 630 executes one or more instructions stored in the memory 620 so that the mobile device 200 touches the display device 110 based on at least one sensed value. A touch area can be determined.

Also, according to an embodiment, the processor 630 compares at least one sensing value detected corresponding to at least one sensor by executing one or more instructions stored in the memory 620, and based on the comparison result , One or more sensors determined to be disposed close to a point where the mobile device 200 is touched on the display device 110 may be determined. Also, the processor 630 may determine the touch area based on the determined one or more sensors.

Also, according to an embodiment, the processor 630 may receive state information about an operation being executed in the mobile device 200 from the mobile device 200 through the communication unit 650 .

In addition, according to an embodiment, the processor 630 executes one or more instructions stored in the memory 620, based on the received state information of the mobile device 200, to set a preset corresponding to the determined touch area. function can be performed.

Also, according to an embodiment, the processor 630 executes one or more instructions stored in the memory 620 so that the mobile device 200 touches the display device 110 based on at least one sensed value. The number of touches can be determined.

Also, according to an embodiment, the processor 630 may perform a preset function corresponding to the number of touches by executing one or more instructions stored in the memory 620 .

Also, according to an embodiment, the processor 630 may preset a function corresponding to at least one touch area on the display device 110 based on a user input by executing one or more instructions stored in the memory 620. can

In addition, the processor 630 may preset a function corresponding to the number of touches performed on at least one touch area on the display device 110 based on a user input by executing one or more instructions stored in the memory 620. there is.

The memory 620 may store various data, programs, or applications for driving and controlling the display device 110 under the control of the processor 630 . The memory 620 includes a video processing unit 680, a display 610, an audio processing unit 615, an audio output unit 626, a power supply unit 630, a tuner unit 640, a communication unit 650, and a sensing unit 660. , input/output signals or data corresponding to driving of the input/output unit 670 may be stored.

The memory 620 includes an operating system 621 for controlling the display device 110 and the processor 630, an application 622 initially provided by a manufacturer or downloaded from the outside, a graphical user interface (GUI) related to the application, and a GUI. Objects (eg, image text, icons, buttons, etc.), user information, documents, databases, or related data may be stored to provide.

In addition, the memory 620 receives an input signal from a remote control device (not shown) and controls a channel corresponding to the input signal accordingly, or a channel scroll user interface mode when the input signal corresponds to a predetermined input. A TV viewer module 623 including one or more instructions for entering , a character recognition module 624 including one or more instructions for recognizing information from content received from an external device (not shown), an external device (not shown) MBR module 625 containing one or more instructions for channel control from

The memory 620 includes ROM, RAM, or a memory card (eg, micro SD card, USB memory, not shown) mounted on the display device 110 . Also, the memory 620 may include a non-volatile memory, a volatile memory, a hard disk drive (HDD), or a solid state drive (SSD).

According to an embodiment, the memory 620 may be a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (eg SD or XD memory, etc.), RAM (RAM, Random Access Memory), SRAM (Static Random Access Memory), ROM (ROM, Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory) ), a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium.

The display 610 displays a video included in a broadcast signal received through the tuner unit 640 on the screen under the control of the processor 630 . Also, the display 610 may display content (eg, video) input through the communication unit 650 or the input/output unit 670 . The display 610 may output an image stored in the memory 620 under the control of the processor 630 .

The display 610 generates driving signals by converting video signals, data signals, OSD signals, control signals, and the like processed by the processor 630 . The display 610 may be implemented with a plasma display panel (PDP), a liquid crystal display (LCD), an organic light emitting diode (OLED), a cathode ray tube (CRT), a flexible display, and the like, and also 3 It can be implemented as a 3D display. Also, the display 610 may be configured as a touch screen and used as an input device in addition to an output device.

The tuner unit 640 selects only the frequency of a channel desired to be received by the display device 110 from many radio wave components through amplification, mixing, resonance, etc. of broadcast signals received by wire or wirelessly. It can be selected by tuning. The broadcast signal includes audio, video, and additional information (eg, Electronic Program Guide (EPG)).

The tuner unit 640 performs a control signal received from a remote control device (not shown) (eg, a channel number input, a channel up-down input, and a channel input on an EPG screen) according to a user input. A broadcast signal can be received in a frequency band corresponding to the channel number.

The tuner unit 640 may receive broadcast signals from various sources such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, Internet broadcasting, and the like. The tuner unit 640 may receive a broadcasting signal from a source such as analog broadcasting or digital broadcasting. The broadcast signal received through the tuner unit 640 is decoded (eg, audio decoding, video decoding, or additional information decoding) and separated into audio, video, and/or additional information. Separated audio, video and/or additional information may be stored in the memory 620 under the control of the processor 630 .

The tuner unit 640 of the display device 110 may be one or plural. The tuner unit 640 may be implemented as an all-in-one with the display device 110 or a separate device having a tuner unit electrically connected to the display device 110 (eg, a set-top box). -top box, not shown), and a tuner unit (not shown) connected to the input/output unit 670).

The communication unit 650 may connect the display device 110 with an external device (eg, an audio device) (not shown) under the control of the processor 630 . The processor 630 may transmit/receive content to/from an external device (not shown) connected through the communication unit 650, download an application from the external device (not shown), or perform web browsing. The communication unit 650 may include one of wireless LAN 651 , Bluetooth 652 , and wired Ethernet 153 corresponding to the performance and structure of the display device 110 . Also, the communication unit 650 may include a combination of wireless LAN 651, Bluetooth 652, and wired Ethernet 153.

Also, the communication unit 650 may communicate with the voice search server 120 or the content search server 130 under the control of the processor 630 .

Also, the communication unit 650 may receive a control signal from a remote control device (not shown) under control of the processor 630 . The control signal may be implemented as a Bluetooth type, an RF signal type, or a Wi-Fi type.

In addition, the communication unit 650 may further include other short-range communication (eg, near field communication (NFC), not shown) and bluetooth low energy (BLE, not shown) other than Bluetooth.

The sensing unit 660 detects a user's voice, a user's video, or a user's interaction, and may include a microphone 661, a camera unit 662, and a light receiving unit 663.

The microphone 661 receives the user's utterance. The microphone 661 may convert the received voice into an electrical signal and output it to the processor 630 . The user's voice may include, for example, a voice corresponding to a menu or function of the display device 110 .

The camera unit 662 may obtain an image bezel such as a still image or a moving image. Images captured through the image sensor may be processed through the processor 630 or a separate image processing unit (not shown).

The image bezel processed by the camera unit 662 may be stored in the memory 620 or transmitted to the outside through the communication unit 650 . Two or more camera units 662 may be provided according to the configuration of the display device 110 .

The light receiving unit 663 receives an optical signal (including a control signal) received from an external remote control device (not shown). The light receiving unit 663 may receive an optical signal corresponding to a user input (eg, touch, pressure, touch gesture, voice, or motion) from a remote control device (not shown). A control signal may be extracted from the received optical signal under the control of the processor 630 . For example, the light receiving unit 663 may receive a control signal corresponding to a channel up/down button for channel switching from a remote control device (not shown).

The input/output unit 670 includes video (eg, motion picture, etc.), audio (eg, voice, music, etc.) and additional information (eg, video, etc.) from the outside of the display device 110 under the control of the processor 630. For example, EPG, etc.) and the like are received. The input/output unit 670 includes at least one of a High-Definition Multimedia Interface port (HDMI) port 671, a component jack 672, a PC port 673, and a USB port 674. can do. The input/output unit 670 may include a combination of at least one of an HDMI port 671 , a component jack 672 , a PC port 673 , and a USB port 674 . An external video providing device (not shown) may be connected through the HDMI port 671 .

The video processor 680 processes video data received by the display device 110 . The video processing unit 680 may perform various image processing such as decoding, scaling, noise filtering, bezel rate conversion, and resolution conversion on video data.

The graphic processing unit 681 uses a calculation unit (not shown) and a rendering unit (not shown) to create a screen including various objects such as icons, images, and text. The calculation unit (not shown) calculates attribute values such as coordinate values, shape, size, color, and the like of each object to be displayed according to the layout of the screen using the user input sensed through the sensor 660 . The rendering unit (not shown) creates screens of various layouts including objects based on the attribute values calculated by the calculation unit (not shown). The screen created by the rendering unit (not shown) is displayed within the display area of the display 610 .

The audio processing unit 615 processes audio data. The audio processor 615 may perform various processes such as decoding or amplifying audio data and filtering noise. Meanwhile, the audio processing unit 615 may include a plurality of audio processing modules to process audio corresponding to a plurality of contents.

The audio output unit 626 outputs audio included in the broadcast signal received through the tuner unit 640 under the control of the processor 630 . The audio output unit 626 may output audio (eg, voice, sound) input through the communication unit 650 or the input/output unit 670 . Also, the audio output unit 626 may output audio stored in the memory 620 under the control of the processor 630 . The audio output unit 626 may include at least one of a speaker 627, a headphone output terminal 628, or a Sony/Philips Digital Interface (S/PDIF) output terminal 629. The audio output unit 626 may include a combination of at least one of a speaker 627, a headphone output terminal 628, and a S/PDIF output terminal 629.

The power supply unit 690 supplies power input from an external power source to components inside the display device 110 under the control of the processor 630 . In addition, the power supply unit 690 may supply power output from one or more batteries (not shown) located inside the display device 110 to internal components under the control of the processor 630 .

The sensing unit 691 may detect a state of the display device 110 or a state around the display device 110 and transmit the sensed information to the processor 630 .

The sensing unit 691 includes a magnetic sensor 692, an acceleration sensor 693, a temperature/humidity sensor 694, an infrared sensor 695, a gyroscope sensor 696, and a position sensor. (eg, a GPS) 697, an air pressure sensor 698, a proximity sensor 699, and an RGB sensor (illuminance sensor) 201, but may include at least one, but is not limited thereto. Since a person skilled in the art can intuitively infer the function of each sensor from its name, a detailed description thereof will be omitted.

The sensing unit 691 according to an embodiment may detect an external impact applied to the display device 110 . For example, when the mobile device 200 touches the display device 110, the sensing unit 691 of the display device 110 may output a sensed value.

Also, the display device 110 including the display 610 may be electrically connected to a separate external device (eg, a set-top box, not shown) including the tuner unit 640 .

In addition, the display device 110 may be implemented as an analog TV, digital TV, 3D-TV, smart TV, LED TV, OLED TV, plasma TV, monitor, etc., but is not limited thereto. It will be easily understood by those who have.

Meanwhile, the block diagram of the illustrated display device 110 is a block diagram for one embodiment. Each component of the block diagram may be integrated, added, or omitted according to specifications of the display device 110 that is actually implemented. That is, if necessary, two or more components may be combined into one component, or one component may be subdivided into two or more components. In addition, the functions performed in each block are for explaining the embodiments, and the specific operation or device does not limit the scope of the present invention.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (20)

receiving a voice command;
checking whether a content app included in the content app list is in an running state;
extracting at least one of information about a content app, title, period, content type, and action to be executed in the voice command;
if the content app included in the content app list is in an running state, checking information about the running content app; and
Searching for content by combining information about the running content app and information extracted from the voice command
A search method considering an utterance intent that includes.
According to claim 1,
The operation of searching for content by combining the information about the running content app and the information extracted from the voice command,
If the extracted information does not include information about the content app to be executed, searching for content provided through the currently running content app using the extracted information.
A search method considering an utterance intent that includes.
According to claim 2,
Searching for content provided through other content apps included in the list of content apps using the extracted information, if the searched content does not exist as a result of searching for content provided through the running content app.
A search method considering an utterance intent that further includes.
According to claim 1,
The operation of searching for content by combining the information about the running content app and the information extracted from the voice command,
If the extracted information includes information about the content app to be executed, searching for content provided through the content app to be executed using remaining information excluding information about the content app to be executed from the extracted information.
A search method considering an utterance intent that includes.
According to claim 4,
The operation of searching for content by combining information about the running content app and the information extracted from the voice command,
As a result of searching for content provided through the content app to be executed, if the searched content does not exist, the content provided through the content app being executed is retrieved using the remaining information excluding the information on the content app to be executed from the extracted information. action to search
A search method considering an utterance intent that includes.
According to claim 5,
The operation of searching for content by combining information about the running content app and the information extracted from the voice command,
If content is not found as a result of searching for content provided through the content app to be executed and the content app being executed, the extracted information is included in the content app list using the remaining information excluding information on the content app to be executed. Searching for content provided through other content apps
A search method considering an utterance intent that includes.
According to claim 1,
obtaining broadcast information currently being output if the content app included in the content app list is not running and is in a broadcasting output state; and
Searching for content by combining information about the broadcast and information extracted from the voice command
A search method considering an utterance intent that further includes.
According to claim 7,
The operation of searching for content by combining the information about the broadcast and the information extracted from the voice command,
If the extracted information does not include information about the content app to be executed, searching for content provided through all content apps included in the list of content apps using the extracted information and information about the broadcasting.
A search method considering an utterance intent that includes.
According to claim 7,
The operation of searching for content by combining the information about the broadcast and the information extracted from the voice command,
If the extracted information includes information about the content app to be executed, information about the content app to be executed is provided through the content app to be executed by using the information about the broadcast and the remaining information except for the information about the content app to be executed in the extracted information. The act of retrieving content
A search method considering an utterance intent that includes.
According to claim 9,
The operation of searching for content by combining the information about the broadcast and the information extracted from the voice command,
As a result of searching for content provided through the content app to be executed, if the searched content does not exist, the list of content apps is created using the remaining information excluding the information on the content app to be executed from the extracted information and the information on the broadcast. Searching for content provided through other included content apps
A search method considering an utterance intent that includes.
According to claim 1,
If the information about the action extracted from the voice command is a search, outputting a search result
A search method considering an utterance intent that further includes.
According to claim 1,
If the information about the action extracted from the voice command is execution and the search result is one, executing the searched content.
A search method considering an utterance intent that further includes.
According to claim 1,
If the information about the action extracted from the voice command is execution and there are a plurality of search results, outputting the search results.
A search method considering an utterance intent that further includes.
According to claim 1,
The operation of extracting at least one of information about the content app, title, period, content type, and action to be executed in the voice command,
Transmitting the voice command to a voice search server, and receiving the extracted information from the voice search server
Search method considering utterance intent.
According to claim 1,
An operation of collecting content information provided from a content providing server corresponding to a content app included in the content app list, storing it in a content database, and managing the content information.
Including more,
The operation of searching for content by combining the information about the running content app and the information extracted from the voice command,
Retrieving content from the content database
Search method considering utterance intent.
According to claim 1,
An operation of determining a priority of a content app by considering an execution frequency of the content app included in the content app list or a user's preference.
Including more,
The operation of searching for content by combining the information about the running content app and the information extracted from the voice command,
When content is searched from a plurality of content apps, sorting and outputting the searched results according to the priority of the content apps
A search method considering an utterance intent that includes.
a microphone to receive voice commands; and
It is checked whether a content app included in the content app list is in an running state, at least one of content apps to be executed by the voice command, title, duration, content type, and action information is extracted, and content apps included in the content app list are included in the content app list. In this running state, a processor that checks information about the running content app and searches for content by combining the information about the running content app and the information extracted from the voice command.
A display device for searching in consideration of an utterance intention comprising a.
According to claim 17,
the processor,
If the extracted information does not include information about the content app to be executed, content provided through the running content app is searched using the extracted information.
A display device that searches in consideration of utterance intention.
According to claim 17,
the processor,
If the content app included in the content app list is not running and is in a broadcast output state, obtaining broadcast information currently output and searching for content by combining information about the broadcast with information extracted from the voice command
A display device that searches in consideration of utterance intention.
a voice search server that extracts at least one of information about a content app to be executed from voice, a title, a period, a content type, and an action;
A content search server that collects content information provided by each content providing server corresponding to the content app, stores it in a content database, and manages the content information; and
When a voice command is received, the voice command is transmitted to the voice search server to receive extracted information, it is checked whether a content app included in the content app list is running, and the content app included in the content app list is running. , a display device that checks information about the running content app and searches for content through the content search server by combining the information about the running content app and the information extracted from the voice command.
A system for searching in consideration of an utterance intent that includes.

Images