KR20200069936A - Apparatus for providing information contained in media and method for the same - Google Patents

Abstract

According to an embodiment of the present invention, provided is a display device which provides information included in media. The display device comprises: a processor converting sound data extracted from media content into character string data, obtaining first meaning information indicating meaning of query data corresponding to queries of a user and second meaning information indicating meaning of the character string data, and generating response data corresponding the query data based on the character string data using the meaning information; and an output unit outputting the generated response data.

Description

Device and method for providing information contained in media {APPARATUS FOR PROVIDING INFORMATION CONTAINED IN MEDIA AND METHOD FOR THE SAME}

The present invention relates to an information providing apparatus and a method included in the media. Specifically, the present invention relates to a method of detecting and providing information requested by a user based on voice information played in a content applied to a terminal for viewing broadcast or streaming content.

Digital TV services using wired or wireless communication networks are becoming common. The digital TV service can provide various services that could not be provided by the existing analog broadcasting service.

For example, IPTV (Internet Protocol Television), a type of digital TV service, and smart TV service provide bi-directionality that allows a user to actively select the type of viewing program, viewing time, and the like. IPTV and smart TV services may provide various additional services, such as Internet search, home shopping, online games, etc., based on this bi-directionality.

In the past, even if you want to check the information provided by the media while watching the media, if the media is real-time streaming, the method for the user to obtain information directly from the media playback device is limited to the provided Electronic Program Guide (EPG). There is only a method of checking or rewinding the latest image stored in the buffer to check again, or a method of obtaining media information from an external information provider.

However, the EPG only contains the overall information of the media, and often does not contain the information contained in the contents of the media. When checking the data stored in the buffer, it is cumbersome and increases the complexity of the device, and the external information provider In the case of requesting media information, there is a possibility that a problem of copyright infringement may occur as the captured image is exported to the outside, and it is inefficient.

In addition, according to Patent Publication No. 10-2010-0072972, when viewing media content, voice information included in the content may be recognized to generate and provide subtitles. However, this is merely to generate the audio information of the content as subtitles, and does not select and provide suitable information among information included in the media content according to a user's query.

The above-mentioned background art is technical information acquired by the inventor for the derivation of the present invention or acquired in the derivation process of the present invention, and is not necessarily a known technology disclosed to the general public prior to the filing of the present invention.

Domestic Patent Publication 10-2010-0072972

The present invention provides an apparatus and method for providing information contained in media that extracts information corresponding to a user's query from information output as sounds included in the media content and provides it to the user while providing media content to the user I want to.

In addition, the present invention is to provide an apparatus and a method for providing information included in media, updating and managing at least one or more engines used to generate response data corresponding to a user's query.

In addition, the present invention is to provide an apparatus and method for providing information included in media, which selects a query range of string data corresponding to media content and generates response data corresponding to a query within a selected query range. .

According to an embodiment of the present invention, information output as sound included in media content provided to a user is converted into string data, and semantic information corresponding to string data and semantic information corresponding to a user's query are generated and generated. By comparing the semantic information with each other, information corresponding to the user's query is extracted from the string data, and response data is generated based on the extracted information and provided to the user.

In addition, an embodiment of the present invention updates engines by receiving update data for at least one engine used to generate response data from a management server connected through wired/wireless communication.

In addition, an embodiment of the present invention classifies a query range for string data using at least one or more of EPG information corresponding to a user's interaction or media content, and corresponds to a user's query among the divided query ranges Determines the range of queries to be made, and generates response data based on the determined range of queries.

According to various embodiments of the present invention, it is possible to reconfirm the information that the user wants to check again while watching the media content in a simple manner, thereby facilitating the user's information search convenience.

In addition, according to various embodiments of the present invention, since various engines used to generate response data are constantly updated and managed, response data can be generated by increasing the accuracy of response data and supporting a wide range of languages. .

In addition, according to various embodiments of the present invention, since a query range for a user query is determined based on at least one of user interaction or EPG data, the query range is matched with a query range recognized by the user with high accuracy. Can be. Furthermore, since response data is generated within the query range, the operation speed can be increased.

1 is a block diagram showing the configuration of a display device according to an embodiment of the present invention.
2 is a block diagram of a remote control device according to an embodiment of the present invention.
Figure 3 shows an actual configuration example of a remote control device according to an embodiment of the present invention.
4 shows an example of using a remote control device according to an embodiment of the present invention.
5 is a block diagram showing the configuration of a system for providing information included in media according to an embodiment of the present invention.
6 is a block diagram showing an example of an apparatus for providing information included in the media shown in FIG. 5.
7 is an operation flowchart illustrating a method for providing information included in media according to an embodiment of the present invention.
8 is an operation flowchart illustrating an example of the step of generating the response data shown in FIG. 7.
9 is an operation flowchart illustrating an example of a step of determining a query range of character string data shown in FIG. 8.
10 to 12 are views showing an aspect of use of an embodiment of the present invention.

Hereinafter, embodiments related to the present invention will be described in more detail with reference to the drawings. The suffixes "modules" and "parts" for components used in the following description are given or mixed only considering the ease of writing the specification, and do not have meanings or roles distinguished from each other in themselves.

The display device according to an exemplary embodiment of the present invention is, for example, an intelligent display device in which a computer support function is added to a broadcast reception function, and while being faithful to the broadcast reception function, an Internet function, etc. is added, a handwritten input device, a touch screen Alternatively, a more convenient interface such as a space remote control may be provided. And, with the support of wired or wireless Internet functions, it is possible to perform functions such as email, web browsing, banking, or games by accessing the Internet and computers. A standardized general-purpose OS can be used for these various functions.

Thus, the display device described in the present invention, for example, on a general-purpose OS kernel, since various applications can be freely added or deleted, various user-friendly functions can be performed. The display device may be more specifically, for example, a network TV, HBBTV, smart TV, LED TV, OLED TV, and the like, and is also applicable to a smartphone in some cases.

1 is a block diagram showing the configuration of a display device according to an embodiment of the present invention.

Referring to FIG. 1, the display device 100 includes a broadcast receiving unit 130, an external device interface unit 135, a storage unit 140, a user input interface unit 150, a control unit 170, and a wireless communication unit 173. , A display unit 180, an audio output unit 185, and a power supply unit 190.

The broadcast reception unit 130 may include a tuner 131, a demodulation unit 132, and a network interface unit 133.

The tuner 131 may select a specific broadcast channel according to a channel selection command. The tuner 131 may receive a broadcast signal for a selected specific broadcast channel.

The demodulator 132 may separate the received broadcast signal into a video signal, an audio signal, and a data signal related to a broadcast program, and restore the separated video signal, audio signal, and data signal to a form capable of output.

The external device interface unit 135 may receive an application or a list of applications in an adjacent external device and transmit it to the control unit 170 or the storage unit 140.

The external device interface unit 135 may provide a connection path between the display device 100 and the external device. The external device interface unit 135 may receive one or more of video and audio output from an external device connected to the display device 100 wirelessly or wiredly, and transmit them to the control unit 170. The external device interface unit 135 may include a plurality of external input terminals. The plurality of external input terminals may include an RGB terminal, one or more High Definition Multimedia Interface (HDMI) terminals, and a component terminal.

The image signal of the external device input through the external device interface unit 135 may be output through the display unit 180. The audio signal of the external device input through the external device interface unit 135 may be output through the audio output unit 185.

The external device connectable to the external device interface unit 135 may be any one of a set-top box, a Blu-ray player, a DVD player, a game machine, a sound bar, a smartphone, a PC, a USB memory, and a home theater, but this is only an example. .

The network interface unit 133 may provide an interface for connecting the display device 100 to a wired/wireless network including an Internet network. The network interface unit 133 may transmit or receive data with other users or other electronic devices through a connected network or another network linked to the connected network.

In addition, some content data stored in the display device 100 may be transmitted to a selected user or a selected electronic device among other users or other electronic devices registered in advance in the display device 100.

The network interface unit 133 may access a predetermined web page through a connected network or another network linked to the connected network. That is, it is possible to connect to a predetermined web page through a network, and transmit or receive data with the corresponding server.

In addition, the network interface unit 133 may receive content or data provided by a content provider or a network operator. That is, the network interface unit 133 may receive content such as a movie, advertisement, game, VOD, broadcast signal, and related information provided by a content provider or a network provider through a network.

Further, the network interface unit 133 may receive update information and update files of firmware provided by the network operator, and may transmit data to the Internet or a content provider or a network operator.

The network interface unit 133 may select and receive a desired application from among applications that are open to the public through a network.

The storage unit 140 may store a program for processing and controlling each signal in the control unit 170 and store a signal-processed image, audio, or data signal.

In addition, the storage unit 140 may perform a function for temporarily storing an image, audio, or data signal input from the external device interface unit 135 or the network interface unit 133, and is predetermined through a channel memory function. You can also store information about the image.

The storage unit 140 may store an application or a list of applications input from the external device interface unit 135 or the network interface unit 133.

The display device 100 may play and provide content files (video files, still image files, music files, document files, application files, etc.) stored in the storage unit 140 to the user.

The user input interface unit 150 may transmit a signal input by the user to the control unit 170 or a signal from the control unit 170 to the user. For example, the user input interface unit 150 may be in accordance with various communication methods, such as Bluetooth, WB (Ultra Wideband), ZigBee, RF (Radio Frequency) communication, or infrared (IR) communication, A control signal such as power on/off, channel selection, screen setting, and the like may be received from the remote control device 200, or may be processed to transmit a control signal from the control unit 170 to the remote control device 200.

In addition, the user input interface unit 150 may transmit a control signal input from a local key (not shown) such as a power key, a channel key, a volume key, and a set value to the control unit 170.

The image signal processed by the controller 170 may be input to the display unit 180 and displayed as an image corresponding to the corresponding image signal. Also, the image signal processed by the control unit 170 may be input to the external output device through the external device interface unit 135.

The audio signal processed by the control unit 170 may be audio output to the audio output unit 185. Also, the audio signal processed by the control unit 170 may be input to the external output device through the external device interface unit 135.

In addition, the control unit 170 may control the overall operation in the display device 100.

In addition, the control unit 170 may control the display device 100 by a user command or an internal program input through the user input interface unit 150, and access the network to display a list of applications or applications desired by the user. You can make it downloadable within (100).

The control unit 170 allows the channel information or the like selected by the user to be output through the display unit 180 or the audio output unit 185 together with the processed image or audio signal.

In addition, the control unit 170 according to the external device image playback command received through the user input interface unit 150, from an external device input through the external device interface unit 135, for example, a camera or a camcorder, The video signal or the audio signal can be output through the display unit 180 or the audio output unit 185.

Meanwhile, the control unit 170 may control the display unit 180 to display an image, for example, a broadcast image input through the tuner 131, or an external input input through the external device interface unit 135 The image or an image input through the network interface unit or an image stored in the storage unit 140 may be controlled to be displayed on the display unit 180. In this case, the image displayed on the display unit 180 may be a still image or a video, and may be a 2D image or a 3D image.

In addition, the control unit 170 may control the content stored in the display device 100 or the received broadcast content or external input content input from the outside to be played, and the content may include a broadcast video, an external input video, and an audio file. , Still images, connected web screens, and document files.

The wireless communication unit 173 may communicate with external devices through wired or wireless communication. The wireless communication unit 173 may perform short range communication with an external device. To this end, the wireless communication unit 173 is Bluetooth (Bluetooth™), Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), Wi-Fi By using at least one of (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technology, short-range communication can be supported. The wireless communication unit 173 may be provided between the display device 100 and the wireless communication system, between the display device 100 and another display device 100, or through the display device 100 through wireless area networks. A wireless communication between networks in which the display device 100 or an external server is located may be supported. The wireless local area network may be wireless personal area networks.

Here, the other display device 100 is a wearable device capable of exchanging data with the display device 100 according to the present invention (or interoperable), for example, a smart watch, a smart watch, and smart glasses (smart glass), HMD (head mounted display), can be a mobile terminal such as a smart phone. The wireless communication unit 173 may detect (or recognize) a wearable device capable of communication around the display apparatus 100. Furthermore, when the detected wearable device is a device authenticated to communicate with the display apparatus 100 according to the present invention, the control unit 170 may transmit at least a portion of data processed by the display apparatus 100 to the wireless communication unit 173. It can be transmitted to the wearable device. Therefore, a user of the wearable device can use data processed by the display device 100 through the wearable device.

The display unit 180 converts the image signal, data signal, OSD signal, or image signal received from the external device interface unit 135 processed by the control unit 170 into R, G, and B signals, respectively, and then drives them. You can generate a signal.

Meanwhile, the display device 100 illustrated in FIG. 1 is only an embodiment of the present invention. Some of the illustrated components may be integrated, added, or omitted depending on the specifications of the display device 100 that is actually implemented.

That is, if necessary, two or more components may be combined into one component, or one component may be divided into two or more components. In addition, the function performed in each block is for explaining an embodiment of the present invention, the specific operation or device does not limit the scope of the present invention.

According to another embodiment of the present invention, unlike the display device 100 shown in FIG. 1, without a tuner 131 and a demodulator 132, the network interface unit 133 or the external device interface unit ( 135) to receive and reproduce the video.

For example, the display device 100 is divided into an image processing device such as a set-top box for receiving broadcast signals or contents according to various network services, and a content playback device for playing content input from the image processing device. Can be implemented.

In this case, the operation method of the display device according to an embodiment of the present invention will be described below, as well as the display device 100 as described with reference to FIG. 1, the image processing device or display unit such as the separated set-top box ( 180) and an audio output unit 185.

Next, a remote control device according to an embodiment of the present invention will be described with reference to FIGS. 2 to 3.

2 is a block diagram of a remote control device according to an embodiment of the present invention, and FIG. 3 shows an actual configuration example of a remote control device 200 according to an embodiment of the present invention.

First, referring to FIG. 2, the remote control device 200 includes a fingerprint recognition unit 210, a wireless communication unit 220, a user input unit 230, a sensor unit 240, an output unit 250, and a power supply unit 260. ), a storage unit 270, a control unit 280, and a voice acquisition unit 290.

Referring to FIG. 2, the wireless communication unit 225 transmits and receives a signal to any one of the display devices according to the embodiments of the present invention described above.

The remote control device 200 includes an RF module 221 capable of transmitting and receiving signals to and from the display device 100 according to RF communication standards, and capable of transmitting and receiving signals to and from the display device 100 according to IR communication standards. An IR module 223 may be provided. In addition, the remote control device 200 may include a Bluetooth module 225 capable of transmitting and receiving signals to and from the display device 100 according to the Bluetooth communication standard. In addition, the remote control device 200 is provided with an NFC module 227 capable of transmitting and receiving signals with the display device 100 according to NFC (Near Field Communication) communication standards, and displays according to WLAN (Wireless LAN) communication standards. A WLAN module 229 capable of transmitting and receiving signals to and from the device 100 may be provided.

In addition, the remote control device 200 transmits a signal including information on the movement of the remote control device 200 to the display device 100 through the wireless communication unit 220.

Meanwhile, the remote control device 200 may receive a signal transmitted from the display device 100 through the RF module 221, and power on/off the display device 100 through the IR module 223 as necessary. Commands for off, channel change, volume change, and the like can be transmitted.

The user input unit 230 may include a keypad, a button, a touch pad, or a touch screen. The user may input a command related to the display device 100 to the remote control device 200 by operating the user input unit 230. When the user input unit 230 includes a hard key button, the user may input a command related to the display device 100 to the remote control device 200 through a push operation of the hard key button. This will be described with reference to FIG. 3.

Referring to FIG. 3, the remote control device 200 may include a plurality of buttons. The plurality of buttons are fingerprint recognition button 212, power button 231, home button 232, live button 233, external input button 234, volume control button 235, voice recognition button 236, It may include a channel change button 237, a confirmation button 238 and a back button 239.

The fingerprint recognition button 212 may be a button for recognizing a user's fingerprint. In one embodiment, the fingerprint recognition button 212 may perform a push operation, and may receive a push operation and a fingerprint recognition operation. The power button 231 may be a button for turning on/off the power of the display device 100. The home button 232 may be a button for moving to the home screen of the display device 100. The live button 233 may be a button for displaying a real-time broadcast program. The external input button 234 may be a button for receiving an external input connected to the display device 100. The volume control button 235 may be a button for adjusting the volume of the volume output by the display device 100. The voice recognition button 236 may be a button for receiving a user's voice and recognizing the received voice. The channel change button 237 may be a button for receiving a broadcast signal of a specific broadcast channel. The confirmation button 238 may be a button for selecting a specific function, and the back button 239 may be a button for returning to a previous screen.

2 will be described again.

When the user input unit 230 includes a touch screen, the user may input a command related to the display device 100 to the remote control device 200 by touching a soft key of the touch screen. In addition, the user input unit 230 may include various types of input means that can be operated by a user, such as a scroll key or a jog key, and this embodiment does not limit the scope of the present invention.

The sensor unit 240 may include a gyro sensor 241 or an acceleration sensor 243, and the gyro sensor 241 may sense information on the movement of the remote control device 200.

For example, the gyro sensor 241 may sense information on the operation of the remote control device 200 based on the x, y, and z axes, and the acceleration sensor 243 may move the speed of the remote control device 200. Information about the back can be sensed. Meanwhile, the remote control device 200 may further include a distance measurement sensor, and sense a distance from the display unit 180 of the display device 100.

The output unit 250 may output an image or audio signal corresponding to the operation of the user input unit 235 or a signal transmitted from the display device 100. The user may recognize whether the user input unit 235 is operated or whether the display device 100 is controlled through the output unit 250.

For example, the output unit 250 includes a LED module 251 that lights when the user input unit 235 is operated or a signal is transmitted and received through the wireless communication unit 225, and a vibration module generating vibration ( 253), an audio output module 255 for outputting sound, or a display module 257 for outputting an image.

In addition, the power supply unit 260 supplies power to the remote control device 200, and when the remote control device 200 has not been moved for a predetermined time, power supply is stopped to reduce power waste. The power supply unit 260 may resume power supply when a predetermined key provided in the remote control device 200 is operated.

The storage unit 270 may store various types of programs and application data required for control or operation of the remote control device 200. If the remote control device 200 transmits and receives signals wirelessly through the display device 100 and the RF module 221, the remote control device 200 and the display device 100 transmit and receive signals through a predetermined frequency band. .

The control unit 280 of the remote control device 200 stores and references information on a frequency band capable of wirelessly transmitting and receiving signals to and from the display device 100 paired with the remote control device 200 in the storage unit 270 for reference. can do.

The control unit 280 controls all matters related to the control of the remote control device 200. The control unit 280 may display a signal corresponding to a predetermined key manipulation of the user input unit 235 or a signal corresponding to the movement of the remote control device 200 sensed by the sensor unit 240 through the wireless communication unit 225 ( 100).

Also, the voice acquisition unit 290 of the remote control device 200 may acquire voice.

The voice acquisition unit 290 may include at least one microphone 291, and may acquire voice through the microphone 291.

Next, FIG. 4 will be described.

4 shows an example of using a remote control device according to an embodiment of the present invention.

4A illustrates that a pointer 205 corresponding to the remote control device 200 is displayed on the display unit 180.

The user may move or rotate the remote control device 200 up, down, left, and right. The pointer 205 displayed on the display unit 180 of the display device 100 corresponds to the movement of the remote control device 200. The remote control device 200 may be referred to as a spatial remote controller because the corresponding pointer 205 is moved and displayed according to the movement on the 3D space, as shown in the drawing.

4B illustrates that when the user moves the remote control device 200 to the left, the pointer 205 displayed on the display unit 180 of the display device 100 also moves to the left correspondingly.

Information on the movement of the remote control device 200 detected through the sensor of the remote control device 200 is transmitted to the display device 100. The display device 100 may calculate the coordinates of the pointer 205 from information related to the movement of the remote control device 200. The display device 100 may display the pointer 205 to correspond to the calculated coordinates.

4C illustrates a case in which a user moves the remote control device 200 away from the display unit 180 while pressing a specific button in the remote control device 200. Accordingly, the selected area in the display unit 180 corresponding to the pointer 205 may be zoomed in and enlarged.

Conversely, when the user moves the remote control device 200 to be closer to the display unit 180, the selected area in the display unit 180 corresponding to the pointer 205 may be zoomed out and reduced.

On the other hand, when the remote control device 200 moves away from the display unit 180, the selection area is zoomed out, and when the remote control device 200 approaches the display unit 180, the selection area may be zoomed in.

In addition, recognition of up, down, left, and right movements may be excluded while a specific button in the remote control device 200 is pressed. That is, when the remote control device 200 moves to move away from or approach the display unit 180, the up, down, left, and right movements are not recognized, and only the forward and backward movements can be recognized. When a specific button in the remote control device 200 is not pressed, only the pointer 205 moves according to the up, down, left, and right movements of the remote control device 200.

Meanwhile, the moving speed or the moving direction of the pointer 205 may correspond to the moving speed or the moving direction of the remote control device 200.

Meanwhile, the pointer in the present specification means an object displayed on the display unit 180 in response to the operation of the remote control device 200. Therefore, objects of various shapes are possible with the pointer 205 in addition to the arrow shape shown in the drawing. For example, it may be a concept including points, cursors, prompts, and thick outlines. In addition, the pointer 205 may be displayed corresponding to any one point of the horizontal axis and the vertical axis on the display unit 180, as well as a plurality of points such as a line and a surface. Do.

Next, with reference to FIG. 5, an operation method of a system according to an embodiment of the present invention will be described.

5 is a block diagram showing the configuration of a system 5 for providing information contained in media according to an embodiment of the present invention.

Referring to FIG. 5, a system 5 for providing information included in media according to an embodiment of the present invention includes a device 500 for providing information included in media, a user terminal 51, and a display device 52 ) And the management server 53.

In one embodiment, the system 5 for providing information contained in the media may be configured without including at least one of the user terminal 51, the display device 52 and the management server 53.

In another embodiment, the system 5 for providing information contained in the media may consist only of the device 500 for providing information contained in the media.

The apparatus 500 for providing information included in the media extracts sound data from the media content, converts sound data into string data, and converts the sound data into string data. Acquire the corresponding semantic information, and use the semantic information to generate response data corresponding to a user query based on string data, and provide response data.

Here, the apparatus 500 for providing information included in the media may be implemented as a television (TV), a projector, a smartphone, a tablet, a set-top box (STB), a DMB receiver, a radio, and the like.

In this case, the user query may include a query voice voice through utterance or a query through interaction with a button in which query contents are set in advance.

In this case, the apparatus 500 that provides information included in the media may refer to the display apparatus 100 illustrated in FIG. 1. In this case, the apparatus 500 that provides information included in the media may use the broadcast signal received through the tuner 131 as media content.

At this time, the device 500 that provides information included in the media may receive and use the media content from another device through wired or wireless communication, but may also use the media content stored in the memory 520 of FIG. 6.

The user terminal 51 is a device 500 that provides information contained in the media and a device capable of transmitting and receiving signals to each other through a wired or wireless communication function, and the function of the device 500 that provides information contained in the media It means a terminal used to control. Here, the user terminal 51 may mean a remote control, a smart phone, or the like.

At this time, the user terminal 51 may refer to the remote control device 200 shown in FIG. 2.

In this case, the user terminal 51 may be configured to include a microphone capable of receiving the user's voice, and in this case, the user's voice may be input and transmitted to the device 500 that provides information included in the media. .

At this time, the user terminal 51 may be configured to include a speaker capable of outputting sound, and in this case, sound data received from the device 500 providing information included in the media may be output as sound. .

At this time, the user terminal 51 may be configured to include a display unit capable of outputting image information, and in this case, the image data received from the device 500 providing information included in the media can be output as an image. have.

At this time, the user may query through the menu or button of the user terminal 51. Accordingly, the user terminal 51 collectively refers to a remote control including a microphone, a remote control of a terminal that can operate a menu, and a remote control app implemented with a smartphone.

The display device 52 is a device that outputs image data as an image, and includes a display unit that outputs image information. In particular, the display device 52 may be configured to additionally include a speaker capable of outputting sound. Here, the display device 52 may mean a television (TV), a monitor, a projector, a tablet, a smartphone, or a laptop.

In this case, the display device 52 may mean the display device 100 illustrated in FIG. 1.

At this time, the display device 52 may receive the media content from the device 500 providing information included in the media and output it. In addition, the display device 52 may receive response data corresponding to a user query from the device 500 providing information included in the media and output it.

The management server 53 is connected to the device 500 providing information contained in the media through wired or wireless communication, and externally managing software or data stored in the device 500 providing information contained in the media. It is a server.

At this time, the management server 53 may provide update data of the Speech To Text (STT) engine to the device 500 that provides information included in the media.

In this case, the management server 53 may provide update data of the semantic analysis engine to the device 500 that provides information included in the media.

At this time, the management server 53 may provide update data to the device 500 providing information included in the media at the request of the device 500 providing information contained in the media, and the information included in the media The device 500 providing the request data may be requested from the management server 53 periodically or at the request of the user.

At this time, when there is a new update for the STT engine or the semantic analysis engine, the management server 53 may automatically transmit the corresponding update data to the device 500 providing the information included in the media.

At this time, the management server 53 may transmit update data that improves the accuracy of the engine to at least one of the STT engine or the semantic analysis engine to the device 500 that provides information included in the media. Since the STT engine generates character string data from the voice information, how accurately it can recognize the voice information can be accurate. Since the semantic analysis engine analyzes the meaning of the string data, how accurate it is to understand the meaning of the string data can be accurate.

At this time, the management server 53 may provide update data for modifying the support language of the engine for at least one of the STT engine or the semantic analysis engine to the device 500 that provides the information included in the media. That is, even if the existing STT engine or the semantic analysis engine supports only English, it is possible to additionally support Korean through an engine update.

At this time, the modification of the support language of the engine may include adding/activating a new support language or deleting/deactivating an existing support language. For example, when the support for a specific language is operated on a fixed-term basis, the corresponding support language may be deleted or deactivated through update data when the corresponding period expires.

Accordingly, when using the apparatus 500 for providing information included in the media according to an embodiment of the present invention, the user can easily check the information he/she wants to check again from among the information heard while watching the media. For example, when playing a song while listening to the radio, it is possible to easily check the information of the song that the DJ provided.

In particular, since the media content itself to be played is generated without a server for retrieving information, response data according to a query is generated, so a device that provides information contained in the media independently from a single manufacturer's point of view is manufactured and service is provided. Can be. And, even if additional information is not provided through a separate information search, sufficient satisfaction can be provided by awakening the memory to the user.

6 is a block diagram illustrating an example of an apparatus 500 that provides information included in the media shown in FIG. 5.

Referring to FIG. 6, an apparatus 500 for providing information included in media according to an embodiment of the present invention includes a processor 510, a memory 520, a communication unit 530, a microphone 540, and an output unit ( 550).

At this time, the processor 510 includes a sound data extraction unit 511, a voice data extraction unit 512, a character string data conversion unit 513, a query data reception unit 514, a response data generation unit 515, and an engine management unit 516. ) And the like.

In detail, the processor 510 is a kind of central processing unit and controls the entire process of providing information included in the media based on speech recognition. That is, the STT engine or the semantic analysis engine stored in the memory 520 is driven, and the communication unit 530, the microphone 540, the output unit 550, the sound data extraction unit 511, the voice data extraction unit 512, Various functions may be provided by controlling the character string data conversion unit 513, the query data reception unit 514, the response data generation unit 515, and the engine management unit 516.

Here, the processor 510 may include any kind of device capable of processing data. Here, the'processor (processor)', for example, may mean a data processing device embedded in hardware having physically structured circuits to perform functions represented by codes or instructions included in a program. As an example of such a data processing device embedded in hardware, a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, and an application-specific integrated ASIC circuit), a field programmable gate array (FPGA), and the like, but the scope of the present invention is not limited thereto.

Here, the processor 510 may refer to the control unit 170 of FIG. 1.

The sound data extraction unit 511 extracts sound data from media content.

In this case, the media content may be received from another device through the communication unit 530 or may be stored in the memory 520.

The voice data extraction unit 512 extracts voice data from sound data.

The character string data conversion unit 513 converts voice data into character string data.

The query data receiving unit 514 receives query data corresponding to the user's query.

At this time, the user's query may include a query through utterance or through interaction with a button or menu of the user terminal 51.

At this time, the query data receiving unit 514 may receive query data corresponding to the user's query voice through the microphone 540.

In this case, the query data receiving unit 514 may receive query data corresponding to the user's query voice from the user terminal 51 through the communication unit 530.

The response data generating unit 515 acquires semantic information corresponding to the meaning of the query data using a semantic analysis engine. And, if the semantic information corresponding to the string data is not obtained, semantic information corresponding to the meaning of the string data is acquired using a semantic analysis engine. Then, response data corresponding to the query data based on the string data is generated using the semantic information.

The engine manager 516 updates at least one of the STT engine or the semantic analysis engine.

At this time, the engine management unit 516 may update the STT engine or the semantic analysis engine by receiving update data from the management server 53 connected through wired/wireless communication.

At this time, the engine management unit 516 may perform an update that improves the accuracy of the STT engine or the semantic analysis engine.

At this time, the engine management unit 516 may perform an update for changing the language supported by the STT engine or the semantic analysis engine. For example, when the STT engine and the semantic analysis engine only support English, an update that adds Korean as a new support language may be performed.

At this time, the engine management unit 516 may check whether the update is possible to the management server 53 periodically or when a user requests the update.

At this time, when the engine management unit 516 receives a signal that a new update is possible from the management server 53, the engine management unit 516 may receive the update data from the management server 53 automatically or at the request of the user to proceed with the update.

The memory 520 temporarily or permanently stores data processed by the processor 510. Here, the memory 520 may include a magnetic storage media or a flash storage media, but the scope of the present invention is not limited thereto.

In this case, the memory 520 may refer to the storage 140 of FIG. 1.

The communication unit 530 provides a communication interface required to transmit and receive signals between the device 500 that provides information included in the media, the user terminal 51, the display device 52, and the management server 53.

Here, the communication unit 530 may be a device including hardware and software necessary for transmitting and receiving a signal such as a control signal or a data signal through a wired or wireless connection with another network device.

In this case, the communication unit 530 may refer to the broadcast receiving unit 130 of FIG. 1, the external device interface unit 135, and the wireless communication unit 173.

At this time, the communication unit 530 may transmit and receive media content. For example, when the apparatus 500 for providing information contained in media according to an embodiment of the present invention is a set-top box (STB), a broadcast signal is received as media content, and the received media content is processed as it is or the process is performed. It can then be transferred to a display device.

At this time, the communication unit 530 may include one of an interface with an RF antenna, a path for receiving media content, an interface for accessing Internet content, and an interface with a cable/satellite/set-top box for Internet services (STB).

In addition, the communication unit 530 may further include an interface for connecting an external device (not shown) for this if additional computational power or storage is required to expand the function. In this case, the connected external device As an extension of the device 500 that provides information contained in the media, it becomes a dependent device under control.

The microphone 540 converts the input sound wave into an electrical signal.

The output unit 550 outputs response data generated by the processor 510.

In this case, the output unit 550 may include an audio output unit 551 and a display unit 552.

The audio output unit 551 converts sound data into sound waves and outputs the sound data. Here, the audio output unit 551 may mean a speaker.

In this case, the audio output unit 551 may refer to the audio output unit 185 of FIG. 1.

The display unit 552 visualizes and outputs image data. Here, the display unit 552 may mean a display panel, a projector, or the like.

In this case, the display unit 552 may refer to the display unit 180 of FIG. 1.

At this time, the output unit 550 may transmit response data to the user terminal 51 or the display device 52 through the communication unit 530. At this time, the output unit 550 may provide response data by visualizing or hearing it.

If the communication unit 530 transmits response data to another device, it can be seen that the communication unit 530 functions as an output unit 550.

7 is an operation flowchart illustrating a method for providing information included in media according to an embodiment of the present invention.

Referring to FIG. 7, in a method of providing information contained in media according to an embodiment of the present invention, a processor 510 of a device 500 providing information contained in media extracts sound data from media content. (S701).

At this time, the media content may include one or more of video data or audio data.

In this case, the media content may refer to a broadcast signal received through the tuner 131, media content received through the communication unit 530 from another device, or media content stored in the memory 520.

Here, the other device means a separate device connected to the device 500 providing information included in the media according to an embodiment of the present invention through wired or wireless communication. Other devices may include set-top boxes, computers and removable storage.

For example, when the apparatus 500 for providing information contained in media according to an embodiment of the present invention is a set-top box (STB), a broadcast signal is received as media content, and the received media content is processed as it is or the processing process is performed. After that, it can be transmitted to the display device 52.

In this case, the processor 510 may extract sound data by separating video data, sound data, and other data from the media content. If the separated data is compressed, data is restored through each decoder. If uncompressed data comes through HDMI, the decoder may not work.

At this time, the processor 510 may extract sound data without performing a separate separation operation when the media content is different from sound data.

At this time, the processor 510 may extract sound data by performing a separation operation that separates sound data from other data when the media content is not separately classified.

At this time, the processor 510 may extract sound data in real time with respect to the media content being played.

The processor 510 of the apparatus 500 that provides information included in the media separates sound data into voice data and non-voice data (S703).

The sound data included in general media content includes various types of data. For example, there are music in concert content, sound effects in a movie, voice of an announcer in news, etc., and information desired by a user is usually voice information. Of course, it can be useful for non-speech information, and the present invention can also be applied to this.

At this time, the processor 510 can extract only voice data in a manner that distinguishes voice from non-voice (background sound) and cancels non-voice. That is, only voice data can be extracted through a noise reduction technique.

At this time, the processor 510 may adjust the equalizer so as to emphasize only the voice by separating the frequency band of the voice from the other frequency band for the voice data. TVs currently sold in the market are equipped with technologies for detecting whether audio is included in audio, and differently processing audio according to the result. Using such a technique, it is possible to detect whether an audio signal is included in the sound data, and highlight the audio signal.

The processor 510 of the apparatus 500 that provides information included in the media converts voice data into character string data (S705).

At this time, the generated string data is in text format and can be recorded in a script-like format.

At this time, the processor 510 may convert voice data into string data using the STT engine.

In this case, the STT engine is not limited in the implementation method, and may be an engine that learns through machine learning or deep learning.

At this time, the STT engine may convert voice data in which one or more languages are mixed into string data.

At this time, the STT engine may be stored in the memory 520, but may also be stored in a separate external storage (not shown).

At this time, data related to the basic language set in the STT engine may be stored in the memory 520, and data related to the additional language of the STT engine may be stored in a separate external storage (not shown).

At this time, the processor 510 may store the converted string data in the memory 520.

At this time, the processor 510 may encrypt and store the converted string data in a separate external storage (not shown).

At this time, the generated string data may include time information.

For example, when a voice of "hungry" is output at 13:32 minutes, the string data may be stored in a format such as "(13:32) hungry".

At this time, the generated string data may include date information together.

In this case, the date information or time information stored in the string data may mean actual time point information in which the media content is played, or time point information in the corresponding media content.

At this time, the processor 510 may generate and store semantic information corresponding to the meaning of the string data in real time or by using a semantic analysis engine at regular intervals.

At this time, the processor 510 may extract one or more keywords indicating meaning from the string data, and generate meaning information based on the extracted keywords.

For example, when the media content includes a voice saying "I'm going to the Eiffel Tower tomorrow", the media content is converted to string data, and "Tomorrow", "Eiffel Tower", and "Go" are used as keywords. Can be extracted.

At this time, the processor 510 may distinguish each keyword through various tags.

For example, the processor 510 may classify the keywords "tomorrow", "eiffel tower", and "go" through tags in the following formats: "start-tomorrow", "place-eiffel tower" and "action-go". .

At this time, when the music is included in the sound data, the processor 510 may convert information about the music into string data using a music search function.

At this time, the information about the music may include information corresponding to the title, artist and album.

The processor 510 of the apparatus 500 providing information included in the media receives query data corresponding to the user's query (S707).

At this time, the user's query is a query that searches information included in the media content provided to the user.

At this time, the user query may include a query set in advance on a button or menu provided in the user terminal 52 and a query voice by the user's speech.

For example, the user's query may be to search for information in media content such as "Where did you just go?"

In this case, the user's query may refer to a query for searching for various information including place information, person information, music information, etc. included in the media content.

At this time, the microphone 540 of the device 500 providing information included in the media may receive query data corresponding to the user's query voice.

At this time, the communication unit 530 may receive query data corresponding to the user's query voice from the user terminal 52. In this case, the microphone of the user terminal 52 may receive the query voice of the user.

At this time, the communication unit 530 may receive query data corresponding to the user's query from the user terminal 52. In this case, the button or menu of the user terminal 52 may generate query data corresponding to the user's input.

At this time, the processor 510 may convert the query data into a string using the STT engine.

The processor 510 of the apparatus 500 that provides information included in the media acquires semantic information corresponding to the meaning of each of the string data and the query data using the semantic analysis engine (S709).

Here, semantic information corresponding to the meaning of the query data may be referred to as first semantic information, and semantic information corresponding to the meaning of the string data may be referred to as second semantic information.

At this time, the semantic analysis engine is not limited in the implementation method, and may be an engine that learns through machine learning or deep learning.

In this case, the semantic analysis engine may acquire semantic information (second semantic information) corresponding to the semantic even for string data in which one or more languages are mixed.

In this case, the semantic analysis engine may be stored in the memory 520, or may be stored in a separate external storage (not shown).

At this time, data related to the basic language set in the semantic analysis engine may be stored in the memory 520, and data related to the additional language of the semantic analysis engine may be stored in a separate external storage (not shown).

At this time, the semantic analysis engine may extract one or more keywords for each of the string data and the query data, and generate semantic information based on the extracted keywords.

For example, in the query voice of the user, "Where did you say you were going?", the keywords "previous", "where" and "go" can be extracted. Then, the extracted keywords, “previous”, “where”, and “go” can be added to the first semantic information.

At this time, the processor 510 may identify each keyword through a tag.

For example, the keywords “previous”, “where” and “go” can be classified as “start-up”, “place-where” and “behavior-go”.

At this time, there are no restrictions on the available tags.

For example, the processor 510 uses tags corresponding to who, when, where, what, how, and why based on the subordinate principle 5W1H. You can distinguish each keyword.

As described above, when the processor 510 extracts a keyword and generates semantic information based on the keyword, the keyword data corresponding to the string data and the keyword corresponding to the user's query data are compared to easily find information suitable for the query and response data Can generate

In the apparatus 500 that provides information included in the media, the processor 510 extracts information corresponding to the query data from the string data using the semantic information to generate response data (S711).

At this time, the processor 510 may receive additional information corresponding to the information extracted from the string data in a network connected through a communication unit 530 by wire or wirelessly and add it to the response data.

For example, when the processor 510 generates response data by extracting "Eiffel Tower" as information corresponding to a user's query from the string data, additional information about the Eiffel Tower is acquired and the response data is displayed as "Eiffel Tower (Paris, France, )".

At this time, the additional information may include text, images, and link information for providing detailed information related to the extracted information.

At this time, the processor 510 calculates the accuracy or association with the query data for each information included in the string data, and extracts and responds to information corresponding to the query data from the string data based on the calculated accuracy or association. Data can be generated.

At this time, the processor 510 may determine a query range of string data for generating response data corresponding to the user's query, and search for information corresponding to the user's query within the determined query range to generate response data. .

In addition, in the method of providing information included in the media according to an embodiment of the present invention, the processor 510 of the apparatus 500 providing information included in the media provides the generated response data (S713).

At this time, the processor 510 may visualize and display the generated response data through the display unit 552.

At this time, the processor 510 may output the generated response data as sound through the audio output unit 551.

At this time, the processor 510 may transmit the generated response data to the display device 52 through the communication unit 530.

At this time, the processor 510 may transmit the generated response data to the user terminal device 51 through the communication unit 530. In this case, the user terminal device 51 may visualize and display the received response data through the display unit 552 or output the sound through the audio output unit 551.

Accordingly, the user can check the response according to the query as visualized information or auditory information.

At this time, the processor 510 may convert response data into sound data using a text to speech (TTS) engine.

At this time, the processor 510 may follow the setting by the user's selection whether to visualize or audit the response data.

In an optional embodiment, the steps (S701, S703 and S705) of generating character string data from the media content may be performed in parallel with the step of receiving user query data (S707 ).

Particularly, FIG. 7 is a simplified view of a method for providing information included in media according to an embodiment of the present invention, and the above steps (S701, S703, S705, S707, S709, S711, and S703) It can be performed repeatedly.

8 is an operation flowchart illustrating an example of step S711 of generating the response data shown in FIG. 7.

Referring to FIG. 8, the processor 510 of the apparatus 500 that provides information included in the media determines a query range of character string data (S801) .

In this case, the query range may refer to a range for extracting (searching) information in media content corresponding to a user's query.

For example, when a user changes and views a channel from 10 to 11, the processor 510 may classify a query range of character string data based on a channel change time point. If the user asks a question in the form of "where did you go from the previous channel?", the processor 510 may generate response data by performing a query on string data corresponding to channel 10.

In addition, when the user continues to watch the channel 10 times, but the media (program) to be broadcast changes, the processor 510 may classify the query range of the string data based on the change point of the media. If the user asks a question in the form of "where did you go from the previous program?", the processor 510 may generate response data by performing a query on string data corresponding to the previous media (program).

The processor 510 of the apparatus 500 that provides the information included in the media calculates the degree of association with the query data for each information included within the query range of the string data (S803) .

In this case, the processor 510 may calculate the association degree by digitizing.

At this time, the processor 510 compares keywords corresponding to each information included in the string data with keywords included in the query data, and calculates the degree of association with the query data of each information based on the number of keywords matched with each other. Can be.

The processor 510 of the apparatus 500 that provides the information included in the media selects a predetermined number in order of high association with the query data from the information included in the string data to generate response data (S805).

For example, in response to a user's query, the processor 510 may select (extract) three pieces of information from the information included in the string data in order of high association with the query data to generate response data.

The processor 510 of the device 500 providing information included in the media acquires additional information from the external device for each selected information and adds it to the response data (S807).

In this case, the external device may mean a network, a client or a server connected to the device 500 providing information included in the media through wired or wireless communication.

For example, the processor 510 may acquire additional information in a networked search engine with respect to the information selected in step S805, and add the additional information to the response data.

As such, the processor can increase the accuracy of response data according to the user's query by determining the query range in the string data, and reduce the amount of computation by performing a query on only the required string data.

9 is an operation flowchart showing an example of a step (S801) of determining a query range of character string data shown in FIG. 8.

Referring to FIG. 9, the processor 510 of the apparatus 500 that provides information contained in the media classifies a query range of string data using at least one of user interaction or EPG information (S901). .

At this time, user interaction may include changing a channel, changing a connection method, changing a media, and changing an output device. EPG information may include program (media content) title information, program airing time information, and the like. The EPG information may include EPG based on SI (Service Information) received with a broadcast signal or online EPG information coming through a separate channel such as the Internet.

That is, the processor 510 may classify the query range of the string data using meta data of the media content.

The processor 510 of the apparatus 500 providing information included in the media selects a query range corresponding to the query data (S903).

In this case, the processor 510 may select a query range suitable for the target media content to be questioned by the user in consideration of semantic information corresponding to the user's query.

At this time, the processor 510 may calculate a correlation between at least one of user interaction or EPG information and query data, and select a query range using the most relevant element.

For example, a user watches channel 10 from 1 PM to 1:30 PM, and changes the channel from 10 to 11 at 1:30 PM to watch until 2:30 PM, and 2 PM It is assumed that the program being aired on channel 11 is changed from program A to program B.

At this time, the processor 510 may classify the query range of the string data based on the user's channel change at 1:30 pm and the media (program) change at channel 11 at 2 pm.

That is, the first query range is string data for channel 10 from 1 PM to 1:30 PM, and the second query range is string for program A of channel 11 from 1:30 PM to 2 PM. Data, and the third query range becomes character string data for program B of channel 11 from 2 pm to 2:30 pm.

If the user asks a question such as "Where did you say you were going on the previous channel?", you can select the first query scope as the query scope. When the user asks a question such as "Who said from the previous program?", the second query scope can be selected as the query scope. When the user asks a question such as "What song was it?", the third query scope can be selected as the query scope.

In this way, it is possible to classify a more accurate query range by classifying the query range of the string data using the user's interaction or EPG information.

In addition, the classification of the query scope is generally matched with the behavior or media classification that can be recognized by the user, so that a high-accuracy response to the query can be expected.

10 to 12 are views showing an aspect of use of an embodiment of the present invention.

First, referring to FIG. 10, the apparatus 1001 for providing information included in media according to an embodiment of the present invention may output media including voice information.

At this time, the device 1001 for providing information included in the media may mean the device 500 for providing information included in the media shown in FIGS. 5 or 6.

In the example of FIG. 10, the media content may include speech sentences 1002b, 1003b, or 1004b spoken by at least one speaker 1002a, 1003a, or 1004a.

For example, the first utterance sentence 1002b spoken by the first speaker 1002a is "Mr. A, where have you been in the afternoon?" The 2nd utterance sentence (1003b) spoken by the 2nd speaker (1003a) is, "Me. I've been to the Galilei Museum. I spent too much time there so I couldn't go anywhere else." The third utterance sentence (1004b) spoken by the third speaker (1004a) is, "I've been to the Galilei Museum. It's good, but if you go to Florence, you should have gone to the Duomo."

At this time, the processor 510 of the device 1001 providing information included in the media may convert voice data corresponding to the utterance sentences 1002b, 1003b, and 1004b into string data as shown in [Table 1] below. .

-Mr. A, where have you been in the afternoon?
- Me. I went to the Galilei Museum. I spent so much time there that I couldn't go anywhere else.
-You went to the Galilei Museum. It's great, but if you go to Florence, you should have been to Duomo.

Here, the processor 510 of the device 1001 providing information included in the media may convert the voice data included in the media content into string data in real time using the STT engine when the media is output (played). . If the media content to be played is not streamed in real time but is stored inside or other device of the device 1001 providing information included in the media, and the entire media content can be collected, the processor 510 is currently playing The voice data at the currently playing position may be converted into character string data in consideration of the voice data at a later point in time than the current location.

As described above, by considering voice data at a later time point than the currently output time point, the voice data at the current time point can be converted into string data with higher accuracy.

Referring to FIG. 11, a user 1101a watching media content through the device 1001 providing information included in the media wonders about the information included in the utterance sentences 1002b, 1003b, and 1004b. You did it?"(1101b).

At this time, the device 1001 providing information included in the media may receive query data corresponding to the query sentence 1101b spoken by the user 1101a through the microphone of the user terminal 1102. That is, the communication unit of the user terminal 1102 may transmit query data corresponding to the query sentence 1101b to the device 1001 providing information included in the media.

If the user terminal 1102 is not equipped with a microphone, the user terminal 1102 is not used, or the user does not want to use the user terminal 1102, providing information included in the media The microphone 540 provided in the terminal 1001 may receive the query sentence 1101b spoken by the user 1101a.

Referring to FIG. 12, the processor 510 of the device 1001 providing information included in the media corresponds to each of the user's query sentence 1101b and string data ([Table 1]) using a semantic analysis engine. Obtain semantic information.

At this time, the processor 510 of the device 1001 providing information included in the media may select the information suitable for the user's query sentence 1101b from the string data ([Table 1]) and generate response data.

At this time, the display unit 552 of the device 1001 providing information included in the media may display the response data 1201 visualized on a part of the screen.

For example, the visualized response data 1201 includes information related to a place'Galilee Museum' (1202), information about'Firenze' (1203), and a Duomo, which is information related to a place included in the string data of [Table 1]. Information about 1204 may be included.

In another embodiment, the audio output unit 551 of the device 1001 providing information included in the media may auditoryly output the response data and output the voice.

In another embodiment, the communication unit of the device 1001 providing information included in the media may transmit response data to the user terminal 1102, and the audio output unit of the user terminal 1102 may output the response data as voice. have.

In another embodiment, the communication unit of the device 1001 providing information included in the media may transmit response data to the user terminal 1102, and the display unit of the user terminal 1102 may visualize and output the response data. .

As described above, when the user wants to reconfirm any voice information while viewing the media containing the voice information, the user can easily check desired information by simply speaking and querying or querying through a button or menu of the user terminal.

According to an embodiment of the present invention, it is possible to implement the above-described method as code readable by a processor on a medium on which a program is recorded. Examples of the medium readable by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like, and implemented in the form of a carrier wave (for example, transmission through the Internet). Includes.

The display device described above is not limited to the configuration and method of the above-described embodiments, and the above embodiments may be configured by selectively combining all or part of each embodiment so that various modifications can be made. It might be.

Claims (12)

In the display device,
Convert sound data extracted from media content to string data, obtain first semantic information indicating the meaning of query data corresponding to a user's query, obtain second semantic information indicating the meaning of the string data, and obtain the second A processor that generates response data corresponding to the query data based on the character string data using the first semantic information and the second semantic information; And
Output unit for outputting the generated response data
Display device comprising a. The method according to claim 1,
The output unit
And at least one of a display unit or an audio output unit for outputting the media content. The method according to claim 1,
The processor
For each information included in the second semantic information, an association degree is calculated through comparison with the first semantic information, and at least one or more of the information is selected based on the calculated association degree to obtain the response data. Produced, display device. The method according to claim 1,
Communication unit that transmits and receives data to and from the management server through wired and wireless communication
Further comprising,
The processor
Converting the sound data into the string data using an STT engine,
Obtaining the first semantic information and the second semantic information using a semantic analysis engine,
A display device for updating at least one of the STT engine or the semantic analysis engine by receiving update data from the management server. The method according to claim 4,
The processor
For at least one of the STT engine or the semantic analysis engine, the display device performs at least one of an update for improving the accuracy of the engine or an update for modifying the language of the engine. The method according to claim 1,
The processor
A display device for determining a query range for the string data for generating the response data based on at least one of user interaction or electronic program guide (EPG) information corresponding to the media content. The method according to claim 1,
A communication unit that receives additional information corresponding to the response data from a network network connected through wired/wireless communication
Further comprising,
The processor
And adding the additional information to the response data. The method according to claim 1,
Microphone receiving the query data
Further comprising,
The processor
The processor converts the query data into the string data using a speech to text (STT) engine. The method according to claim 1,
Communication unit that receives the query data from the user terminal through wired or wireless communication
Further comprising, a display device. The method according to claim 9,
The output unit
A display device for transmitting the response data to the user terminal through the communication unit. In the output device,
Convert sound data extracted from media content to string data, obtain first semantic information indicating the meaning of query data corresponding to a user's query, obtain second semantic information indicating the meaning of the string data, and obtain the second A processor that generates response data corresponding to the query data based on the character string data using the first semantic information and the second semantic information; And
Communication unit for transmitting the response data to an external device
Included, the output device. The processor converting sound data extracted from the media content into string data;
Obtaining, by the processor, first semantic information indicating the meaning of query data corresponding to the user's query;
Obtaining, by the processor, second semantic information indicating the meaning of the string data; And
Generating, by the processor, response data corresponding to the query data based on the character string data by using the first semantic information and the second semantic information.
Method of providing information contained in the media, including.

Images