KR20170047547A - Display device and method for controlling the same - Google Patents

Abstract

A display device according to an embodiment of the present invention comprises: a memory for storing first information on subtitles related to specific content and playing time of the subtitles; a display unit for displaying video data of the specific content; an audio output unit for outputting audio data of the specific content; and a control unit for controlling the display unit to output the subtitles along with the video data by using the first information. The control unit recognizes second information on the time when a voice is sensed from the outputted audio data of the specific content, recognizes third information on the time when the subtitles are outputted by using the first information, and adjusts sync of the outputted subtitles in accordance with the comparison result of the second information and the third information.

Description

DISPLAY DEVICE AND METHOD FOR CONTROLLING THE SAME

The present invention relates to a display device and a control method thereof.

Mobile devices such as a smart phone and a tablet PC have attracted attention in addition to a standing device such as a personal computer (PC) and a television (TV). Fixed devices and mobile devices originally developed in their respective domains, but the area has become obscure due to the recent boom of digital convergence.

In recent years, there have been many cases in which a subtitle is output using a subtitle file stored in a memory when a video stored in a memory is reproduced.

However, the present technology has a disadvantage that the user has to manually synchronize the audio data when the subtitles are not synchronized with the audio data currently being output. Thus, the way to automatically synchronize subtitles remains a matter of debate.

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above-described situations and problems, and it is an object of the present invention to automatically control the synchronization of subtitles.

Another object of the present invention is to recognize, when a plurality of speakers are present in contents output from the display device, whether the subtitle being output is a subtitle related to a certain speaker and give different visual effects to the subtitle associated with each speaker .

The technical problem to be solved by the present invention is not limited to the above-described technical problems and other technical problems which are not mentioned can be clearly understood by those skilled in the art from the following description .

This document discloses various embodiments (s) of digital devices and processing methods in the digital devices.

A display device according to an embodiment of the present invention includes a memory for storing first information on a subtitle associated with a specific content and a reproduction time of the subtitle, a display unit for displaying video data of the specific content, And a controller for controlling the display unit so that the subtitles are output together with the video data using the first information, wherein the control unit controls the display unit so that audio is output from the audio data of the specific content being output, Recognizing second information on the detected time, recognizing third information on a time when the subtitles are output using the first information, determining, based on the comparison result of the second information and the third information, Adjusts the sync of the output subtitle.

According to another aspect of the present invention, there is provided a method of controlling a display device, the method comprising: displaying video data of a specific content; outputting audio data of the specific content; Outputting a subtitle associated with the specific content stored in the memory together with the video data, recognizing second information on a time when audio is detected in the audio data of the specific content being output, Recognizing third information on a time at which the caption is output using the information, and adjusting the sink of the caption being output according to the comparison result of the second information and the third information.

The technical solutions obtained by the present invention are not limited to the above-mentioned solutions, and other solutions not mentioned are clearly described to those skilled in the art from the following description. It can be understood.

Effects of the display device and the control method according to the present invention will be described as follows. According to an embodiment of the present invention, the user can automatically adjust the synchronization of subtitles on the TV without having to manually adjust the sync of the subtitles on the TV, so that the user can concentrate on the content being output.

According to an embodiment of the present invention, when there are a plurality of speakers in the output content, it is possible to recognize which subtitle is related to which speaker and output a different visual effect to each subtitle associated with each speaker, Has the advantage that the content can be easily viewed and understood.

The effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description will be.

1 is a schematic diagram illustrating a service system including a digital device according to an exemplary embodiment of the present invention.
2 is a block diagram illustrating a digital device according to an embodiment of the present invention
3 is a block diagram illustrating a digital device according to another embodiment of the present invention
4 is a block diagram illustrating a digital device according to another embodiment of the present invention
5 is a block diagram illustrating a detailed configuration of the control unit of FIGS. 2 to 4 according to an embodiment of the present invention
Figure 6 illustrates an input means coupled to the digital device of Figures 2 through 4 according to one embodiment of the present invention
7 is a block diagram illustrating configuration modules of a display device according to an embodiment of the present invention.
FIG. 8 is a flowchart illustrating an example of a method for adjusting the synchronization of subtitles in a display device according to an exemplary embodiment of the present invention. Referring to FIG.
9 and 10 are diagrams for explaining an example of a method of adjusting the synchronization of the caption when the subtitle output from the display device according to the embodiment of the present invention recognizes that the audio data being output is out of sync with the audio data being output FIG.
11 is a flowchart for explaining an example of a method of analyzing a voice of a recognized voice to give a visual effect to an outputting subtitle when a voice is recognized in audio data being output from a display device according to an embodiment of the present invention .
12 is a view for explaining an example of a method of analyzing a voice of a recognized voice and giving a visual effect to a subtitle being output when a voice is recognized in audio data being outputted from a display device according to an embodiment of the present invention FIG.
13 is a view for explaining an example of a method of mapping a visual effect assigned in accordance with a grammar in a display device according to an embodiment of the present invention to a memory.
FIG. 14 is a flowchart for explaining an example of a method of converting a voice of audio data being outputted from a display device into text and adjusting the synchronization of the caption using the translation result of the converted text according to an embodiment of the present invention.

Hereinafter, various embodiments (s) of a digital device according to the present invention and a method of processing application data in the digital device will be described in detail with reference to the drawings.

The suffix "module "," part ", and the like for components used in the present specification are given only for ease of specification, and both may be used as needed. Also, even when described in ordinal numbers such as " 1st ", "2nd ", and the like, it is not limited to such terms or ordinal numbers.

In addition, although the terms used in the present specification have been selected from the general terms that are widely used in the present invention in consideration of the functions according to the technical idea of the present invention, they are not limited to the intentions or customs of the artisan skilled in the art, It can be different. However, in certain cases, some terms are arbitrarily selected by the applicant, which will be described in the related description section. Accordingly, it should be understood that the term is to be interpreted based not only on its name but on its practical meaning as well as on the contents described throughout this specification.

It is to be noted that the contents of the present specification and / or drawings are not intended to limit the scope of the present invention.

The term "digital device" as used herein refers to a device that transmits, receives, processes, and outputs data, content, service, And includes all devices that perform at least one or more. The digital device can be paired or connected (hereinafter, referred to as 'pairing') with another digital device, an external server, or the like through a wire / wireless network, Can be transmitted / received. At this time, if necessary, the data may be appropriately converted before the transmission / reception. The digital device may be a standing device such as a network TV, a Hybrid Broadcast Broadband TV (HBBTV), a Smart TV, an IPTV (Internet Protocol TV), a PC (Personal Computer) And a mobile device or handheld device such as a PDA (Personal Digital Assistant), a smart phone, a tablet PC, a notebook, and the like. In order to facilitate understanding of the present invention and to facilitate the description of the present invention, FIG. 2, which will be described later, describes a digital TV, and FIG. 3 illustrates and describes a mobile device as an embodiment of a digital device. In addition, the digital device described in this specification may be a configuration having only a panel, a configuration such as a set-top box (STB), a device, a system, etc. and a set configuration .

The term "wired / wireless network" as used herein collectively refers to communication networks that support various communication standards or protocols for pairing and / or data transmission / reception between digital devices or digital devices and external servers. Such a wired / wireless network includes all of the communication networks to be supported by the standard now or in the future, and is capable of supporting one or more communication protocols therefor. Such a wired / wireless network includes, for example, a USB (Universal Serial Bus), a Composite Video Banking Sync (CVBS), a Component, an S-Video (Analog), a DVI (Digital Visual Interface) A communication standard or protocol for a wired connection such as an RGB or a D-SUB, a Bluetooth standard, a radio frequency identification (RFID), an infrared data association (IrDA), an ultra wideband (UWB) (ZigBee), DLNA (Digital Living Network Alliance), WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) A long term evolution (LTE-Advanced), and Wi-Fi direct, and a communication standard or protocol for the network.

In addition, when the term is simply referred to as a digital device in this specification, the meaning may mean a fixed device or a mobile device depending on the context, and may be used to mean both, unless specifically stated otherwise.

Meanwhile, a digital device is an intelligent device that supports, for example, a broadcast receiving function, a computer function or a support, at least one external input, and the like. The digital device may be an e-mail, web browsing, Banking, game, application, and so on. In addition, the digital device may include an interface for supporting at least one input or control means (hereinafter, " input means ") such as a handwriting input device, a touch- .

In addition, the digital device can use a standardized general-purpose OS (Operating System), but in particular, the digital device described in this specification uses the Web OS as an embodiment. Therefore, a digital device can handle adding, deleting, amending, and updating various services or applications on a general-purpose OS kernel or a Linux kernel. And through which a more user-friendly environment can be constructed and provided.

Meanwhile, the above-described digital device can receive and process an external input. The external input is connected to an external input device, that is, the digital device, through the wired / wireless network, An input means or a digital device. For example, the external input may be a game device such as a high-definition multimedia interface (HDMI), a playstation or an X-Box, a smart phone, a tablet PC, a pocket photo devices such as digital cameras, printing devices, smart TVs, Blu-ray device devices and the like.

In addition, the term "server" as used herein refers to a digital device or system that supplies data to or receives data from a digital device, that is, a client, and may be referred to as a processor do. The server provides a Web server, a portal server, and advertising data for providing a web page, a web content or a web content or a web service, An advertising server, a content server for providing content, an SNS server for providing a social network service (SNS), a service server provided by a manufacturer, a video on demand (VoD) server, A service server providing a Multichannel Video Programming Distributor (MVPD) for providing a streaming service, a pay service, and the like.

In addition, in the following description for convenience of explanation, only the application is described in the context of the present invention, and the meaning may include not only the application but also the service based on the context and the like. In addition, the application may refer to a web application according to the webOS platform.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic diagram illustrating a service system including a digital device according to an exemplary embodiment of the present invention.

1, a service system includes a content provider 10, a service provider 20, a network provider 30, and a Home Network End User (HNED) (Customer) 40. Here, the HNED 40 includes, for example, a client 100, i.e., a digital device according to the present invention.

The content provider 10 produces and provides various contents. As shown in FIG. 1, the content provider 10 may include a terrestrial broadcast sender, a cable SO (System Operator), a MSO (Multiple SO), a satellite broadcast sender, Personal content providers, and the like. On the other hand, the content provider 10 can produce and provide various services and applications in addition to the broadcast content.

The service provider 20 provides service packetizing of the content produced by the content provider 10 to the HNED 40. [ For example, the service provider 20 packages at least one of the first terrestrial broadcast, the second terrestrial broadcast, the cable MSO, the satellite broadcast, the various internet broadcast, and the contents produced by the application for service, (40).

The service provider 20 provides services to the client 100 in a uni-cast or multi-cast manner. Meanwhile, the service provider 20 can transmit data to a plurality of clients 100 registered in advance, and an Internet Group Management Protocol (IGMP) protocol can be used for this purpose.

The above-described content provider 10 and the service provider 20 may be the same entity. For example, the service provided by the content provider 10 may be packaged and provided to the HNED 40, thereby performing the function of the service provider 20 or vice versa.

The network provider 30 provides a network for exchanging data between the content provider 10 and / or the service provider 20 and the client 100.

The client 100 is a consumer belonging to the HNED 40. The client 100 constructs a home network through the network provider 30 to receive data and stores data related to various services and applications such as VoD and streaming / RTI >

Meanwhile, the content provider 10 or / and the service provider 20 in the service system can use conditional access or content protection means for protecting the content to be transmitted. Accordingly, the client 100 can use a processing means such as a cable card (or a POD: Point of Deployment), a DCAS (Downloadable CAS) or the like in response to the restriction reception or the content protection.

In addition, the client 100 can use the two-way service through the network. Accordingly, the client 100 may rather perform the function or role of the content provider, and the service provider 20 may receive the content and transmit the same to another client or the like.

In FIG. 1, the content provider 10 and / or the service provider 20 may be a server that provides a service described later in this specification. In this case, the server may also mean to own or include network provider 30 as needed. The service or service data includes an internal service or an application, as well as a service or an application received from the outside, and the service or the service includes a service for a Web OS-based client 100 Or application data.

2 is a block diagram illustrating a digital device according to an exemplary embodiment of the present invention.

The digital device described herein corresponds to the client 100 of FIG. 1 described above.

The digital device 200 includes a network interface 201, a TCP / IP manager 202, a service delivery manager 203, an SI decoder 204, A demultiplexer (demux or demultiplexer) 205, an audio decoder 206, a video decoder 207, a display A / V and OSD module 208, a service management manager 209, a service discovery manager 210, an SI & metadata DB 211, a metadata manager 212, a service manager 213, A UI manager 214, and the like.

The network interface unit 201 receives the IP packet (s) (IP packet (s)) or the IP datagram (s) ) Is transmitted / received. For example, the network interface unit 201 can receive services, applications, content, and the like from the service provider 20 of FIG. 1 through a network.

The TCP / IP manager 202 determines whether the IP packets received by the digital device 200 and the IP packets transmitted by the digital device 200 are packet delivery (i.e., packet delivery) between a source and a destination packet delivery). The service discovery manager 210, the service control manager 209, the meta data manager 212, the service discovery manager 210, the service control manager 209, and the metadata manager 212. The TCP / IP manager 202 classifies the received packet (s) ) Or the like.

The service delivery manager 203 is responsible for controlling the received service data. For example, the service delivery manager 203 may use RTP / RTCP when controlling real-time streaming data. When the real-time streaming data is transmitted using the RTP, the service delivery manager 203 parses the received data packet according to the RTP and transmits the packet to the demultiplexing unit 205 or the control of the service manager 213 In the SI & meta data database 211. [ Then, the service delivery manager 203 feedbacks the network reception information to the server providing the service using RTCP.

The demultiplexer 205 demultiplexes the received packets into audio, video, SI (System Information) data, and transmits them to the audio / video decoder 206/207 and the SI decoder 204, respectively.

The SI decoder 204 decodes the demultiplexed SI data, that is, Program Specific Information (PSI), Program and System Information Protocol (PSIP), Digital Video Broadcasting Service Information (DVB-SI), Digital Television Terrestrial Multimedia Broadcasting / Coding Mobile Multimedia Broadcasting). Also, the SI decoder 204 may store the decoded service information in the SI & meta data database 211. The stored service information can be read out and used by the corresponding configuration, for example, by a user's request.

The audio / video decoder 206/207 decodes each demultiplexed audio data and video data. The decoded audio data and video data are provided to the user through the display unit 208. [

The application manager may include, for example, the UI manager 214 and the service manager 213 and may perform the functions of the controller of the digital device 200. [ In other words, the application manager can manage the overall state of the digital device 200, provide a user interface (UI), and manage other managers.

The UI manager 214 provides a GUI (Graphic User Interface) / UI for a user using an OSD (On Screen Display) or the like, and receives a key input from a user to perform a device operation according to the input. For example, the UI manager 214 transmits the key input signal to the service manager 213 upon receipt of a key input from the user regarding channel selection.

The service manager 213 controls the manager associated with the service such as the service delivery manager 203, the service discovery manager 210, the service control manager 209, and the metadata manager 212.

The service manager 213 also generates a channel map and controls the selection of a channel using the generated channel map according to the key input received from the UI manager 214. [ The service manager 213 receives the service information from the SI decoder 204 and sets an audio / video PID (Packet Identifier) of the selected channel in the demultiplexer 205. The PID thus set can be used in the demultiplexing process described above. Accordingly, the demultiplexer 205 filters (PID or section) audio data, video data, and SI data using the PID.

The service discovery manager 210 provides information necessary for selecting a service provider that provides the service. Upon receiving a signal regarding channel selection from the service manager 213, the service discovery manager 210 searches for the service using the information.

The service control manager 209 is responsible for selection and control of services. For example, the service control manager 209 uses IGMP or RTSP when a user selects a live broadcasting service such as an existing broadcasting system, and selects a service such as VOD (Video on Demand) , RTSP is used to select and control services. The RTSP protocol may provide a trick mode for real-time streaming. In addition, the service control manager 209 can initialize and manage a session through the IMS gateway 250 using an IP Multimedia Subsystem (IMS) and a Session Initiation Protocol (SIP). The protocols are one embodiment, and other protocols may be used, depending on the implementation.

The metadata manager 212 manages the metadata associated with the service and stores the metadata in the SI & metadata database 211.

The SI & meta data database 211 stores service information decoded by the SI decoder 204, meta data managed by the meta data manager 212, and information necessary for selecting a service provider provided by the service discovery manager 210 . In addition, the SI & meta data database 211 may store set-up data for the system and the like.

The SI & meta data database 211 may be implemented using a non-volatile RAM (NVRAM) or a flash memory.

Meanwhile, the IMS gateway 250 is a gateway that collects functions necessary for accessing the IMS-based IPTV service.

3 is a block diagram illustrating a digital device according to another embodiment of the present invention.

If the above-described Fig. 2 is described with an example of a digital device as a fixing device, Fig. 3 shows a mobile device as another embodiment of a digital device.

3, the mobile device 300 includes a wireless communication unit 310, an A / V input unit 320, a user input unit 330, a sensing unit 340, an output unit 350, A memory 360, an interface unit 370, a control unit 380, a power supply unit 390, and the like.

Hereinafter, each component will be described in detail.

The wireless communication unit 310 may include one or more modules that enable wireless communication between the mobile device 300 and the wireless communication system or between the mobile device and the network in which the mobile device is located. For example, the wireless communication unit 310 may include a broadcast receiving module 311, a mobile communication module 312, a wireless Internet module 313, a short range communication module 314, and a location information module 315 .

The broadcast receiving module 311 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. Here, the broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 312.

The broadcast-related information may exist in various forms, for example, in the form of an EPG (Electronic Program Guide) or an ESG (Electronic Service Guide).

The broadcast receiving module 311 may be, for example, an ATSC, a Digital Video Broadcasting-Terrestrial (DVB-T), a Satellite (DVB-S), a Media Forward Link Only And Integrated Services Digital Broadcast-Terrestrial (DRS). Of course, the broadcast receiving module 311 may be adapted to not only the above-described digital broadcasting system but also other broadcasting systems.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 311 may be stored in the memory 360.

The mobile communication module 312 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data depending on a voice signal, a video call signal, or a text / multimedia message transmission / reception.

The wireless Internet module 313 may be embedded or external to the mobile device 300, including a module for wireless Internet access. WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) and the like can be used as wireless Internet technologies.

The short-range communication module 314 is a module for short-range communication. Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IRDA), Ultra Wideband (UWB), ZigBee, RS-232 and RS-485 are used as short range communication technology. .

The position information module 315 is a module for acquiring position information of the mobile device 300, and may be a GPS (Global Position System) module.

The A / V input unit 320 is for inputting audio and / or video signals. The A / V input unit 320 may include a camera 321 and a microphone 322. The camera 321 processes an image frame such as a still image or moving image obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame can be displayed on the display section 351. [

The image frame processed by the camera 321 may be stored in the memory 360 or transmitted to the outside via the wireless communication unit 310. [ At least two cameras 321 may be provided depending on the use environment.

The microphone 322 receives an external sound signal by a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 312 in the case of the communication mode, and output. The microphone 322 may be implemented with various noise reduction algorithms for eliminating noise generated in receiving an external sound signal.

The user input unit 330 generates input data for user's operation control of the terminal. The user input unit 330 may include a key pad, a dome switch, a touch pad (static pressure / static electricity), a jog wheel, a jog switch, and the like.

The sensing unit 340 senses the current state of the mobile device 300 such as the open / closed state of the mobile device 300, the position of the mobile device 300, the user's contact, the orientation of the mobile device, And generates a sensing signal for controlling the operation of the mobile device 300. For example, when the mobile device 300 is moved or tilted, it may sense the position, slope, etc. of the mobile device. It is also possible to sense whether power is supplied to the power supply unit 390, whether the interface unit 370 is connected to an external device, and the like. Meanwhile, the sensing unit 240 may include a proximity sensor 341 including NFC (Near Field Communication).

The output unit 350 may include a display unit 351, an acoustic output module 352, an alarm unit 353, and a haptic module 354 for generating output related to visual, auditory, have.

The display unit 351 displays (outputs) information processed by the mobile device 300. [ For example, if the mobile device is in call mode, it displays a UI or GUI associated with the call. When the mobile device 300 is in the video communication mode or the photographing mode, the photographed and / or received video or UI and GUI are displayed.

The display unit 351 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED) flexible display, and a three-dimensional display.

Some of these displays may be transparent or light transmissive so that they can be seen through. This can be referred to as a transparent display, and a typical example of the transparent display is TOLED (Transparent OLED) and the like. The rear structure of the display portion 351 may also be of a light transmission type. With this structure, the user can see an object located behind the terminal body through the area occupied by the display unit 351 of the terminal body.

There may be two or more display units 351 depending on the implementation of the mobile device 300. [ For example, in the mobile device 300, the plurality of display portions may be spaced apart or integrally arranged on one surface, and may be disposed on different surfaces, respectively.

The display unit 351 includes a display unit 351 and a display unit 351. When the display unit 351 and the sensor for sensing a touch operation (hereinafter, referred to as 'touch sensor') have a mutual layer structure It can also be used as a device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display portion 351 or a capacitance generated in a specific portion of the display portion 351 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits corresponding data to the controller 380. Thus, the control unit 380 can know which area of the display unit 351 is touched or the like.

A proximity sensor 341 may be disposed in the interior area of the mobile device or in proximity to the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as "proximity touch & The act of actually touching the pointer on the screen is called "contact touch. &Quot; The position where the pointer is proximately touched on the touch screen means a position where the pointer is vertically corresponding to the touch screen when the pointer is touched.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

The sound output module 352 can output audio data received from the wireless communication unit 310 or stored in the memory 360 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 352 also outputs sound signals associated with functions performed on the mobile device 300 (e.g., call signal receive tones, message receive tones, etc.). The sound output module 352 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 353 outputs a signal for notifying the occurrence of an event of the mobile device 300. [ Examples of events that occur in the mobile device include receiving a call signal, receiving a message, inputting a key signal, and touch input. The alarm unit 353 may output a signal for informing occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. The video signal or the audio signal may be output through the display unit 351 or the audio output module 352 so that they may be classified as a part of the alarm unit 353.

The haptic module 354 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 354 is vibration. The intensity and pattern of the vibration generated by the haptic module 354 are controllable. For example, different vibrations may be synthesized and output or sequentially output. In addition to the vibration, the haptic module 354 may be arranged in a variety of ways, such as a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or suction force of the air through the injection port or the suction port, a spit on the skin surface, contact with an electrode, Various effects such as an effect of heat generation and an effect of reproducing a cool / warm feeling using a heat absorbing or heatable element can be generated. The haptic module 354 can be implemented not only to transmit the tactile effect through direct contact but also to allow the user to feel the tactile effect through the muscular sensation of a finger or an arm. More than one haptic module 354 may be provided according to the configuration of the mobile device 300.

The memory 360 may store a program for the operation of the control unit 380 and temporarily store input / output data (e.g., phone book, message, still image, moving picture, etc.). The memory 360 may store data related to vibration and sound of various patterns output during touch input on the touch screen.

The memory 360 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory) (Random Access Memory), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM) A magnetic disk, and / or an optical disk. The mobile device 300 may operate in association with a web storage that performs storage functions of the memory 360 on the Internet.

The interface unit 370 serves as a pathway to all the external devices connected to the mobile device 300. The interface unit 370 receives data from an external device or receives power from the external device and transfers the data to each component in the mobile device 300 or transmits data in the mobile device 300 to an external device. For example, it may be provided with a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port connecting a device with an identification module, an audio I / O port, A video I / O port, an earphone port, and the like may be included in the interface unit 370.

The identification module is a chip for storing various information for authenticating the usage right of the mobile device 300 and includes a user identification module (UIM), a subscriber identity module (SIM), a general user authentication module A Universal Subscriber Identity Module (USIM), and the like. A device having an identification module (hereinafter referred to as 'identification device') can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 200 through the port.

The interface unit 370 may be a communication path through which the power from the cradle is supplied to the mobile device 300 when the mobile device 300 is connected to an external cradle, A command signal may be the path through which it is communicated to the mobile device. The various command signals or the power supply input from the cradle may be operated with a signal for recognizing that the mobile device is correctly mounted on the cradle.

The control unit 380 typically controls the overall operation of the mobile device 300. The control unit 380 performs related control and processing, for example, for voice call, data communication, video call, and the like. The control unit 380 may include a multimedia module 381 for multimedia playback. The multimedia module 381 may be implemented in the control unit 380 or separately from the control unit 380. The control unit 380 can perform pattern recognition processing for recognizing the handwriting input or the drawing input performed on the touch-screen as characters and images, respectively.

The power supply unit 390 receives external power and internal power under the control of the controller 380 and supplies power necessary for operation of the respective components.

The various embodiments described herein may be implemented in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of a processor, a controller, micro-controllers, microprocessors, and an electrical unit for performing other functions. In some cases, the implementation described herein Examples may be implemented by the control unit 380 itself.

According to a software implementation, embodiments such as the procedures and functions described herein may be implemented with separate software modules. Each of the software modules may perform one or more of the functions and operations described herein. Software code may be implemented in a software application written in a suitable programming language. Here, the software code is stored in the memory 360 and can be executed by the control unit 380. [

4 is a block diagram illustrating a digital device according to another embodiment of the present invention.

Another example of the digital device 400 includes a broadcast receiving unit 405, an external device interface unit 435, a storage unit 440, a user input interface unit 450, a control unit 470, a display unit 480, An output unit 485, a power supply unit 490, and a photographing unit (not shown). The broadcast receiver 405 may include at least one tuner 410, a demodulator 420, and a network interface 430. In some cases, the broadcast receiver 405 may include a tuner 410 and a demodulator 420, but may not include the network interface 430, or vice versa. The broadcast receiving unit 405 may include a multiplexer to receive a demodulated signal from the demodulator 420 via the tuner 410 and a signal received via the network interface 430, May be multiplexed. In addition, although not shown, the broadcast receiver 425 includes a demultiplexer to demultiplex the multiplexed signals, demultiplex the demodulated signals or the signals passed through the network interface 430 .

The tuner 410 tunes a channel selected by the user or all pre-stored channels of an RF (Radio Frequency) broadcast signal received through the antenna, and receives the RF broadcast signal. In addition, the tuner 410 converts the received RF broadcast signal into an intermediate frequency (IF) signal or a baseband signal.

For example, if the received RF broadcast signal is a digital broadcast signal, the signal is converted into a digital IF signal (DIF). If the received RF broadcast signal is an analog broadcast signal, the signal is converted into an analog baseband image or a voice signal (CVBS / SIF). That is, the tuner 410 can process both a digital broadcast signal and an analog broadcast signal. The analog baseband video or audio signal (CVBS / SIF) output from the tuner 410 can be directly input to the controller 470.

In addition, the tuner 410 can receive RF broadcast signals of a single carrier or a multiple carrier. Meanwhile, the tuner 410 sequentially tunes and receives RF broadcast signals of all the broadcast channels stored through the channel storage function among the RF broadcast signals received through the antenna, converts the RF broadcast signals into intermediate frequency signals or baseband signals (DIF: Digital Intermediate Frequency or baseband signal).

The demodulator 420 may receive and demodulate the digital IF signal DIF converted by the tuner 410 and perform channel decoding. For this, the demodulator 420 may include a trellis decoder, a de-interleaver, a Reed-Solomon decoder, or a convolution decoder, a deinterleaver, - Solomon decoder and the like.

The demodulation unit 420 may perform demodulation and channel decoding, and then output a stream signal TS. At this time, the stream signal may be a signal in which a video signal, a voice signal, or a data signal is multiplexed. For example, the stream signal may be an MPEG-2 TS (Transport Stream) multiplexed with an MPEG-2 standard video signal, a Dolby AC-3 standard audio signal, or the like.

The stream signal output from the demodulation unit 420 may be input to the control unit 470. The control unit 470 controls demultiplexing, video / audio signal processing, and the like, and controls the output of audio through the display unit 480 and audio through the audio output unit 485.

The external device interface unit 435 provides an interface environment between the digital device 300 and various external devices. To this end, the external device interface unit 335 may include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

The external device interface unit 435 may be a digital versatile disk (DVD), a Blu-ray, a game device, a camera, a camcorder, a computer (notebook), a tablet PC, a smart phone, device, an external device such as a cloud, or the like. The external device interface unit 435 transmits a signal including data such as image, image, and voice input through the connected external device to the control unit 470 of the digital device. The control unit 470 can control the processed image, image, voice, and the like to be output to the external device to which the data signal is connected. To this end, the external device interface unit 435 may further include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

The A / V input / output unit includes a USB terminal, a CVBS (Composite Video Banking Sync) terminal, a component terminal, an S-video terminal (analog) terminal, A DVI (Digital Visual Interface) terminal, an HDMI (High Definition Multimedia Interface) terminal, an RGB terminal, a D-SUB terminal, and the like.

The wireless communication unit can perform short-range wireless communication with another digital device. The digital device 400 may be a Bluetooth device such as Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (UDA), Ultra Wideband (UWB), ZigBee, Digital Living Network Alliance (DLNA) Lt; RTI ID = 0.0 > and / or < / RTI > other communication protocols.

Also, the external device interface unit 435 may be connected to the set-top box STB through at least one of the various terminals described above to perform input / output operations with the set-top box STB.

Meanwhile, the external device interface unit 435 may receive an application or an application list in an adjacent external device, and may transmit the received application or application list to the control unit 470 or the storage unit 440.

The network interface unit 430 provides an interface for connecting the digital device 400 to a wired / wireless network including the Internet network. The network interface unit 430 may include an Ethernet terminal or the like for connection with a wired network and may be a WLAN (Wireless LAN) (Wi- Fi, Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), and HSDPA (High Speed Downlink Packet Access) communication standards.

The network interface unit 430 can transmit or receive data to another user or another digital device via the connected network or another network linked to the connected network. In particular, some of the content data stored in the digital device 400 can be transmitted to a selected user or selected one of other users or other digital devices previously registered with the digital device 400. [

Meanwhile, the network interface unit 430 can access a predetermined web page through the connected network or another network linked to the connected network. That is, it is possible to access a predetermined web page through a network and transmit or receive data with the server. In addition, content or data provided by a content provider or a network operator may be received. That is, it can receive contents and related information of movies, advertisements, games, VOD, broadcasting signals, etc., provided from a content provider or a network provider through a network. In addition, it can receive update information and an update file of firmware provided by the network operator. It may also transmit data to the Internet or to a content provider or network operator.

In addition, the network interface unit 430 can select and receive a desired application from the open applications through the network.

The storage unit 440 may store a program for each signal processing and control in the control unit 470 or may store a signal-processed video, audio, or data signal.

The storage unit 440 may also function for temporarily storing video, audio, or data signals input from the external device interface unit 435 or the network interface unit 430. [ The storage unit 440 can store information on a predetermined broadcast channel through the channel memory function.

The storage unit 440 may store a list of applications or applications input from the external device interface unit 435 or the network interface unit 330.

In addition, the storage unit 440 may store various platforms described later.

The storage unit 440 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD Memory, etc.), RAM (RAM), and ROM (EEPROM, etc.). The digital device 400 may reproduce and provide a content file (a moving image file, a still image file, a music file, a document file, an application file, etc.) stored in the storage unit 440 to a user.

4 illustrates an embodiment in which the storage unit 440 is provided separately from the control unit 470, the present invention is not limited thereto. In other words, the storage unit 440 may be included in the control unit 470.

The user input interface unit 450 transfers a signal input by the user to the controller 470 or a signal from the controller 470 to the user.

For example, the user input interface unit 450 may control power on / off, channel selection, screen setting, etc. from the remote control device 500 according to various communication methods such as an RF communication method and an infrared (IR) And may transmit the control signal of the control unit 470 to the remote control device 500. [

In addition, the user input interface unit 450 can transmit a control signal input from a local key (not shown) such as a power key, a channel key, a volume key, and a set value to the controller 470.

The user input interface unit 450 transmits a control signal input from a sensing unit (not shown) that senses a gesture of a user to the control unit 470 or transmits a signal of the control unit 470 to a sensing unit (Not shown). Here, the sensing unit (not shown) may include a touch sensor, an audio sensor, a position sensor, an operation sensor, and the like.

The control unit 470 demultiplexes the streams input through the tuner 410, the demodulation unit 420, or the external device interface unit 435 or processes the demultiplexed signals to generate a signal for video or audio output And can output.

The video signal processed by the control unit 470 may be input to the display unit 480 and displayed as an image corresponding to the video signal. The video signal processed by the controller 470 may be input to the external output device through the external device interface unit 435. [

The audio signal processed by the control unit 470 may be audio-output to the audio output unit 485. The voice signal processed by the control unit 470 may be input to the external output device through the external device interface unit 435. [

Although not shown in FIG. 4, the control unit 470 may include a demultiplexing unit, an image processing unit, and the like.

The control unit 470 can control the overall operation of the digital device 400. [ For example, the controller 470 may control the tuner 410 to control tuning of a RF broadcast corresponding to a channel selected by the user or a previously stored channel.

The control unit 470 can control the digital device 400 by a user command or an internal program input through the user input interface unit 450. [ In particular, the user can access the network and allow a user to download a desired application or application list into the digital device 400.

For example, the control unit 470 controls the tuner 410 so that a signal of a selected channel is input according to a predetermined channel selection command received through the user input interface unit 450. And processes video, audio or data signals of the selected channel. The control unit 470 allows the display unit 480 or the audio output unit 485 to output the video or audio signal processed by the user through the channel information selected by the user.

The control unit 470 may be connected to the external device interface unit 435 through an external device such as a camera or a camcorder in accordance with an external device video playback command received through the user input interface unit 450. [ So that the video signal or the audio signal of the video signal can be output through the display unit 480 or the audio output unit 485.

On the other hand, the control unit 470 can control the display unit 480 to display an image. For example, a broadcast image input through the tuner 410, an external input image input through the external device interface unit 435, an image input through the network interface unit, or an image stored in the storage unit 440 , And display on the display unit 480. At this time, the image displayed on the display unit 480 may be a still image or a moving image, and may be a 2D image or a 3D image.

In addition, the control unit 470 can control to reproduce the content. The content at this time may be the content stored in the digital device 400, or the received broadcast content, or an external input content input from the outside. The content may be at least one of a broadcast image, an external input image, an audio file, a still image, a connected web screen, and a document file.

On the other hand, when entering the application view item, the control unit 470 can control to display a list of applications or applications that can be downloaded from the digital device 300 or from an external network.

The control unit 470, in addition to various user interfaces, can control to install and drive an application downloaded from the external network. In addition, by the user's selection, it is possible to control the display unit 480 to display an image related to the executed application.

Although not shown in the drawing, a channel browsing processing unit for generating a channel signal or a thumbnail image corresponding to an external input signal may be further provided.

The channel browsing processing unit receives a stream signal TS output from the demodulation unit 320 or a stream signal output from the external device interface unit 335 and extracts an image from an input stream signal to generate a thumbnail image . The generated thumbnail image may be encoded as it is or may be input to the controller 470. In addition, the generated thumbnail image may be encoded in a stream form and input to the controller 470. The control unit 470 may display a thumbnail list having a plurality of thumbnail images on the display unit 480 using the input thumbnail images. On the other hand, the thumbnail images in this thumbnail list can be updated in sequence or simultaneously. Accordingly, the user can easily grasp the contents of a plurality of broadcast channels.

The display unit 480 converts an image signal, a data signal, an OSD signal processed by the control unit 470 or a video signal and a data signal received from the external device interface unit 435 into R, G, and B signals, respectively Thereby generating a driving signal.

The display unit 480 may be a PDP, an LCD, an OLED, a flexible display, a 3D display, or the like.

Meanwhile, the display unit 480 may be configured as a touch screen and used as an input device in addition to the output device.

The audio output unit 485 receives a signal processed by the control unit 470, for example, a stereo signal, a 3.1 channel signal, or a 5.1 channel signal, and outputs it as a voice. The audio output unit 485 may be implemented by various types of speakers.

In order to detect the gesture of the user, a sensing unit (not shown) having at least one of a touch sensor, a voice sensor, a position sensor, and an operation sensor may be further included in the digital device 400 . A signal sensed by a sensing unit (not shown) may be transmitted to the controller 3470 through the user input interface unit 450.

On the other hand, a photographing unit (not shown) for photographing a user may be further provided. The image information photographed by the photographing unit (not shown) may be input to the control unit 470.

The control unit 470 may detect the gesture of the user by combining the images photographed by the photographing unit (not shown) or the sensed signals from the sensing unit (not shown).

The power supply unit 490 supplies the corresponding power to the digital device 400.

Particularly, it is possible to supply power to a control unit 470, a display unit 480 for displaying an image, and an audio output unit 485 for audio output, which can be implemented in the form of a system on chip (SoC) .

To this end, the power supply unit 490 may include a converter (not shown) for converting AC power to DC power. Meanwhile, for example, when the display unit 480 is implemented as a liquid crystal panel having a plurality of backlight lamps, an inverter (not shown) capable of PWM (Pulse Width Modulation) operation for variable luminance or dimming driving and an inverter (not shown).

The remote control device 500 transmits the user input to the user input interface unit 450. To this end, the remote control device 500 can use Bluetooth, RF (radio frequency) communication, infrared (IR) communication, UWB (Ultra Wideband), ZigBee, or the like.

Also, the remote control device 500 can receive the video, audio, or data signal output from the user input interface unit 450 and display it on the remote control device 500 or output sound or vibration.

The digital device 400 may be a digital broadcast receiver capable of processing digital broadcast signals of a fixed or mobile ATSC scheme or a DVB scheme.

In addition, the digital device according to the present invention may further include a configuration that omits some of the configuration shown in FIG. On the other hand, unlike the above, the digital device does not have a tuner and a demodulator, and can receive and reproduce the content through the network interface unit or the external device interface unit.

FIG. 5 is a block diagram illustrating a detailed configuration of the control unit of FIGS. 2 to 4 according to an embodiment of the present invention. Referring to FIG.

An example of the control unit includes a demultiplexer 510, an image processor 5520, an OSD generator 540, a mixer 550, a frame rate converter (FRC) 555, And may include a formatter 560. The control unit may further include a voice processing unit and a data processing unit.

The demultiplexer 510 demultiplexes the input stream. For example, the demultiplexer 510 may demultiplex the received MPEG-2 TS video, audio, and data signals. Here, the stream signal input to the demultiplexer 510 may be a stream signal output from a tuner, a demodulator, or an external device interface.

The image processing unit 420 performs image processing of the demultiplexed image signal. To this end, the image processing unit 420 may include a video decoder 425 and a scaler 435.

The video decoder 425 decodes the demultiplexed video signal, and the scaler 435 scales the decoded video signal so that the resolution of the decoded video signal can be output from the display unit.

The video decoder 525 may support various standards. For example, the video decoder 525 performs the function of an MPEG-2 decoder when the video signal is encoded in the MPEG-2 standard, and the video decoder 525 encodes the video signal in the DMB (Digital Multimedia Broadcasting) It can perform the function of the H.264 decoder.

On the other hand, the video signal decoded by the video processor 520 is input to the mixer 450.

The OSD generation unit 540 generates OSD data according to a user input or by itself. For example, the OSD generating unit 440 generates data for displaying various data in graphic or text form on the screen of the display unit 380 based on the control signal of the user input interface unit. The generated OSD data includes various data such as a user interface screen of a digital device, various menu screens, a widget, an icon, and viewing rate information. The OSD generation unit 540 may generate data for displaying broadcast information based on the caption of the broadcast image or the EPG.

The mixer 550 mixes the OSD data generated by the OSD generator 540 and the image signal processed by the image processor, and provides the mixed image to the formatter 560. Since the decoded video signal and the OSD data are mixed, the OSD is overlaid on the broadcast image or the external input image.

A frame rate conversion unit (FRC) 555 converts a frame rate of an input image. For example, the frame rate conversion unit 555 may convert the frame rate of the input 60 Hz image to have a frame rate of 120 Hz or 240 Hz, for example, in accordance with the output frequency of the display unit. As described above, there are various methods for converting the frame rate. For example, when converting the frame rate from 60 Hz to 120 Hz, the frame rate conversion unit 555 may insert the same first frame between the first frame and the second frame, Three frames can be inserted. As another example, when converting the frame rate from 60 Hz to 240 Hz, the frame rate conversion unit 555 may insert and convert three or more identical or predicted frames between existing frames. On the other hand, when the frame conversion is not performed, the frame rate conversion unit 555 may be bypassed.

The formatter 560 changes the output of the input frame rate conversion unit 555 to match the output format of the display unit. For example, the formatter 560 may output R, G, and B data signals, and the R, G, and B data signals may be output as low voltage differential signals (LVDS) or mini-LVDS . If the output of the input frame rate converter 555 is a 3D video signal, the formatter 560 may configure and output the 3D format according to the output format of the display unit to support the 3D service through the display unit.

On the other hand, the voice processing unit (not shown) in the control unit can perform the voice processing of the demultiplexed voice signal. Such a voice processing unit (not shown) may support processing various audio formats. For example, even when a voice signal is coded in a format such as MPEG-2, MPEG-4, AAC, HE-AAC, AC-3, or BSAC, a corresponding decoder can be provided.

In addition, the voice processing unit (not shown) in the control unit can process the base, the treble, the volume control, and the like.

A data processing unit (not shown) in the control unit can perform data processing of the demultiplexed data signal. For example, the data processing unit can decode the demultiplexed data signal even when it is coded. Here, the encoded data signal may be EPG information including broadcast information such as a start time and an end time of a broadcast program broadcasted on each channel.

On the other hand, the above-described digital device is an example according to the present invention, and each component can be integrated, added, or omitted according to specifications of a digital device actually implemented. That is, if necessary, two or more components may be combined into one component, or one component may be divided into two or more components. In addition, the functions performed in each block are intended to illustrate the embodiments of the present invention, and the specific operations and devices thereof do not limit the scope of rights of the present invention.

Meanwhile, the digital device may be a video signal processing device that performs signal processing of an image stored in the device or an input image. Other examples of the video signal processing device include a set-top box (STB), a DVD player, a Blu-ray player, a game device, a computer Etc. can be further exemplified.

FIG. 6 is a diagram illustrating input means coupled to the digital device of FIGS. 2 through 4 according to one embodiment of the present invention.

A front panel (not shown) or a control means (input means) provided on the digital device 600 is used to control the digital device 600.

The control means includes a remote controller 610, a keyboard 630, a pointing device 620, and a keyboard 620, which are mainly implemented for the purpose of controlling the digital device 600, as a user interface device (UID) A touch pad, or the like, but may also include control means dedicated to external input connected to the digital device 600. [ In addition, a control device may include a mobile device such as a smart phone, a tablet PC, or the like that controls the digital device 600 through a mode switching or the like, although it is not a control object of the digital device 600. In the following description, a pointing device will be described as an example, but the present invention is not limited thereto.

The input means may be a communication protocol such as Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (UDA), Ultra Wideband (UWB), ZigBee, Digital Living Network Alliance (DLNA) At least one can be employed as needed to communicate with the digital device.

The remote controller 610 is a conventional input device having various key buttons necessary for controlling the digital device 600. [

The pointing device 620 may include a gyro sensor or the like to implement a corresponding pointer on the screen of the digital device 600 based on the user's motion, The control command is transmitted. Such a pointing device 620 may be named with various names such as a magic remote controller, a magic controller, and the like.

Since the digital device 600 provides a variety of services such as a web browser, an application, and a social network service (SNS) as an intelligent integrated digital device beyond the conventional digital device 600 providing only conventional broadcasting, It is not easy, and it is implemented to complement input and realize input convenience such as text by implementing similar to PC keyboard.

On the other hand, the control means such as the remote control 610, the pointing device 620 and the keyboard 630 may be provided with a touch pad as required to provide more convenient and various control purposes such as text input, pointer movement, .

The digital device described in this specification uses the Web OS as an OS and / or platform. Hereinafter, the processing such as the configuration or algorithm based on the web OS can be performed in the control unit of the above-described digital device or the like. Here, the control unit includes the control unit in FIGS. 2 to 5 described above and uses it as a broad concept. Accordingly, in order to process services, applications, contents, and the like related to the web OS in the digital device, the hardware and components including related software, firmware, and the like are controlled by a controller Named and explained.

Such a web OS-based platform is intended to enhance development independence and function expandability by integrating services, applications, and the like based on, for example, a luna-service bus, Productivity can be increased. Also, multi-tasking can be supported by efficiently utilizing system resources and the like through a Web OS process and resource management.

Meanwhile, the web OS platform described in this specification can be used not only in fixed devices such as a PC, a TV, and a set-top box (STB) but also in mobile devices such as mobile phones, smart phones, tablet PCs, notebooks, wearable devices .

The structure of the software for digital devices is based on a single process and a closed product based on multi-threading with conventional problem solving and market-dependent monolithic structure, And has been pursuing new platform-based development since then, has pursued cost innovation through chip-set replacement, UI application and external application development efficiency, and developed layering and componentization Layer structure and an add-on structure for add-ons, single-source products, and open applications. More recently, the software architecture provides a modular architecture for functional units, a Web Open API (Application Programming Interface) for echo-systems, and a game engine And a native open API (Native Open API), and thus, a multi-process structure based on a service structure is being created.

Hereinafter, embodiments of the present invention will be described with reference to FIGS. 7 to 14. FIG. In the following, reference will be made to the above description with reference to Figs. 1 to 6 in explaining and understanding the embodiments of the present invention. Also, the above-described display device or digital device may be a display device according to an embodiment of the present invention.

The display device described in the present specification can be applied to various devices such as a TV, a smart TV, a Hybrid Brodacast Broadband Television (HBBTV), a network TV, a Web TV, an Internet Protocol Television (IPTV), a personal computer, a tablet PC, a notebook, A smart phone, a smart watch, a smart glass, and the like.

7 is a block diagram illustrating configuration modules of a display device according to an embodiment of the present invention.

The display device 700 according to an exemplary embodiment of the present invention may include a memory 710, an audio output unit 720, a display unit 730, and a control unit 740. The configuration module shown in FIG. 7 and other configuration modules not shown in FIG. 7 may be referred to FIGS. 1-6.

The memory 710 may store various programs, applications, algorithms, and the like necessary for the operation of the display device 700. The memory 710 may be implemented with at least one of the SI & meta data database 211 of FIG. 2, the memory 360 of FIG. 3, and the storage unit 440 of FIG.

According to an embodiment of the present invention, the memory 710 may store at least one of the specific content, the caption related to the specific content, and the first information on the playback time of the caption. The subtitle associated with the specific content and the first information may be stored in a memory in the form of a subtitle file (e.g., * .smi file). The specific content may be an image stored in the memory 710. However, the present invention is not limited thereto.

Meanwhile, according to the embodiment, the specific contents may be a moving picture file in which audio data and video data are encoded.

The audio output unit 720 may output the audio data of the specific content. The audio output unit 740 may be implemented by at least one of the audio decoder 206 of FIG. 2, the audio output unit 352 of FIG. 3, and the audio output unit 485 of FIG.

The display unit 730 may output the video data of the specific content. In addition, the display unit 730 may display a subtitle associated with a specific content together with the video data using the first information stored in the memory 710. [ The display unit 730 may be implemented by at least one of the display unit 208 of FIG. 2, the display unit 351 of FIG. 3, and the display unit 480 of FIG.

The control unit 750 controls operations of the display device 700 and overall functions of the memory 710, the audio output unit 720, and the display unit 730. The control unit 750 includes the TCP / IP manager 201, the service delivery manager 203, the SI decoder 204, the DEMUX 205, the audio decoder 206, the video decoder 207, The service discovery manager 210 and the service manager 213, the control unit 380 of FIG. 3, and the control unit 470 of FIG. 4.

According to an embodiment of the present invention, the control unit 740 may control the outputting of subtitles in accordance with the comparison result of the information on the time when the audio is detected in the audio data of the specific content being output, Can be adjusted. This will be described in more detail with reference to FIG.

FIG. 8 is a flowchart illustrating an example of a method for adjusting the synchronization of subtitles in a display device according to an exemplary embodiment of the present invention. Referring to FIG.

According to an embodiment of the present invention, the memory 710 may store the caption related to the specific content and the first information on the playback time of the caption. Specifically, the subtitle associated with the specific content and the first information may be stored in a memory in the form of a subtitle file (for example, * .smi file). The specific content may be an image stored in the memory 710 or an image reproduced in streaming through a specific application. However, the present invention is not limited thereto.

The first information on the playback time of the subtitle may include information about a frame in which the subtitle is output. Accordingly, the control unit 740 can recognize the third information on the time when the subtitles are output using the first information. Specifically, the control unit 740 recognizes the frame in which the subtitles are output, using the information on the frame in which the subtitles are output, stored in the first information, The third information on the time when the subtitles are output can be recognized.

The specific content may be content received through an external server or content stored by the user in the memory 710 through an external device. Meanwhile, the specific content may be a moving picture file in which audio data and video data are encoded.

The subtitle may be a subtitle received through an external server or a subtitle stored in a memory 710 by a user via an external device. The language of the subtitles may be Korean or English. However, the present invention is not limited thereto.

The control unit 740 controls the display unit 730 to display the video data of the specific content and may control the audio output unit 720 to output the audio data of the specific content in operation S810.

The control unit 740 may control the display unit 730 to display the subtitles together with the video data using the first information on the playback time of the subtitles (S820). That is, the controller 740 can recognize the time at which the caption is displayed using the specific information included in the first information (for example, information about a frame from which the caption is output) The display unit 730 can be controlled to display the subtitles at the time when the display unit 730 is to be displayed. Therefore, the user can see the subtitle matching with the audio data being outputted.

According to an embodiment of the present invention, when displaying the caption, the controller 740 may assign a different visual effect to the caption according to a speaker associated with the caption. Specifically, the memory 710 may store a first visual effect corresponding to a first voiceprint and a second visual effect corresponding to a second voiceprint. The control unit 740 recognizes the voice in the output audio data, recognizes the voice of the recognized voice, and outputs the first visual effect to the outputting subtitle if the recognized voiceprint corresponds to the first voiceprint And to give the second visual effect to the caption outputting when the recognized caption corresponds to the second caption. A detailed description thereof will be described later in more detail with reference to FIG. 11 to FIG.

On the other hand, in step S820, the controller 740 may need to synchronize the output audio data with the subtitle if the subtitle does not match. A method of synchronizing the output audio data with the subtitle is described below.

The control unit 740 may recognize the second information on the time when the audio is detected from the audio data of the output content (S830).

In step S830, the second information may include information on a time when the detection of the voice is started, information on a time when the detection of the voice is terminated, information on a time interval during which the voice is not detected, And information on a time interval during which the mobile terminal 100 is operating.

Specifically, in step S830, the controller 740 detects the range of the audio data of the specific content being output, and when the detected range corresponds to a preset range, the controller 740 recognizes the range .

For example, the control unit 740 can detect the sound of the audio data of the specific content being output, and recognize that the sound is detected when the sound range corresponds to the range of 300 to 3,400 Hz. The control unit 740 may calculate the time at which the voice is detected to recognize the time interval during which the voice was detected. That is, the control unit 740 determines whether there is a time interval in which a voice exists when the specific content is reproduced (for example, a time interval from the time when the detection of the voice starts to the time when the detection of the voice is finished) Time interval (e.g., time interval in which voice and voice exist) can be recognized.

The control unit 740 can recognize the third information on the time when the subtitles are output using the first information (S840).

In step S840, the third information may include information about a time at which the output of the caption is started, information about a time at which the output of the caption is terminated, information about a time period during which the caption is not output, And information on a time interval during which the mobile terminal 100 is operating.

Specifically, in step S840, the control unit 740 outputs information on the time when the output of the output subtitle is started and information on the time when the output of the subtitle is finished, to the specific information (e.g., For example, information on a frame in which subtitles are output). The control unit 740 may obtain information on a time period during which the subtitles are output, using information on the time at which the output of the subtitles being output is started and the time at which the subtitles are output.

For example, the control unit 740 can use the subtitle file (e.g., * .smi file) to recognize which of the subtitles is output in which frame and which of the subtitles is missing. The control unit 740 can recognize the information about the time at which the output of the caption outputting is started and the information about the time at which the caption output ends, using the caption file.

The control unit 740 may adjust the output of the subtitle in accordance with the comparison result of the second information and the third information (S840). Here, the adjustment of the synchronization of the caption may mean that the time at which the caption is output is changed by entirely changing the frame in which the caption is output. However, the present invention is not limited to this, and the control unit 740 may partially change the frame in which the caption is output.

In step S840, the control unit 740 controls the synchronization of the subtitles as follows.

In the first method, when the time interval during which the voice is detected and the time period during which the caption is output are different using the second information and the third information, the control unit 740 recognizes that the caption is not synchronized can do. The control unit 740 may adjust the synchronization of the output subtitle so that the time interval during which the subtitle is output corresponds to the time period during which the audio is recognized.

For example, when the time at which the voice starts to be detected and the time at which the caption starts to be output are different (for example, when the voice is output but no caption is output), the controller 740 determines that the caption of the caption is not correct Can be recognized. The control unit 740 may adjust the output of the subtitles so that the time at which the subtitles are output corresponds to the time at which the audio is detected.

In another example, the control unit 740 recognizes the pattern of the time interval in which the voice is detected, and when the pattern is different from the pattern of the time interval in which the captions are output, the control unit 740 adjusts the synchronization of the captions It is possible.

As a second method, when the time period during which the voice is not detected and the time period during which the caption is not output are different using the second information and the third information, the control unit 740 determines that the caption of the caption is incompatible . The control unit 740 may adjust the sync of the output subtitle so that the time interval during which the subtitles are not output corresponds to the time period during which the audio is not recognized.

For example, when the time at which the detection of the voice is ended and the time at which the output of the subtitles are terminated are different (for example, when the subtitles are outputted although the voice is not outputted), the control unit 740 determines that the subtitles are not synchronized . The control unit 740 may adjust the output of the subtitles so that the output of the subtitles corresponds to the time at which the detection of the voice ends.

In another example, the control unit 740 may calculate an interval between time intervals during which the voice is sensed and compare the time interval with a time interval at which the subtitles are output, , It is possible to adjust the sync of the output subtitle.

Meanwhile, according to the embodiment, the control unit 740 can recognize a voice in the audio data being output, convert the recognized voice into text, and translate the converted text. The control unit 740 can adjust the synchronization of the output subtitle according to the translation result of the converted text and the result of the subtitle comparison. A detailed description thereof will be described later in detail with reference to FIG.

According to the present embodiment, there is an advantage that the user can control the sink of the subtitles automatically on the TV without having to manually adjust the synchronization of the subtitles on the TV, so that the user can concentrate on the content being output.

According to an embodiment of the present invention, the control unit 740 can adjust the synchronization of the subtitles when recognizing that the output subtitles are not in sync with the output audio data. This will be described in more detail with reference to FIG. 9 and FIG.

9 and 10 are diagrams for explaining an example of a method of adjusting the synchronization of the caption when the subtitle output from the display device according to the embodiment of the present invention recognizes that the audio data being output is out of sync with the audio data being output FIG.

According to an embodiment of the present invention, the memory 710 may store the caption related to the specific content and the first information on the reproduction time of the caption.

Referring to FIG. 9A, the control unit 740 may control the display unit 730 to display the video data 920 of the specific content. The control unit 740 may control the audio output unit 720 to output the audio data 910 of the specific content.

9B, the control unit 740 may control the display unit 730 to output the subtitle 930 together with the video data 910 using the first information.

The control unit 740 can recognize whether the audio data 911 output through the audio output unit 720 is in sync with the caption 930. The control unit 740 can recognize whether the audio data 911 output from the audio output unit 720 is in sync with the caption 930, When the data of the data 911 is not synchronized, it is possible to adjust the synchronization of the subtitle. A method of synchronizing the output audio data with the subtitle is described below.

The control unit 740 may recognize the second information on the time when the audio is detected from the audio data of the specific content being output. The second information may include at least one of information about a time at which the voice was detected, information about a time at which the voice was detected, information about a time period during which the voice was not detected, And / or information.

Specifically, the control unit 740 detects the range of the audio data of the specific content being output, and recognizes the range of the audio data when the detected range corresponds to the preset range.

For example, the control unit 740 can detect the sound of the audio data of the specific content being output, and recognize that the sound is detected when the sound range corresponds to the range of 300 to 3,400 Hz. The control unit 740 may calculate the time at which the voice is detected to recognize the time interval during which the voice was detected. That is, the control unit 740 determines whether there is a time interval in which a voice exists when the specific content is reproduced (for example, a time interval from the time when the detection of the voice starts to the time when the detection of the voice is finished) Time interval (e.g., time interval in which voice and voice exist) can be recognized.

The control unit 740 can recognize the third information on the time when the subtitles are output using the first information. The third information may include at least one of information on a time when the output of the caption is started, information on a time when the output of the caption is terminated, information on a time period during which the caption is not output, And / or information.

Specifically, the control unit 740 outputs information on the time at which the output of the output subtitle is started and information about the time at which the output of the subtitle ends, based on the specific information included in the third information (for example, Information on the frame to be outputted). The control unit 740 may obtain information on a time period during which the subtitles are output, using information on the time at which the output of the subtitles being output is started and the time at which the subtitles are output.

For example, referring to FIG. 10, the control unit 740 outputs a caption " father was a farmer " to the 74784th frame using a caption file (for example, * .smi file) stored in the memory 710 Information can be recognized that the subtitle disappears in the 77333th frame. Then, the controller 740 can recognize the information about the time when the output of the caption outputting is started and the information about the time when the output of the caption is finished using the recognized information.

The control unit 740 may adjust the output of the subtitle in accordance with the comparison result of the second information and the third information (S840). Here, the adjustment of the synchronization of the caption may mean that the time at which the caption is output is changed by entirely changing the frame in which the caption is output. A method by which the control unit 740 controls the synchronization of the caption is as follows.

The control unit 740 can recognize that the subtitles are not synchronized when the time interval during which the voice is detected and the time interval during which the subtitles are output are different using the second information and the third information.

For example, referring to FIGS. 9A and 9B, the control unit 740 detects that the "My father was farmer" 910 output through the audio output unit 720 is speech, It is possible to recognize whether or not the output of the subtitles exists at the time when the voice is detected. The control unit 740 can adjust the sink of the output subtitle so that the time when the subtitle starts to be output corresponds to the time when the audio starts to be detected when the voice recognizes that the subtitle 930 is not outputted have. That is, the control unit 740 controls the audio output unit 720 to output the subtitle 'My father was farmer' (930) when the 'My father was farmer' (910) Can be adjusted.

According to the embodiment, when the time interval during which the voice is not detected and the time period during which the caption is not output are different using the second information and the third information, the controller 740 determines whether the subtitles are synchronized It can be recognized that this is not the case. The control unit 740 may adjust the sync of the output subtitle so that the time interval during which the subtitles are not output corresponds to the time period during which the audio is not recognized.

For example, when the time at which the detection of the voice is ended and the time at which the output of the subtitles are terminated are different (for example, when the subtitles are outputted although the voice is not outputted), the control unit 740 determines that the subtitles are not synchronized . The control unit 740 may adjust the output of the subtitles so that the output of the subtitles corresponds to the time at which the detection of the voice ends.

In another example, the control unit 740 may calculate an interval between time intervals during which the voice is sensed and compare the time interval with a time interval at which the subtitles are output, , It is possible to adjust the sync of the output subtitle.

According to an embodiment of the present invention, the control unit 740 recognizes the voiceprint of the audio data being output through the audio output unit 720, and gives a visual effect to the caption being output according to the recognized voiceprint can do. This will be described in more detail with reference to FIGS. 11 to 13. FIG.

11 is a flowchart for explaining an example of a method of analyzing a voice of a recognized voice to give a visual effect to an outputting subtitle when a voice is recognized in audio data being output from a display device according to an embodiment of the present invention .

According to an embodiment of the present invention, the memory 710 may store the caption related to the specific content and the first information on the playback time of the caption. The control unit 740 may control the display unit 730 to display the video data of the specific content and may control the audio output unit 720 to output the audio data of the specific content.

The control unit 740 may control the display unit 730 to output the subtitles together with the video data using the first information. At this time, the control unit 740 can recognize the voice from the audio data being output (S910).

Specifically, in step S910, the controller 740 detects the range of the audio data of the specific content being output, and recognizes the range if the detected range corresponds to a preset range.

For example, the control unit 740 may detect a range of audio data of the specific content being output, and may recognize the range if the detected range corresponds to a range of 300 to 3,400 Hz.

The control unit 740 compares the stored voice stored in the memory 710 with the recognized voice to recognize whether the recognized voice corresponds to the stored voice in the memory 710 at operation S920. Here, the grammar is a graph obtained as a result of the time-series decomposition of the frequency distribution electronically, and plays a role of an identifier for distinguishing a person associated with the voice.

In the memory 710, a visual effect may be mapped and stored in a text associated with a character appearing in the specific content. For example, the first visual effect associated with the first person may be mapped and stored, and the second visual effect associated with the second person may have the second visual effect mapped and stored. Here, the first visual effect and the second visual effect may be different visual effects, different colors may be assigned, or different fonts may be assigned.

The control unit 740 compares the stored voice in the memory 710 with the recognized voice and if the recognized voice corresponds to the stored voice in the memory 710 (S920, Yes), the control unit 740 compares the time stored in the memory 710 The visual effect corresponding to the recognized gates of the effect can be given to the output subtitle (S930).

For example, when the recognized voice corresponds to a first sentence associated with the first person stored in the memory 710, the control unit 740 may assign a first visual effect (e.g., red) to the outputting subtitle .

In another example, when the recognized voice corresponds to a second sentence associated with the second person stored in the memory 710, the control unit 740 displays the second visual effect (e.g., yellow) on the outputting subtitle .

That is, when the recognized voice corresponds to the stored voice in the memory 710, the control unit 740 may assign the visual effect stored in the voice mail to the output subtitle. Accordingly, the user has an advantage that the user can recognize the caption for a certain person through the visual effect given to the caption being output.

If the recognized voice does not correspond to the stored voice in the memory 710 (S920, No), the controller 740 compares the stored voice stored in the memory 710 with the recognized voice, A visual effect other than the visual effect stored in the subtitle can be given to the caption during output (S940).

The control unit 740 can recognize the visual effect mapped to the stored voice in the memory 710 when the recognized voice does not correspond to the stored voice in the memory 710. [ Then, the control unit 740 can assign a visual effect different from the recognized visual effect to the output caption.

For example, if it is assumed that the first visual effect is mapped and stored in the memory 710 and the second visual effect is mapped and stored in the second text, the control unit 740 determines that the recognition (For example, black) different from the first visual effect and the second visual effect is given to the outputting subtitle in the case where the first visual effect and the second visual effect do not correspond to the first visual effect and the second visual statement can do.

According to this embodiment, when there are a plurality of speakers in the output content, it is possible to recognize which subtitles the user is outputting and which subtitles are related to which speakers, and give different visual effects to the subtitles associated with each speaker, Is easy to see and easy to understand.

12 is a view for explaining an example of a method of analyzing a voice of a recognized voice and giving a visual effect to a subtitle being output when a voice is recognized in audio data being outputted from a display device according to an embodiment of the present invention FIG.

According to an embodiment of the present invention, the memory 710 may store the caption related to the specific content and the first information on the playback time of the caption. The control unit 740 may control the display unit 730 to display the video data of the specific content and may control the audio output unit 720 to output the audio data of the specific content. The control unit 740 may control the display unit 730 to output the subtitles together with the video data using the first information. At this time, the control unit 740 can recognize the voice from the audio data being output.

The control unit 740 may compare the stored voice stored in the memory 710 with the recognized voice to recognize whether the recognized voice corresponds to the stored voice in the memory 710. [ Here, the grammar is a graph obtained as a result of the time-series decomposition of the frequency distribution electronically, and plays a role of an identifier for distinguishing a person associated with the voice.

12A, when the recognized voice corresponds to a first sentence associated with the first person A1 stored in the memory 710, the control unit 740 controls the first visual effect (For example, red) on the display unit 730 to output a subtitle.

12 (b), when the recognized voice corresponds to a second sentence associated with the second person A2 stored in the memory 710, the control unit 740 controls the second time It is possible to control the display unit 730 to output a subtitle given an effect (for example, yellow).

That is, when the recognized voice corresponds to the stored voice in the memory 710, the control unit 740 may assign the visual effect stored in the voice mail to the output subtitle. Accordingly, the user has an advantage that the user can recognize the caption for a certain person through the visual effect given to the caption being output.

On the other hand, if the recognized voice is not corresponding to the stored voice in the memory 710, the control unit 740 compares the stored voice stored in the memory 710 with the recognized voice, Can be given to the subtitles during output.

The control unit 740 can recognize the visual effect mapped to the stored voice in the memory 710 when the recognized voice does not correspond to the stored voice in the memory 710. [ Then, the control unit 740 can assign a visual effect different from the recognized visual effect to the output caption.

For example, if it is assumed that the first visual effect is mapped and stored in the memory 710 and the second visual effect is mapped and stored in the second text, the control unit 740 determines that the recognition (For example, black) that is different from the first visual effect and the second visual effect is given to the outputting subtitles when the first visual effect and the second visual effect do not correspond to the first text and the second text can do.

According to this embodiment, when there are a plurality of speakers in the output content, it is possible to recognize which subtitles the user is outputting and which subtitles are related to which speakers, and give different visual effects to the subtitles associated with each speaker, Is easy to see and easy to understand.

13 is a view for explaining an example of a method of mapping a visual effect assigned in accordance with a grammar in a display device according to an embodiment of the present invention to a memory.

Referring to FIG. 13, the memory 710 may map the visual effect to the gates of each character of the output content, and store the visual effect. Here, the grammar is a graph obtained as a result of the time-series decomposition of the frequency distribution electronically, and plays a role of an identifier for distinguishing a person associated with the voice.

For example, the memory 710 may map and store a first visual effect (e.g., red or obscure) in the first character's first text (first text). The memory 710 may map and store a second visual effect (e.g., yellow or clear Gothic material) in the gates (second gates) of the second character.

The control unit 740 may assign a specific visual effect to the output subtitle by using the visual effect mapped to the respective gates stored in the memory 710. [ The description thereof is similar to that described above with reference to FIG. 12, and a detailed description thereof will be omitted.

Meanwhile, according to an embodiment of the present invention, the controller 740 may convert the audio of the output audio data into text and adjust the synchronization of the caption using the translation result of the text. This will be described in detail with reference to FIG.

FIG. 14 is a flowchart for explaining an example of a method of converting a voice of audio data being outputted from a display device into text and adjusting the synchronization of the caption using the translation result of the converted text according to an embodiment of the present invention.

The control unit 740 can recognize the voice from the audio data being output (S1010).

Specifically, the control unit 740 detects the range of the audio data of the specific content being output, and recognizes the range if the detected range corresponds to a preset range.

For example, the control unit 740 detects the range of the audio data of the specific content being output, and when the detected range corresponds to the range of 300 to 3,400 Hz, . ≪ / RTI >

The control unit 740 may convert the recognized voice into text (S1020).

In one example, an algorithm for converting the recognized speech into text may be stored in memory 710 in advance. The controller 740 may convert the recognized speech into text using an algorithm stored in the memory 710. [

However, according to the embodiment, the control unit 740 may control the communication unit to transmit the recognized voice to the Speech to text (STT) server to convert the recognized voice into text. The control unit 740 may control the communication unit to receive the text through the STT server. The communication unit may be implemented by the wireless communication unit 310 of FIG. 3 or the network interface unit 430 of FIG. Here, the STT server may be an external server storing an algorithm for converting voice to text.

The control unit 740 can translate the converted text (S1030).

However, according to the embodiment, the control unit 740 may control the communication unit to receive the translated text through the STT server. That is, the text translation algorithm may be stored in the STT server, and the control unit 740 may control the communication unit to receive the translated text from the STT server.

The control unit 740 may adjust the output of the subtitle in accordance with the translation result of the text and the output result of the caption in operation S1040. Specifically, the control unit 740 compares the caption portion and the translation result of the text in the caption file stored in the memory 710, and recognizes the caption portion of the caption file corresponding to the translation result of the text. For example, the control unit 740 recognizes that the similar part of the subtitle file having the largest number of similar words is included in the translation result of the text, and outputs a voice related to the translation result of the text at the time when the recognized subtitle part is outputted It is possible to adjust the synchronization of the caption.

For example, when "I'm your farther." Is being output through the audio data, the control unit 740 may convert the audio data being output into text and translate the converted text. Then, the control unit 740 can entirely adjust the synchronization of the subtitles so that the translation result of the translated text, " I am your father, " can be output at the same time as the output audio data. The reason for controlling the synchronization of the subtitles is that, if the subtitles do not partially match, there is a high possibility that the subtitles do not completely match. However, the present invention is not limited to this, and the control unit 740 may partially control the synchronization of the subtitles.

According to the embodiment, the control unit 740 can recognize the time taken for voice recognition and the like, and further adjust the synchronization of the caption using the recognized time. Thus, the user has the advantage of being able to receive a subtitle that is well suited to the sink.

Meanwhile, according to the embodiment, the control unit 740 may stop the reproduction of the currently outputting content and proceed to steps S1010, S1020, S1030, and S1040. Accordingly, the control unit 740 may adjust the synchronization of the subtitles without considering the time taken for speech recognition.

According to the present embodiment, there is an advantage that the synchronization of subtitles can be adjusted more precisely.

According to at least one embodiment of the present invention, the user can adjust the synchronization of the subtitles automatically on the TV without manually adjusting the subtitles of the user on the TV so that the user can concentrate on the content being output . According to an embodiment of the present invention, when there are a plurality of speakers in the output content, it is possible to recognize which subtitle is related to which speaker and output a different visual effect to each subtitle associated with each speaker, Has the advantage that the content can be easily viewed and understood.

The display device and the control method of a display device disclosed in this specification can be applied to a configuration and a method of the embodiments described above in a limited manner, but the embodiments can be applied to all or a part of each embodiment Or may be selectively combined.

Meanwhile, the operation method of the digital device disclosed in this specification can be implemented as a code readable by a processor in a recording medium readable by a processor included in the digital device. The processor-readable recording medium includes all kinds of recording devices in which data that can be read by the processor is stored. Examples of the recording medium readable by the processor include ROM (Read Only Memory), RAM (Random Access Memory), CD-ROM, magnetic tape, floppy disk, optical data storage device, And may be implemented in the form of a carrier-wave. In addition, the processor-readable recording medium may be distributed over network-connected computer systems so that code readable by the processor in a distributed fashion can be stored and executed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Of the right. Further, such modifications are not to be understood individually from the technical idea of the present invention.

700: Display device
710:
720:
730: Memory
740: audio output section
750:

Claims (16)

A memory for storing a subtitle associated with a specific content and first information on a reproduction time of the subtitle;
A display unit for displaying video data of the specific content;
An audio output unit for outputting audio data of the specific content; And
And a control unit for controlling the display unit such that the subtitle is output together with the video data using the first information,
Wherein,
Recognizing second information on a time when audio is detected from the audio data of the specific content being output,
Recognizing third information on a time when the subtitles are output using the first information,
And adjusts the sink of the outputting subtitle according to a result of comparison between the second information and the third information. The method according to claim 1,
Wherein the second information comprises:
Information about a time at which the detection of the voice is started, information about a time at which the detection of the voice is finished, information about a time period during which the voice is not detected, and information about a time interval during which the voice is detected and,
Wherein the third information comprises:
Information on a time at which the output of the subtitles is started, information about a time at which the output of the subtitles is terminated, information about a time period during which the subtitles are not output, and information about time periods during which the subtitles are output / RTI > 3. The method of claim 2,
Wherein,
And controls the sink of the output subtitle so that a time interval during which the subtitle is output corresponds to a time interval during which the audio is recognized when the time interval during which the audio is detected and the time interval during which the subtitle is output are different. 3. The method of claim 2,
Wherein,
And adjusts the sink of the output subtitle if the time interval during which the speech is not recognized and the time period during which the subtitle is not output are different. 5. The method of claim 4,
Wherein,
And adjusts the sink of the output subtitle so that a time period during which the subtitle is not output corresponds to a time period during which the speech is not recognized. 3. The method of claim 2,
Wherein,
Detecting a range of audio data of the specific content being output,
And recognizes the detected sound range as a time interval in which the sound is sensed when the sensed sound range corresponds to a preset sound range. The method according to claim 1,
The memory,
Storing a first visual effect corresponding to a first voiceprint and a second visual effect corresponding to a second voiceprint,
Wherein,
Recognizes a voice in the output audio data,
Recognizes a voiceprint of the recognized voice,
When the recognized gateways correspond to the first sentence, giving the first visual effect to the outputting subtitle,
And gives the second visual effect to the outputting subtitle if the recognized gates correspond to the second gates. The method according to claim 1,
Wherein,
Recognizes a voice in the output audio data,
Converts the recognized speech into text,
Translating the converted text,
And adjusts the sink of the output subtitle according to a result of translation of the converted text and a result of comparison of the subtitle. Displaying video data of a specific content;
Outputting the audio data of the specific content;
Outputting a subtitle associated with the specific content stored in the memory together with the video data using first information on the reproduction time of the subtitle stored in the memory;
Recognizing second information on a time when audio is detected from audio data of the specific content being output;
Recognizing third information on a time when the subtitles are output using the first information; And
And adjusting a sync of the output subtitle according to a result of comparison between the second information and the third information. 10. The method of claim 9,
Wherein the second information comprises:
Information about a time at which the detection of the voice is started, information about a time at which the detection of the voice is finished, information about a time period during which the voice is not detected, and information about a time interval during which the voice is detected and,
Wherein the third information comprises:
Information on a time at which the output of the subtitles is started, information about a time at which the output of the subtitles is terminated, information about a time period during which the subtitles are not output, and information about time periods during which the subtitles are output Of the display device. 11. The method of claim 10,
Adjusting the synchronization of the output subtitle according to the comparison result of the second information and the third information,
Wherein the control unit controls the sink of the output subtitle so that a time interval during which the subtitle is output corresponds to a time interval during which the audio is recognized when the time interval during which the audio is detected and the time interval during which the subtitle is output are different Control method. 11. The method of claim 10,
Adjusting the synchronization of the output subtitle according to the comparison result of the second information and the third information,
Wherein the control unit controls the sink of the subtitle being output when the time period during which the speech is not recognized and the time period during which the subtitle is not output are different. 13. The method of claim 12,
Adjusting the synchronization of the output subtitle according to the comparison result of the second information and the third information,
And adjusts the sink of the output subtitle so that a time interval during which the subtitle is not output corresponds to a time interval during which the speech is not recognized. 11. The method of claim 10,
The step of recognizing the second information on the time when the audio is detected from the audio data of the specific content being output includes:
Sensing a range of audio data of the specific content being output; And
And recognizing the sensed range as a time interval in which a voice is sensed when the sensed range corresponds to a preset range. 10. The method of claim 9,
The step of outputting the caption associated with the specific content stored in the memory together with the video data using the first information on the reproduction time of the caption stored in the memory,
Recognizing speech in the output audio data;
Recognizing a voiceprint of the recognized voice;
Applying a first visual effect to the outputting subtitle if the recognized gates correspond to a first sentence;
And when the recognized gates correspond to the second sentence, giving a second visual effect to the outputting subtitle. 10. The method of claim 9,
Recognizing speech in the output audio data;
Converting the recognized speech into text;
Translating the translated text;
And adjusting the synchronization of the output subtitle according to the translation result of the converted text and the comparison result of the subtitle.

Images