KR20200084563A - Display device, and controlling method thereof - Google Patents

Abstract

The present invention relates to a display device and a control method thereof, which executes a gallery application, downloads a theme image from an external server, and edits a playback order of at least one of the theme image and the downloaded image stored in a memory, thereby allowing users to enjoy beautiful images of nature or famous paintings when the users are not watching TV.

Description

Display device and control method thereof

The present invention relates to a display device and a control method thereof, and more particularly, in a display device such as a smart TV, when a user does not watch a TV, a display device capable of viewing a beautiful natural environment or a masterpiece and a control method thereof It is related invention.

Recently, many users are using display devices such as smart TVs. In the prior art, when a user does not watch TV at home, a black screen is displayed on the screen. In addition, in the store, when it is not a regular broadcast time, only a black screen is displayed, so that a user who visits the store cannot see anything, and cannot check TV picture quality and TV sound performance, so that the user feels uncomfortable.

An embodiment of the present invention provides a display device and a control method for a user to view an image or video by executing a gallery application when the user does not watch TV, and downloading a theme image and video from an external server It aims to do.

Another embodiment of the present invention is to provide a display device and a control method for freely editing a theme image by receiving input from a user to select a pre-stored theme image and a downloaded theme image when the gallery application is executed The purpose.

According to another embodiment of the present invention, when a gallery application is executed, an input for selecting all selections, cancellations, and playbacks in a selection mode is received from a user, and a display device capable of adjusting a playback order and a grouping order and a control method thereof are provided. It aims to do.

According to an embodiment of the present invention, a display device includes: a tuner that receives a broadcast signal from a broadcasting station; A communication unit performing communication with at least one of an external server and an external remote control; A display for displaying content included in the received broadcast signal on a screen, the content being received from an external server or stored in an internal memory; A microphone that receives a user's voice from the user; And a controller that controls at least one of the tuner, the communication unit, the display, the microphone, and the speaker, wherein the controller displays at least one first image in a first area when the content is not displayed. , When an input for selecting any one of the at least one first image is received from a user, a plurality of second images corresponding to the selected first image are displayed in a second area, and an input for reproducing the first image is input. When received from the user, the plurality of second images are displayed on the screen at predetermined time intervals.

According to another embodiment of the present invention, a control method of a display device includes receiving a broadcast signal from a broadcasting station through a tuner; Displaying content included in the received broadcast signal on a screen; When not displaying content, displaying at least one first image in the first area; When an input for selecting any one of the at least one first image is received from a user, displaying a plurality of second images corresponding to the selected first image in a second area; And when an input for reproducing the first image is received from a user, displaying the plurality of second images on a screen at predetermined time intervals.

According to an embodiment of the present invention, when the user does not watch TV, the gallery application is executed, and the theme image and video are downloaded from an external server, so that the user can view the image or video, thereby improving user convenience. have.

According to another embodiment of the present invention, when the gallery application is executed, an input for selecting a pre-stored theme image and a downloaded theme image is received from a user, and the theme image can be freely edited, thereby improving user convenience. .

According to another embodiment of the present invention, when the gallery application is executed, an input for selecting all selections, cancellations, and playbacks in a selection mode is received from a user, so that the playback order and grouping order can be adjusted, thereby improving user convenience. Can.

1 is a view schematically illustrating a service system including a digital device according to an embodiment of the present invention.
2 is a block diagram illustrating a digital device according to an embodiment of the present invention.
3 is a block diagram illustrating a digital device according to another embodiment of the present invention.
4 is a block diagram illustrating a digital device according to another embodiment of the present invention.
5 is a block diagram illustrating a detailed configuration of the control unit of FIGS. 2 to 4 according to an embodiment of the present invention.
6 is a diagram illustrating input means connected to the digital devices of FIGS. 2 to 4 according to an embodiment of the present invention.
7 is a diagram illustrating a web OS architecture according to an embodiment of the present invention.
8 is a diagram illustrating an architecture of a web OS device according to an embodiment of the present invention.
9 is a diagram illustrating a graphic composition flow in a web OS device according to an embodiment of the present invention.
10 is a diagram illustrating a media server according to an embodiment of the present invention.
11 is a diagram illustrating a configuration block diagram of a media server according to an embodiment of the present invention.
12 is a diagram illustrating a relationship between a media server and a TV service according to an embodiment of the present invention.
13 is a diagram illustrating a control method of a remote control device that controls any one of video display devices according to embodiments of the present invention.
14 is an internal block diagram of a remote control device controlling any one of video display devices according to embodiments of the present invention.
15 is a configuration diagram of a display device according to an embodiment of the present invention.
16 is a diagram illustrating a control method of a display device according to an embodiment of the present invention.
17 is a diagram illustrating an initial screen when a gallery app of a display device is executed according to an embodiment of the present invention.
18 is a diagram illustrating downloading a downloadable gallery image according to an embodiment of the present invention.
19 is a diagram illustrating editing of a user's personal gallery image according to an embodiment of the present invention.
20 is a diagram illustrating a selection mode according to an embodiment of the present invention.
21 is a diagram illustrating a selection mode execution process according to an embodiment of the present invention.
22 is a diagram illustrating a deletion mode according to an embodiment of the present invention.
23 is a diagram illustrating a gallery app execution screen according to an embodiment of the present invention.
24 is a diagram illustrating an operation sequence of an afterimage prevention image for a gallery according to an embodiment of the present invention.
25 is a diagram illustrating a pop-up message for each specific situation according to an embodiment of the present invention.
26 is a diagram illustrating a pop-up message for each specific situation according to an embodiment of the present invention.
27 is a diagram illustrating a pop-up message for each specific situation according to an embodiment of the present invention.
28 is a diagram illustrating an audio guide process when a gallery app is executed according to an embodiment of the present invention.
29 is a diagram illustrating an audio guide process when editing a personal gallery according to an embodiment of the present invention.
30 is a diagram illustrating an audio guide process when a mode is selected according to an embodiment of the present invention.
31 is a diagram illustrating an audio guide process in a gallery app execution screen according to an embodiment of the present invention.
32 is a diagram illustrating a front recognition utterance process in a gallery app execution screen according to an embodiment of the present invention.
33 is a diagram illustrating a full recognition speech process when editing a personal gallery, according to an embodiment of the present invention.
34 is a diagram illustrating a full recognition speech process when editing a personal gallery according to an embodiment of the present invention.
35 is a diagram illustrating a front recognition utterance process when a specific screen is played according to an embodiment of the present invention.
36 is a diagram illustrating a screen structure when a gallery app is executed according to an embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

The suffixes "modules" and "parts" for components used in the following description are simply given in consideration of the ease of writing this specification, and the "modules" and "parts" may be used interchangeably.

Meanwhile, the video display device described in the present specification is, for example, an intelligent video display device in which a computer support function is added to a broadcast reception function, while an Internet function or the like is added while being faithful to the broadcast reception function, a handwritten input device, a touch screen Or, it may have a more convenient interface such as a space remote control. In addition, it is connected to the Internet and a computer with the support of wired or wireless Internet functions, and can perform functions such as email, web browsing, banking, or games. A standardized general-purpose OS can be used for these various functions.

Accordingly, in the video display device described in the present invention, various applications can be freely added or deleted, for example, on a general-purpose OS kernel, so that various user-friendly functions can be performed. The video display device may be, for example, a network TV, an HBBTV, a smart TV, and the like, and may be applied to a smartphone in some cases.

Furthermore, embodiments of the present invention will be described in detail with reference to the accompanying drawings and the contents described in the accompanying drawings, but the present invention is not limited or limited by the embodiments.

The terminology used in the present specification is a general terminology that is currently widely used while considering functions in the present invention, but this may vary depending on the intention or custom of a person skilled in the art or the appearance of a new technology. Also, in certain cases, some terms are arbitrarily selected by the applicant, and in this case, their meaning will be described in the description of the applicable invention. Therefore, the terms used in the present specification are intended to clarify that the terms should be interpreted based on the actual meaning of the terms and the contents of the present specification rather than simply the names of the terms.

Hereinafter, the term “digital device” described herein is, for example, transmitting, receiving, processing, and outputting data, content, service, application, and the like. All devices that perform at least one or more are included. The digital device can be paired or connected (hereinafter referred to as'pairing') with another digital device, an external server, or the like through a wire/wireless network, through which a predetermined data is transmitted. You can send/receive. At this time, if necessary, the data can be converted appropriately before the transmission/reception. The digital device includes, for example, a fixed device such as a network TV, a HBBTV (Hybrid Broadcast Broadband TV), a smart TV, an IPTV (Internet Protocol TV), or a personal computer (PC). In addition, a mobile device (mobile device or handheld device) such as a PDA (Personal Digital Assistant), a smart phone (Smart Phone), a tablet PC (Tablet PC), a notebook (Notebook), etc. may be all included. In this specification, for convenience of understanding of the present invention and for convenience of the applicant's explanation, FIG. 2, which will be described later, illustrates and describes a digital TV, and FIG. 3 illustrates a mobile device as an example of a digital device. In addition, the digital device described in this specification may be a configuration having only a panel, a configuration such as a set-top box (STB), a device, a system, and the like, and may be one set (SET) configuration. .

Meanwhile, the term “wired/wireless network” described herein refers to a communication network that supports various communication standards or protocols for pairing and/or data transmission and reception between digital devices or digital devices and external servers. Such a wired/wireless network includes all communication networks to be supported by the standard, either presently or in the future, and can support all one or more communication protocols therefor. Such wired/wireless networks include, for example, Universal Serial Bus (USB), Composite Video Banking Sync (CVBS), Component, S-Video (analog), Digital Visual Interface (DVI), High Definition Multimedia Interface (HDMI), Network for wired connection such as RGB, D-SUB and communication standards or protocols therefor, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), Zigbee (ZigBee), Digital Living Network Alliance (DLNA), Wireless LAN (WLAN) (Wi-Fi), Wireless broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), LTE/LTE -A (Long Term Evolution / LTE-Advanced), can be formed by a network for a wireless connection, such as Wi-Fi direct (direct) and communication standards or protocols therefor.

In addition, in the case of referring only to a digital device in the present specification, the meaning may mean a fixed device or a mobile device depending on the context, and may be used to include both unless otherwise specified.

Meanwhile, the digital device is an intelligent device that supports, for example, a broadcast reception function, a computer function or support, at least one external input, and the like, e-mail, web browsing through a wired/wireless network described above. , Banking, games, applications, and the like. In addition, the digital device may include an interface for supporting at least one input or control means (hereinafter referred to as an'input means') such as a handwritten input device, a touch-screen, and a space remote controller. have.

In addition, the digital device may use a standardized general-purpose operating system (OS), but in particular, the digital device described in this specification uses a web OS as an embodiment. Therefore, the digital device can process various services or applications such as adding, deleting, modifying, and updating various services or applications on the general-purpose OS kernel or Linux kernel. It is possible, through which a more user-friendly environment can be configured and provided.

On the other hand, the above-described digital device can receive and process an external input. At this time, the external input is connected to an external input device, that is, the above-mentioned digital device through a wired/wireless network to transmit/receive data. Input means to digital devices. For example, as the external input, a high-definition multimedia interface (HDMI), a game device such as a Playstation or an X-Box, a smart phone, a tablet PC, a pocket photo, etc. photo) and other digital devices such as a printing device, a smart TV, and a Blu-ray device.

In addition, the term “server” described herein refers to a digital device, that is, a digital device or system that supplies data to or receives data from the client, and is also called a processor. do. To the server, for example, a web server (Web server) or a portal server (portal server) that provides a web page (web page), web content or web service (web content or web service), providing advertising data (advertising data) Advertising server, content server providing content, SNS server providing social network service (SNS), service server provided by manufacturer, video on demand (VoD) B. It may include a multichannel video programming distributor (MVPD) for providing a streaming service, a service server providing a pay service, and the like.

In addition, in the following description, for convenience of description, even when only described as an application, based on the context and the like, the meaning may include not only an application but also a service. Further, the application may refer to a web application according to the web OS platform.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

1 is a view schematically illustrating a service system including a digital device according to an embodiment of the present invention.

As shown in FIG. 1, the service system includes a content provider 10, a service provider 20, a network provider 30, and a Home Network End (HNED). User) (Customer) 40. Here, the HNED 40 includes, for example, a client 100, that is, a digital device according to the present invention.

The content provider 10 produces and provides various content. As shown in FIG. 1, such a content provider 10 is a terrestrial broadcaster, a cable broadcaster (cable system operator (SO) or multiple SO), a satellite broadcaster, various Internet broadcasters, Personal content providers, and the like. Meanwhile, the content provider 10 may produce and provide various services or applications in addition to broadcast content.

The service provider 20 provides the HNED 40 with service packetizing the content produced by the content provider 10. For example, the service provider 20 packages at least one of contents produced by the first terrestrial broadcast, the second terrestrial broadcast, the cable MSO, the satellite broadcast, various Internet broadcasts, applications, etc., for service, and this is HNED (40).

The service provider 20 provides services to the client 100 in a uni-cast or multi-cast manner. Meanwhile, the service provider 20 may transmit data to a plurality of pre-registered clients 100 at a time, and for this, an Internet Group Management Protocol (IGMP) protocol may be used.

The above-described content provider 10 and service provider 20 may be the same entity. For example, by providing the content produced by the content provider 10 as a service package and providing it to the HNED 40, the function of the service provider 20 may also be performed or vice versa.

The network provider 30 provides a network for data exchange between the content provider 10 or/and the service provider 20 and the client 100.

As a consumer belonging to the HNED 40, the client 100 receives data by constructing a home network, for example, through a network provider 30, and receives data related to various services or applications such as VoD and streaming. You can also send/receive.

Meanwhile, the content provider 10 or/and the service provider 20 in the service system may use conditional access or content protection means to protect transmitted content. Therefore, the client 100 may use a processing means such as a cable card (or POD: Point of Deployment), DCAS (Downloadable CAS), etc. in response to the restriction reception or content protection.

In addition, the client 100 can also use the interactive service through the network. Therefore, the client 100 may rather perform the role or function of the content provider, and the service provider 20 may receive it and transmit it to another client again.

In FIG. 1, the content provider 10 or/and the service provider 20 may be a server that provides a service described later in this specification. In this case, the server may mean to own or include the network provider 30 as needed. Although not specifically mentioned below, the service or service data includes not only the service or application received from the above-described external service, but also an internal service or application, and the service or application is a service for a web OS-based client 100 To application data.

2 is a block diagram illustrating a digital device according to an embodiment of the present invention.

Hereinafter, the digital device described in this specification corresponds to the client 100 of FIG. 1 described above.

The digital device 200 includes a network interface unit 201, a TCP/IP manager 202, a service delivery manager 203, an SI decoder 204, Demultiplexer (demux or demultiplexer) 205, audio decoder (audio decoder) 206, video decoder (video decoder) 207, display unit (display A/V and OSD module) 208, service control manager (service control manager) 209, service discovery manager 210, SI&metadata DB 211, metadata manager 212, service manager 213, It comprises a UI manager 214 and the like.

The network interface unit 201 is an IP packet(s) (Internet Protocol(IP) packet(s)) or IP datagram(s) (IP datagram(s)) (hereinafter referred to as IP packet(s) through a network network to be accessed. )) is transmitted/received. For example, the network interface unit 201 may receive a service, application, content, etc. from the service provider 20 of FIG. 1 through a network network.

The TCP/IP manager 202 transmits packets between the source and destination of IP packets received by the digital device 200 and IP packets transmitted by the digital device 200. packet delivery). The TCP/IP manager 202 classifies the received packet(s) to correspond to an appropriate protocol, and a service delivery manager 205, a service discovery manager 210, a service control manager 209, and a metadata manager 212 ) And the like to output the classified packet(s).

The service delivery manager 203 is responsible for controlling received service data. For example, the service delivery manager 203 may use RTP/RTCP when controlling real-time streaming data. When the real-time streaming data is transmitted using RTP, the service delivery manager 203 parses the received data packet according to the RTP and transmits it to the demultiplexer 205 or control of the service manager 213 According to the SI & meta data database 211. Then, the service delivery manager 203 uses the RTCP to feed back the network reception information to a server providing a service.

The demultiplexer 205 demultiplexes the received packets into audio, video, and system information (SI) data, and transmits them to the audio/video decoders 206/207 and the SI decoder 204, respectively.

The SI decoder 204 includes demultiplexed SI data, that is, Program Specific Information (PSI), Program and System Information Protocol (PSIP), Digital Video Broadcasting-Service Information (DVB-SI), and Digital Television Terrestrial Multimedia (DTMB/CMMB). Decoding service information such as Broadcasting/Coding Mobile Multimedia Broadcasting). Also, the SI decoder 204 may store the decoded service information in the SI&metadata database 211. The stored service information may be read and used by the corresponding configuration, for example, by a user's request.

The audio/video decoder 206/207 decodes each demultiplexed audio data and video data. The audio data and video data thus decoded are provided to the user through the display unit 208.

The application manager includes, for example, a UI manager 214 and a service manager 213, and may perform a control function of the digital device 200. In other words, the application manager may manage the overall state of the digital device 200, provide a user interface (UI), and manage other managers.

The UI manager 214 provides a graphical user interface (GUI)/UI for a user using an OSD (On Screen Display) or the like, and receives a key input from a user to perform device operation according to the input. For example, when the UI manager 214 receives key input for channel selection from the user, the UI manager 214 transmits the key input signal to the service manager 213.

The service manager 213 controls a manager associated with a service, such as a service delivery manager 203, a service discovery manager 210, a service control manager 209, and a metadata manager 212.

In addition, the service manager 213 generates a channel map and controls channel selection and the like using the generated channel map according to the key input received from the UI manager 214. The service manager 213 receives service information from the SI decoder 204 and sets the audio/video PID (Packet Identifier) of the selected channel to the demultiplexer 205. The PID set in this way can be used in the demultiplexing process described above. Accordingly, the demultiplexer 205 filters audio data, video data, and SI data using the PID (PID or section filtering).

The service discovery manager 210 provides information necessary to select a service provider providing a service. When a signal regarding channel selection is received from the service manager 213, the service discovery manager 210 uses the information to find a service.

The service control manager 209 is responsible for selecting and controlling services. For example, the service control manager 209 uses IGMP or RTSP when a user selects a live broadcasting service such as an existing broadcasting method, and selects a service such as Video on Demand (VOD). The RTSP is used to select and control services. The RTSP protocol may provide a trick mode for real-time streaming. In addition, the service control manager 209 may initialize and manage a session through the IMS gateway 250 using an IP Multimedia Subsystem (IMS) and a Session Initiation Protocol (SIP). The above protocols are examples, and other protocols may be used according to implementation examples.

The metadata manager 212 manages metadata associated with a service and stores the metadata in the SI & metadata database 211.

The SI&metadata database 211 provides service information decoded by the SI decoder 204, metadata managed by the metadata manager 212, and information necessary to select a service provider provided by the service discovery manager 210. To save. In addition, the SI&metadata database 211 may store set-up data and the like for the system.

The SI&metadata database 211 may be implemented using non-volatile memory (NVRAM) or flash memory.

Meanwhile, the IMS gateway 250 is a gateway that collects functions necessary for accessing the IMS-based IPTV service.

3 is a block diagram illustrating a digital device according to another embodiment of the present invention.

If the above-described FIG. 2 described a fixed device as an example of a digital device, FIG. 3 illustrates a mobile device as another example of a digital device.

Referring to FIG. 3, the mobile device 300 includes a wireless communication unit 310, an audio/video (A/V) input unit 320, a user input unit 330, a sensing unit 340, an output unit 350, and a memory 360, an interface unit 370, a control unit 380, and a power supply unit 390.

Hereinafter, each component will be described in detail as follows.

The wireless communication unit 310 may include one or more modules that enable wireless communication between the mobile device 300 and the wireless communication system or between the mobile device and the network where the mobile device is located. For example, the wireless communication unit 310 may include a broadcast reception module 311, a mobile communication module 312, a wireless Internet module 313, a short-range communication module 314, and a location information module 315. .

The broadcast reception module 311 receives broadcast signals and/or broadcast related information from an external broadcast management server through a broadcast channel. Here, the broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may mean a server that generates and transmits broadcast signals and/or broadcast-related information or a server that receives previously generated broadcast signals and/or broadcast-related information and transmits them to a terminal. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, and may also include a TV broadcast signal or a radio broadcast signal combined with a data broadcast signal.

The broadcast-related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 312.

Broadcast-related information may exist in various forms, for example, an electronic program guide (EPG) or an electronic service guide (ESG).

The broadcast reception module 311 includes, for example, ATSC, Digital Video Broadcasting-Terrestrial (DVB-T), DVB-S (Satellite), MediaFLO (Media Forward Link Only), DVB-H (Handheld), ISDB-T ( Digital broadcasting signals can be received using a digital broadcasting system such as Integrated Services Digital Broadcast-Terrestrial). Of course, the broadcast receiving module 311 may be configured to be suitable for other broadcasting systems as well as the digital broadcasting system described above.

The broadcast signal and/or broadcast-related information received through the broadcast receiving module 311 may be stored in the memory 360.

The mobile communication module 312 transmits and receives wireless signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice signal, a video call signal, or a text/multimedia message.

The wireless Internet module 313 may include a module for wireless Internet access, or may be built in or external to the mobile device 300. Wireless Internet technology (Wireless LAN) (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) may be used.

The short-range communication module 314 refers to a module for short-range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, RS-232, RS-485, etc. may be used as short range communication technology. Can.

The location information module 315 is a module for obtaining location information of the mobile device 300, and may use, for example, a Global Position System (GPS) module.

The A/V input unit 320 is for inputting an audio or/and video signal, and may include a camera 321 and a microphone 322. The camera 321 processes image frames such as still images or moving pictures obtained by an image sensor in a video call mode or a shooting mode. The processed image frame may be displayed on the display unit 351.

The image frames processed by the camera 321 may be stored in the memory 360 or transmitted to the outside through the wireless communication unit 310. Two or more cameras 321 may be provided according to the use environment.

The microphone 322 receives an external sound signal by a microphone in a call mode or a recording mode, a voice recognition mode, etc., and processes it as electrical voice data. The processed voice data may be converted and output in a form that can be transmitted to the mobile communication base station through the mobile communication module 312 when in a call mode. The microphone 322 may be implemented with various noise removal algorithms for removing noise generated in the process of receiving an external sound signal.

The user input unit 330 generates input data for the user to control the operation of the terminal. The user input unit 330 may be configured with a key pad, a dome switch, a touch pad (static pressure/power outage), a jog wheel, a jog switch, or the like.

The sensing unit 340 displays the current state of the mobile device 300, such as the open/closed state of the mobile device 300, the location of the mobile device 300, the presence or absence of user contact, the orientation of the mobile device, and acceleration/deceleration of the mobile device. Sensing to generate a sensing signal for controlling the operation of the mobile device 300. For example, when the mobile device 300 is moved or tilted, the position or tilt of the mobile device may be sensed. In addition, it is also possible to sense whether the power supply unit 390 is supplied with power, whether the interface unit 370 is coupled with an external device, or the like. Meanwhile, the sensing unit 240 may include a proximity sensor 341 including Near Field Communication (NFC).

The output unit 350 is for generating output related to vision, hearing, or tactile sense, and may include a display unit 351, an audio output module 352, an alarm unit 353, and a haptic module 354. have.

The display unit 351 displays (outputs) information processed by the mobile device 300. For example, when the mobile device is in a call mode, it displays a UI or GUI related to the call. When the mobile device 300 is in a video call mode or a photographing mode, a photographed and/or received image or UI, GUI is displayed.

The display unit 351 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display ( flexible display), and a 3D display.

Some of these displays may be of a transparent type or a light transmissive type so that the outside can be seen through them. This may be referred to as a transparent display, and a typical example of the transparent display is TOLED (Transparant OLED). The rear structure of the display unit 351 may also be configured as a light transmissive structure. With this structure, the user can see an object located behind the terminal body through an area occupied by the display unit 351 of the terminal body.

Two or more display units 351 may be present according to an implementation form of the mobile device 300. For example, a plurality of display units may be spaced apart from one surface or integrally disposed on the mobile device 300, or may be respectively disposed on different surfaces.

When the display unit 351 and a sensor detecting a touch operation (hereinafter referred to as a “touch sensor”) form a mutual layer structure (hereinafter, referred to as a “touch screen”), the display unit 351 inputs in addition to an output device. It can also be used as a device. The touch sensor may have, for example, a form of a touch film, a touch sheet, or a touch pad.

The touch sensor may be configured to convert changes in pressure applied to a specific portion of the display unit 351 or capacitance generated in a specific portion of the display unit 351 into an electrical input signal. The touch sensor may be configured to detect not only the touched position and area, but also pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal(s) is sent to the touch controller. The touch controller processes the signal(s) and then transmits corresponding data to the control unit 380. Accordingly, the control unit 380 can know which area of the display unit 351 has been touched, and the like.

A proximity sensor 341 may be disposed in an inner area of the mobile device surrounded by the touch screen or near the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object present in the vicinity without mechanical contact using electromagnetic force or infrared rays. Proximity sensors have a longer lifespan and higher utilization than contact sensors.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, and an infrared proximity sensor. When the touch screen is capacitive, it is configured to detect the proximity of the pointer due to a change in electric field according to the proximity of the pointer. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of description, the act of causing the pointer to be recognized as being located on the touch screen without being touched by the pointer on the touch screen is referred to as a “proximity touch”, and the touch The act of actually touching the pointer on the screen is referred to as "contact touch". The location on the touch screen that is a proximity touch with a pointer refers to a location where the pointer is perpendicular to the touch screen when the pointer is touched close.

The proximity sensor detects a proximity touch and a proximity touch pattern (eg, proximity touch distance, proximity touch direction, proximity touch speed, proximity touch time, proximity touch position, proximity touch movement state, etc.). Information corresponding to the sensed proximity touch operation and proximity touch pattern may be output on the touch screen.

The audio output module 352 may output audio data received from the wireless communication unit 310 or stored in the memory 360 in a call signal reception, call mode or recording mode, voice recognition mode, broadcast reception mode, or the like. The sound output module 352 may also output sound signals related to functions (for example, call signal reception sound, message reception sound, etc.) performed in the mobile device 300. The sound output module 352 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 353 outputs a signal for notifying the occurrence of the event of the mobile device 300. Examples of events generated in the mobile device include call signal reception, message reception, key signal input, and touch input. The alarm unit 353 may output a signal for notifying the occurrence of an event in a form other than a video signal or an audio signal, for example, vibration. The video signal or the audio signal may also be output through the display unit 351 or the audio output module 352, so that they 351 and 352 may be classified as part of the alarm unit 353.

The haptic module 354 generates various tactile effects that the user can feel. A typical example of the tactile effect generated by the haptic module 354 is vibration. The intensity and pattern of vibration generated by the haptic module 354 can be controlled. For example, different vibrations may be synthesized and output or sequentially output. The haptic module 354, in addition to vibration, is a pin array that vertically moves with respect to the contact skin surface, the ejection force or inhalation force of the air through the injection port or the intake port, the stimulation of the skin surface, the contact of the electrode (eletrode), the stimulus, etc. It can generate various tactile effects, such as the effect by, and the effect by reproducing the cold/warm feeling using the heat absorbing or heat generating element. The haptic module 354 may not only transmit the tactile effect through direct contact, but may also implement the user to feel the tactile effect through muscle sensations such as fingers or arms. Two or more haptic modules 354 may be provided according to a configuration aspect of the mobile device 300.

The memory 360 may store a program for the operation of the control unit 380, and may temporarily store input/output data (eg, a phone book, a message, a still image, a video, etc.). The memory 360 may store data related to various patterns of vibration and sound that are output upon touch input on the touch screen.

The memory 360 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory, etc.), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM), magnetic memory, It may include at least one type of storage medium among magnetic disks and optical disks. The mobile device 300 may operate in connection with a web storage performing a storage function of the memory 360 on the Internet.

The interface unit 370 serves as a passage with all external devices connected to the mobile device 300. The interface unit 370 receives data from an external device, receives power, and transmits data to each component inside the mobile device 300, or allows data inside the mobile device 300 to be transmitted to the external device. For example, wired/wireless headset port, external charger port, wired/wireless data port, memory card port, port for connecting devices equipped with an identification module, audio input/output (I/O) port, A video I/O port, an earphone port, and the like may be included in the interface unit 370.

The identification module is a chip that stores various information for authenticating the usage rights of the mobile device 300, a user identification module (UIM), a subscriber identification module (SIM), and a universal user authentication module ( Universal Subscriber Identity Module (USIM). The device provided with the identification module (hereinafter referred to as'identification device') may be manufactured in a smart card format. Therefore, the identification device may be connected to the terminal 200 through a port.

When the mobile device 300 is connected to an external cradle, the interface unit 370 becomes a passage through which power from the cradle is supplied to the mobile device 300, or various input by the user from the cradle. A command signal may be a passage to the mobile device. Various command signals or power input from the cradle may be operated as signals for recognizing that the mobile device is correctly mounted on the cradle.

The control unit 380 typically controls the overall operation of the mobile device 300. The control unit 380 performs, for example, related control and processing for voice calls, data communication, video calls, and the like. The control unit 380 may include a multimedia module 381 for multimedia playback. The multimedia module 381 may be implemented in the control unit 380 or may be implemented separately from the control unit 380. The control unit 380 may perform a pattern recognition process capable of recognizing handwriting input or picture drawing input performed on a touch-screen as text and images, respectively.

The power supply unit 390 receives external power and internal power under the control of the control unit 380 and supplies power required for the operation of each component.

The various embodiments described herein can be implemented in a computer- or similar device-readable recording medium using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), It may be implemented using at least one of a processor, a controller, micro-controllers, microprocessors, and electrical units for performing other functions. Examples may be implemented by the control unit 380 itself.

According to the software implementation, embodiments such as procedures and functions described herein may be implemented as separate software modules. Each of the software modules can perform one or more functions and operations described herein. Software code can be implemented in a software application written in an appropriate programming language. Here, the software code is stored in the memory 360 and can be executed by the control unit 380.

4 is a block diagram illustrating a digital device according to another embodiment of the present invention.

Other examples of the digital device 400 include a broadcast receiving unit 405, an external device interface unit 435, a storage unit 440, a user input interface unit 450, a control unit 470, a display unit 480, audio It may include an output unit 485, a power supply unit 490, and a photographing unit (not shown). Here, the broadcast reception unit 405 may include at least one tuner 410, a demodulation unit 420, and a network interface unit 430. However, in some cases, the broadcast reception unit 405 includes a tuner 410 and a demodulation unit 420, but may not include the network interface unit 430, and vice versa. In addition, although the broadcast receiving unit 405 is not shown, a signal demodulated by the demodulator 420 via the tuner 410 and a signal received through the network interface unit 430 is provided with a multiplexer. Can also be multiplexed. In addition, although the broadcast receiving unit 425 is not shown, a demultiplexer is provided to demultiplex the multiplexed signal or demultiplex the demodulated signal or the signal that has passed through the network interface unit 430. Can.

The tuner 410 receives an RF broadcast signal by tuning a channel selected by a user or all previously stored channels among radio frequency (RF) broadcast signals received through an antenna. In addition, the tuner 410 converts the received RF broadcast signal into an intermediate frequency (IF) signal or a baseband signal.

For example, if the received RF broadcast signal is a digital broadcast signal, it is converted into a digital IF signal (DIF), and if it is an analog broadcast signal, it is converted into an analog baseband video or audio signal (CVBS/SIF). That is, the tuner 410 can process both digital broadcast signals or analog broadcast signals. The analog baseband video or audio signal (CVBS/SIF) output from the tuner 410 may be directly input to the controller 470.

In addition, the tuner 410 may receive a single carrier (single carrier) or multiple carriers (multiple carrier) RF broadcast signal. Meanwhile, the tuner 410 sequentially tunes and receives RF broadcast signals of all broadcast channels stored through a channel memory function among RF broadcast signals received through an antenna, and receives the intermediate frequency signals or base band signals (DIF: Digital Intermediate). Frequency or baseband signal).

The demodulator 420 may receive and demodulate the digital IF signal DIF converted by the tuner 410, and perform channel decoding and the like. To this end, the demodulator 420 includes a trellis decoder, a de-interleaver, a reed-solomon decoder, or a convolution decoder, de-interleaver, and read. -It can be equipped with a solo decoder.

The demodulator 420 may output a stream signal TS after demodulation and channel decoding. In this case, the stream signal may be a signal in which a video signal, an audio signal or a data signal is multiplexed. For example, the stream signal may be an MPEG-2 Transport Stream (TS) in which an MPEG-2 standard video signal, a Dolby AC-3 standard audio signal, and the like are multiplexed.

The stream signal output from the demodulator 420 may be input to the controller 470. The control unit 470 may control demultiplexing, video/audio signal processing, and the like, and control an image output through the display unit 480 and an audio output unit through the audio output unit 485.

The external device interface unit 435 provides an interfacing environment between the digital device 300 and various external devices. To this end, the external device interface unit 335 may include an A/V input/output unit (not shown) or a wireless communication unit (not shown).

The external device interface unit 435 includes a Digital Versatile Disk (DVD), Blu-ray, game device, camera, camcorder, computer (laptop), tablet PC, smart phone, and Bluetooth device (Bluetooth). device), an external device such as a cloud, and the like can be connected by wire/wireless. The external device interface unit 435 transmits a signal including data, such as image, video, and voice input through the connected external device to the controller 470 of the digital device. The control unit 470 may control the processed image, video, audio, etc. to be output to a connected external device. To this end, the external device interface unit 435 may further include an A/V input/output unit (not shown) or a wireless communication unit (not shown).

The A/V input/output unit allows a USB terminal, a CVBS (Composite Video Banking Sync) terminal, a component terminal, and an S-video terminal (analog) to input video and audio signals from an external device to the digital device 400. It may include a digital visual interface (DVI) terminal, a high definition multimedia interface (HDMI) terminal, an RGB terminal, and a D-SUB terminal.

The wireless communication unit may perform short-range wireless communication with other digital devices. The digital device 400 is, for example, Bluetooth (Bluetooth), Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Digital Living Network Alliance (DLNA) It may be networked with other digital devices according to a communication protocol such as.

In addition, the external device interface unit 435 may be connected to the set-top box STB through at least one of the various terminals described above, and perform input/output operations with the set-top box STB.

Meanwhile, the external device interface unit 435 may receive an application or an application list in an adjacent external device and transmit it to the control unit 470 or the storage unit 440.

The network interface unit 430 provides an interface for connecting the digital device 400 to a wired/wireless network including an Internet network. The network interface unit 430 may include, for example, an Ethernet terminal or the like for connection with a wired network, and for example, a wireless LAN (WLAN) (Wi-) for connection with a wireless network. Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), and HSDPA (High Speed Downlink Packet Access) communication standards.

The network interface unit 430 may transmit or receive data with other users or other digital devices through a connected network or another network linked to the connected network. In particular, some content data stored in the digital device 400 may be transmitted to another user registered in advance in the digital device 400 or to a selected user or selected digital device among other digital devices.

Meanwhile, the network interface unit 430 may access a predetermined web page through a connected network or another network linked to the connected network. That is, it is possible to connect to a predetermined web page through a network and transmit or receive data with the corresponding server. In addition, content or data provided by a content provider or a network operator may be received. That is, it is possible to receive content such as a movie, advertisement, game, VOD, broadcast signal, and related information provided by a content provider or a network provider through a network. Further, it is possible to receive update information and update files of firmware provided by a network operator. It can also send data to the Internet or a content provider or network operator.

In addition, the network interface unit 430 may select and receive a desired application from among applications opened through the network.

The storage unit 440 may store a program for processing and controlling each signal in the control unit 470, or may store a signal-processed image, audio, or data signal.

Also, the storage unit 440 may perform a function for temporarily storing an image, audio, or data signal input from the external device interface unit 435 or the network interface unit 430. The storage unit 440 may store information regarding a predetermined broadcast channel through a channel storage function.

The storage unit 440 may store an application or application list input from the external device interface unit 435 or the network interface unit 330.

Also, the storage unit 440 may store various platforms described later.

The storage unit 440 may be, for example, a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (for example, SD or XD) Memory, etc.), RAM (RAM), and ROM (EEPROM, etc.). The digital device 400 may play a content file (video file, still image file, music file, document file, application file, etc.) stored in the storage unit 440 and provide it to the user.

4 illustrates an embodiment in which the storage unit 440 is provided separately from the control unit 470, but the present invention is not limited thereto. In other words, the storage unit 440 may be included in the control unit 470.

The user input interface 450 transmits a signal input by the user to the controller 470 or a signal from the controller 470 to the user.

For example, the user input interface unit 450 controls power on/off, channel selection, screen setting, etc. from the remote control device 500 according to various communication methods such as an RF communication method and an infrared (IR) communication method. The signal may be received and processed or the control signal of the control unit 470 may be transmitted to the remote control device 500.

In addition, the user input interface unit 450 may transmit a control signal input from a local key (not shown), such as a power key, a channel key, a volume key, and a set value, to the control unit 470.

The user input interface unit 450 transmits a control signal input from a sensing unit (not shown) that senses a user's gesture to the control unit 470 or senses a signal from the control unit 470 (Not shown). Here, the sensing unit (not shown) may include a touch sensor, a voice sensor, a position sensor, and a motion sensor.

The control unit 470 de-multiplexes the stream input through the tuner 410, the demodulator 420, or the external device interface unit 435 or processes the demultiplexed signals to generate a signal for video or audio output. And output.

The image signal processed by the control unit 470 may be input to the display unit 480 and displayed as an image corresponding to the corresponding image signal. Also, the image signal processed by the controller 470 may be input to the external output device through the external device interface unit 435.

The audio signal processed by the control unit 470 may be audio output to the audio output unit 485. Also, the voice signal processed by the control unit 470 may be input to the external output device through the external device interface unit 435.

Although not illustrated in FIG. 4, the control unit 470 may include a demultiplexing unit, an image processing unit, and the like.

The controller 470 may control the overall operation of the digital device 400. For example, the control unit 470 may control the tuner 410 to tune an RF broadcast corresponding to a channel selected by a user or a pre-stored channel.

The control unit 470 may control the digital device 400 by a user command input through the user input interface unit 450 or an internal program. In particular, it is possible to connect to a network so that a user can download a desired application or application list into the digital device 400.

For example, the control unit 470 controls the tuner 410 to input a signal of a selected channel according to a predetermined channel selection command received through the user input interface unit 450. And it processes the video, audio or data signal of the selected channel. The control unit 470 allows the user to select the channel information selected by the user and output the processed image or audio signal through the display unit 480 or the audio output unit 485.

As another example, the control unit 470 may be input from an external device input through the external device interface unit 435, for example, a camera or camcorder, according to an external device image playback command received through the user input interface unit 450. The video signal or the audio signal can be output through the display unit 480 or the audio output unit 485.

Meanwhile, the control unit 470 may control the display unit 480 to display an image. For example, a broadcast image input through the tuner 410, an external input image input through the external device interface unit 435, an image input through the network interface unit, or an image stored in the storage unit 440. , It can be controlled to be displayed on the display unit 480. At this time, the image displayed on the display unit 480 may be a still image or a video, and may be a 2D video or a 3D video.

Also, the control unit 470 may control to play content. The content at this time may be content stored in the digital device 400, or received broadcast content, or external input content input from the outside. The content may be at least one of a broadcast image, an external input image, an audio file, a still image, a connected web screen, and a document file.

On the other hand, when entering the application view item, the controller 470 may control to display a list of applications or applications that can be downloaded from the digital device 300 or from an external network.

The control unit 470 may control to install and operate an application downloaded from an external network along with various user interfaces. In addition, by selecting the user, it is possible to control an image related to an application to be executed to be displayed on the display unit 480.

On the other hand, although not shown in the drawing, it is also possible that a channel browsing processing unit for generating a thumbnail image corresponding to a channel signal or an external input signal is further provided.

The channel browsing processing unit receives a stream signal TS output from the demodulator 320 or a stream signal output from the external device interface unit 335, extracts an image from the input stream signal, and generates a thumbnail image. Can. The generated thumbnail image may be input as it is or as encoded. In addition, the generated thumbnail image may be encoded in a stream form and input to the control unit 470. The controller 470 may display a list of thumbnails having a plurality of thumbnail images on the display unit 480 using the input thumbnail images. Meanwhile, thumbnail images in the thumbnail list may be updated sequentially or simultaneously. Accordingly, the user can easily grasp the contents of a plurality of broadcast channels.

The display unit 480 converts the image signal, data signal, OSD signal, or image signal received from the external device interface unit 435 processed by the control unit 470 into R, G, and B signals, respectively. Generate a drive signal.

The display unit 480 may be a PDP, LCD, OLED, flexible display, 3D display, or the like.

Meanwhile, the display unit 480 may be configured as a touch screen and used as an input device in addition to the output device.

The audio output unit 485 receives a signal processed by the control unit 470, for example, a stereo signal, a 3.1 channel signal, or a 5.1 channel signal, and outputs the audio. The audio output unit 485 may be implemented as various types of speakers.

Meanwhile, in order to sense a user's gesture, as described above, a sensing unit (not shown) having at least one of a touch sensor, a voice sensor, a position sensor, and a motion sensor may be further provided in the digital device 400. . The signal detected by the sensing unit (not shown) may be transmitted to the control unit 3470 through the user input interface unit 450.

Meanwhile, a photographing unit (not shown) for photographing a user may be further provided. Image information photographed by the photographing unit (not shown) may be input to the control unit 470.

The controller 470 may detect a user's gesture by individually or in combination with an image captured by the photographing unit (not shown) or a signal detected from the sensing unit (not shown).

The power supply unit 490 supplies corresponding power throughout the digital device 400.

In particular, the controller 470 can be implemented in the form of a System on Chip (SoC), a display unit 480 for displaying images, and an audio output unit 485 for audio output. Can.

To this end, the power supply unit 490 may include a converter (not shown) that converts AC power into DC power. Meanwhile, for example, when the display unit 480 is implemented as a liquid crystal panel having a plurality of backlight lamps, an inverter capable of PWM (Pulse Width Modulation) operation for variable luminance or dimming driving (Inverter) (not shown) may be further provided.

The remote control device 500 transmits a user input to the user input interface unit 450. To this end, the remote control device 500 may use Bluetooth, Radio Frequency (RF) communication, Infrared (IR) communication, Ultra Wideband (UWB), ZigBee, or the like.

In addition, the remote control device 500 may receive an image, audio or data signal output from the user input interface unit 450, display it on the remote control device 500, or output voice or vibration.

The above-described digital device 400 may be a digital broadcast receiver capable of processing a fixed or mobile ATSC type or DVB type digital broadcast signal.

In addition, the digital device according to the present invention may further omit some components or further include components that are not shown, as required, among the illustrated components. On the other hand, as described above, the digital device does not have a tuner and a demodulator, and may receive and play content through a network interface unit or an external device interface unit.

5 is a block diagram illustrating a detailed configuration of the control unit of FIGS. 2 to 4 according to an embodiment of the present invention.

Examples of the control unit, the demultiplexing unit 510, the image processing unit 5520, the OSD generation unit 540, a mixer (mixer) 550, a frame rate converter (FRC: Frame Rate Converter) 555, and A formatter 560 may be included. In addition, although not illustrated, the control unit may further include a voice processing unit and a data processing unit.

The demultiplexing unit 510 demultiplexes the input stream. For example, the demultiplexer 510 may demultiplex the input MPEG-2 TS video, audio, and data signals. Here, the stream signal input to the demultiplexer 510 may be a stream signal output from a tuner or demodulator or an external device interface.

The image processing unit 420 performs image processing of the demultiplexed image signal. To this end, the image processing unit 420 may include an image decoder 425 and a scaler 435.

The video decoder 425 decodes the demultiplexed video signal, and the scaler 435 scales the resolution of the decoded video signal to be output from the display unit.

The video decoder 525 may support various standards. For example, the video decoder 525 performs the function of the MPEG-2 decoder when the video signal is encoded in the MPEG-2 standard, and the video signal is encoded in the Digital Multimedia Broadcasting (DMB) method or the H.264 standard. In this case, the function of the H.264 decoder can be performed.

Meanwhile, the video signal decoded by the video processing unit 520 is input to the mixer 450.

The OSD generation unit 540 generates OSD data according to a user input or by itself. For example, the OSD generating unit 440 generates data for displaying various data in the form of a graphic or text on the screen of the display unit 380 based on the control signal of the user input interface unit. The generated OSD data includes various data such as a user interface screen of a digital device, various menu screens, widgets, icons, and viewing rate information. The OSD generator 540 may generate data for displaying subtitles of broadcast images or broadcast information based on EPG.

The mixer 550 mixes the OSD data generated by the OSD generator 540 and the image signal processed by the image processor to provide the formatter 560. Because the decoded video signal and the OSD data are mixed, the OSD is displayed overlaid on a broadcast video or an external input video.

The frame rate converter (FRC) 555 converts a frame rate of an input video. For example, the frame rate converter 555 may convert a frame rate of an input 60 Hz image to have a frame rate of, for example, 120 Hz or 240 Hz according to the output frequency of the display unit. As described above, various methods may exist in the method for converting the frame rate. For example, when the frame rate converter 555 converts the frame rate from 60 Hz to 120 Hz, the same first frame is inserted between the first frame and the second frame, or the first frame and the second frame are predicted from the first frame. It can be converted by inserting 3 frames. As another example, when the frame rate converter 555 converts the frame rate from 60 Hz to 240 Hz, three or more identical frames or predicted frames may be inserted and converted between existing frames. Meanwhile, if a separate frame conversion is not performed, the frame rate conversion unit 555 may be bypassed.

The formatter 560 changes the output of the input frame rate conversion unit 555 to match the output format of the display unit. For example, the formatter 560 may output R, G, and B data signals, and the R, G, and B data signals may be output as a low voltage differential signal (LVDS) or a mini-LVDS Can be. In addition, when the output of the input frame rate conversion unit 555 is a 3D video signal, the formatter 560 may configure and output a 3D format according to the output format of the display unit, thereby supporting 3D service through the display unit.

Meanwhile, the voice processing unit (not shown) in the control unit may perform voice processing of the demultiplexed voice signal. The voice processing unit (not shown) may support various audio formats. For example, even when an audio signal is encoded in formats such as MPEG-2, MPEG-4, AAC, HE-AAC, AC-3, BSAC, a decoder corresponding thereto may be provided and processed.

In addition, the voice processing unit (not shown) in the control unit may process a base, treble, volume control, and the like.

The data processing unit (not shown) in the control unit may perform data processing of the demultiplexed data signal. For example, the data processing unit can decode the demultiplexed data signal even when it is encoded. Here, the encoded data signal may be EPG information including broadcast information such as start time and end time of a broadcast program broadcast on each channel.

Meanwhile, the above-described digital device is an example according to the present invention, and each component may be integrated, added, or omitted depending on the specification of the actual digital device. That is, if necessary, two or more components may be combined into one component, or one component may be subdivided into two or more components. In addition, the function performed in each block is for explaining an embodiment of the present invention, the specific operation or device does not limit the scope of the present invention.

Meanwhile, the digital device may be an image signal processing device that performs signal processing of an image stored in the device or an input image. As another example of the video signal processing device, the display unit 480 and the audio output unit 485 shown in FIG. 4 are excluded from the set-top box (STB), the above-described DVD player, Blu-ray player, game device, computer And the like can be further exemplified.

6 is a diagram illustrating input means connected to the digital devices of FIGS. 2 to 4 according to an embodiment of the present invention.

A front panel (not shown) or control means (input means) provided on the digital device 600 is used to control the digital device 600.

On the other hand, the control means is a user interface device (UID; User Interface Device) capable of wired and wireless communication, a remote control 610, a keyboard 630, a pointing device 620 implemented mainly for the control purpose of the digital device 600, A touch-pad or the like is included, but control means dedicated to external input connected to the digital device 600 may also be included. In addition, the control means also includes a mobile device such as a smart phone or a tablet PC that controls the digital device 600 through mode switching or the like, although it is not intended to control the digital device 600. However, in this specification, for convenience, a pointing device is described as an example, but is not limited thereto.

The input means are communication protocols such as Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Digital Living Network Alliance (DLNA), RS, etc. If necessary, at least one is employed to communicate with the digital device.

The remote controller 610 refers to a conventional input means equipped with various key buttons necessary for controlling the digital device 600.

The pointing device 620 is equipped with a gyro sensor or the like to implement a pointer corresponding to the screen of the digital device 600 based on the user's movement, pressure, rotation, and the like, thereby implementing the digital device 600 ) To a certain control command. The pointing device 620 may be named with various names such as a magic remote controller and a magic controller.

The keyboard 630 is an intelligent integrated digital device that provides a variety of services, such as a web browser, an application, and a social network service (SNS), beyond the digital device 600 providing only a conventional broadcast, so that the control is performed only with the conventional remote controller 610. It was not easy, and it was implemented to enhance the convenience of inputting text, etc. by implementing it similar to a PC keyboard.

On the other hand, the control means such as the remote control 610, the pointing device 620, and the keyboard 630 are provided with a touch pad as necessary, thereby providing more convenient and various control purposes such as text input, pointer movement, and enlargement/reduction of pictures or videos. Can be used in.

The digital device described in this specification uses a web OS as an OS and/or platform. Hereinafter, a processing process such as a configuration or algorithm based on WebOS may be performed by the control unit of the above-described digital device. Here, the control unit includes the control unit in FIGS. 2 to 5 described above, and is used as a broad concept. Accordingly, hereinafter, for processing of webOS based or related services, applications, and contents in a digital device, hardware or components including related software, firmware, and the like are configured as a controller. Explain by name.

The webOS-based platform is intended to enhance development independence and function scalability by integrating services, applications, etc., for example, based on a Luna-service bus, and developing applications based on a web application framework. Productivity can also be increased. In addition, multi-tasking can be supported by efficiently utilizing system resources through web OS processes and resource management.

On the other hand, the web OS platform described in this specification can be used in mobile devices such as mobile phones, smart phones, tablet pcs, notebooks, wearable devices, as well as fixed devices such as PCs, TVs, and set top boxes (STBs). .

The structure of software for digital devices is a single process and a closed product based on multi-threading with a conventional problem solving and market-dependent monolithic structure. As a result, there were difficulties in external applications, and after that, aimed for new platform-based development and pursued cost innovation through chip-set replacement and efficiency in UI application and external application development. Layering & componentization ) To have a three-layered structure, an add-on structure for add-on, single source products, and open applications. In recent years, the software structure has been further improved to provide a modular architecture of functional units, a Web Open API (Application Programming Interface) for an eco-system, and a game engine. A modular design for a native open API or the like has been made, and accordingly, it has been generated as a multi-process structure based on a service structure.

7 is a diagram illustrating a web OS architecture according to an embodiment of the present invention.

Referring to FIG. 7, the architecture of the web OS platform will be described as follows.

The platform can be roughly classified into a kernel, a system library based Web OS core platform, an application, a service, and the like.

The architecture of the WebOS platform is a layered structure, with the OS at the lowest layer, the system library(s) at the next layer, and the applications at the top.

First, a lowermost layer may include a Linux kernel as an OS layer and include Linux as an OS of the digital device.

Above the OS layer, a BSP (Board Support Package)/HAL (Hardware Abstraction Layer) layer, a Web OS core modules layer, a service layer, and a Luna-Service bus layer (Luna-Service) Bus layer), Nyo framework/NDK (Native Developer◎ Kit)/QT layer (Enyo framework/NDK/QT layer) and the application layer are sequentially present in the top layer.

Meanwhile, some of the above-described web OS layer structures may be omitted, and a plurality of layers may be layered or a layer may be a plurality of layer structures.

The webOS core module layer is based on Luna Surface Manager (LSM) that manages surface windows, SAM (System & Application Manage) that manages execution and execution status of applications, and WebKit. It may include a Web Application Manager (WAM) for managing web applications.

The LSM manages an application window displayed on the screen. The LSM controls the display hardware (Display HW), provides a buffer for applications to render necessary content, and composites the results rendered by a plurality of applications on the screen. Can print

The SAM manages execution policies for various conditions of systems and applications.

On the other hand, WAM, the web OS can view the web application (Web App) as a basic application, which is based on the Enyo Framework (Enyo Framework).

The service use of the application is performed through a Luna-service bus, and a service can be newly registered on the bus, and the application can also find and use a service that it needs.

The service layer may include various service level services such as TV service and WebOS service. Meanwhile, the web OS service may include a media server, Node.JS, etc. In particular, the Node.JS service supports, for example, JavaScript.

WebOS service is a Linux process that implements function logic and can communicate through the bus. It can be largely divided into four parts, TV processes and services that are migrated to WebOS from existing TVs, or services that are manufacturer differentiated services, webOS common services and JavaScript, and used through Node.js. It consists of Node.js service.

The application layer may include all applications that can be supported by a digital device, such as a TV application, a showcase application, a native application, and a web application.

Depending on the implementation method, the application on the web OS can be divided into a web application, a PDK (Palm Development Kit) application, and a Qt Meta Language or Qt Modeling Language (QML) application.

The web application is based on a WebKit engine and is executed on the WAM runtime. The web application may be based on the ENYO framework, or developed and executed based on general HTML5, Cascading Style Sheets (CSS), and JavaScript.

The PDK application includes a native application developed in C/C++ based on a PDK provided for a third-party or external developer. The PDK refers to a set of development libraries and tools provided for third parties such as games to develop native applications (C/C++). For example, PDK applications can be used to develop applications where performance is important.

The QML application is a native application based on Qt, and includes a basic application provided with a web OS platform, such as a card view, a home dashboard, and a virtual keyboard. Here, QML is a script-type mark-up language instead of C++.

Meanwhile, in the above, the native application refers to an application developed and compiled in C/C++ and executed in a binary form, and such a native application has an advantage of fast execution speed.

8 is a diagram illustrating an architecture of a web OS device according to an embodiment of the present invention.

8 is a block diagram based on a runtime of a web OS device, which can be understood by referring to the layered structure of FIG. 7.

Hereinafter, description will be given with reference to FIGS. 7 and 8.

Referring to FIG. 8, services and applications and WebOS core modules are included on a system OS (Linux) and system libraries, and communication between them may be performed through a Luna-service bus.

Node.js services based on HTML5, CSS, and JavaScript, including e-mail, contact, and calendar, logging, backup, and file notify webOS services such as notify, database (DB), activity manager, system policy, audio daemon (AudioDmon), update, media server, etc., EPG TV services such as (Electronic Program Guide), PVR (Personal Video Recorder), data broadcasting, voice recognition, Now on, Notification, search , CP services such as ACR (Auto Content Recognition), CBOX (Contents List Broswer), wfdd, DMR, Remote Application, Download, SDPIF (Sony Philips Digital Interface Format), PDK applications, browser , Native applications such as QML applications, and UI related TV applications and web applications based on the Enyo framework are processed through the web OS core modules such as SAM, WAM, and LSM described above through the Luna-service bus. Meanwhile, in the above, TV applications and web applications may not necessarily be based on the Enyo framework or UI.

CBOX can manage the list and metadata of the contents of external devices such as USB, DLNA, and cloud connected to the TV. Meanwhile, CBOX can output content listings of various content containers, such as USB, DMS, DVR, cloud, etc., as an integrated view. In addition, CBOX can display various types of content listings such as pictures, music, and videos, and manage their metadata. In addition, the CBOX can output the contents of the attached storage in real-time. For example, when a storage device such as USB is plugged in, the CBOX should be able to immediately output a content list of the storage device. At this time, a standardized method for processing the content listing may be defined. In addition, CBOX can accommodate a variety of connection protocols.

The SAM is intended to improve module complexity and improve scalability. For example, the existing system manager handles several functions such as system UI, window management, web application runtime, and constraint processing on UX in a single process. By clarifying the inter-interface, the implementation complexity is reduced.

LSM supports system UX implementation such as card view and launcher to be independently developed and integrated, and supports to easily respond to changes in product requirements. Meanwhile, LSM enables multi-tasking by maximizing the use of hardware resources (HW resource) when synthesizing a plurality of application screens such as app-on-app, multi-window and 21:9. It can provide a window management mechanism (window management mechanism) for.

LSM supports the implementation of system UI based on QML, and improves the development productivity. QML UX is based on MVC, so it is easy to construct views for screen layout and UI components, and it is also easy to develop code for processing user input. On the other hand, the interface between the QML and the webOS component is done through the QML extension plug-in, and the graphic operation of the application can be based on the wayland protocol, luna-service call, etc. have.

LSM is an abbreviation of Luna Surface Manager, as described above, and functions as an application window compositor.

LSM composes and outputs independently developed applications and UI components on the screen. In relation, when components such as a Recents application, a showcase application, and a launcher application respectively render their contents, LSM defines an output area, an interworking method, and the like as a compositor. In other words, the compositor LSM handles graphic composition, focus management, and input events. At this time, the LSM receives events, focus, and the like from an input manager. Such input managers may include a remote controller, HID such as a mouse & keyboard, joystick, game pad, application remote, pen touch, and the like.

As such, the LSM supports multiple window models, which can be executed simultaneously in all applications due to the system UI characteristics. Related to: Launcher, Recents, Settings, Notifications, System Keyboard, Volume UI, Search, Finger gesture, Voice Recognition (STT (Sound to Text), TTS (Text to) LSM can support Sound, NLP (Natural Language Processing), etc., pattern gesture (camera, Mobile Radio Control Unit (MRCU)), Live menu, ACR (Auto Content Recognition), etc. .

9 is a diagram illustrating a graphic composition flow in a web OS device according to an embodiment of the present invention.

Referring to FIG. 9, the graphic composition processing includes a web application manager 910 in charge of a UI process, a webkit 920 in charge of a web process, a LSM 930 and a graphic manager (GM) 940.

When the web application-based graphic data (or application) is generated as a UI process in the web application manager 910, if the generated graphic data is not a full-screen application, it is transferred to the LSM 930. On the other hand, the web application manager 910 receives the application generated by the web kit 920 to share the graphic processing unit (GPU) memory for graphic management between the UI process and the web process, and this is full-screen as described above. If it is not an application, it is delivered to the LSM 930. In the case of the full-screen application, the LSM 930 may be bypassed, and in this case, it may be directly transferred to the graphic manager 940.

The LSM 930 transmits the received UI application through the Wayland surface to the Wayland Compositor, and processes it appropriately in the Wayland Compositor and delivers it to the graphic manager. The graphic data transmitted from the LSM 930 is transmitted to the graphic manager compositor through the LSM GM surface of the graphic manager 940, for example.

Meanwhile, as described above, the full-screen application is delivered directly to the graphic manager 940 without going through the LSM 930, which is processed by the graphic manager compositor through the WAM GM surface.

The graphic manager processes all graphic data in the webOS device, such as the data through the LSM GM surface described above, the data through the WAM GM surface, as well as the GM surface, such as a data broadcasting application and a caption application. All graphic data passed through is received and processed to be properly displayed on the screen. Here, the function of the GM compositor is the same or similar function to the above-described compositor.

10 is a view illustrating a media server according to an embodiment of the present invention, and FIG. 11 is a diagram illustrating a configuration block diagram of a media server according to an embodiment of the present invention, and FIG. 12 Is a diagram illustrating a relationship between a media server and a TV service according to an embodiment of the present invention.

The media server supports the execution of various multimedia in the digital device and manages necessary resources. The media server can efficiently use hardware resources required for media play. For example, the media server needs audio/video hardware resources to execute multimedia, and can efficiently utilize the resource usage status. In general, a fixed device having a larger screen than a mobile device requires more hardware resources when executing multimedia, and the encoding/decoding and graphic data transfer speeds need to be fast due to a large amount of data. Meanwhile, in addition to streaming and file-based playback, the media server performs broadcasting, recording, and tuning tasks, recording simultaneously with viewing, or simultaneously displaying a sender and a receiver screen during a video call. It should be able to handle the back. However, in the media server, hardware resources such as an encoder, a decoder, a tuner, and a display engine are limited in a chip-set unit, so it is difficult to execute multiple tasks at the same time. Process the input.

The media server may enhance system stability, for example, by removing and restarting a playback pipeline for which an error occurred during media playback for each pipeline, and thus generating an error as described above. This may not affect other media play. The pipeline is a chain that connects each unit functions such as decoding, analysis, and output when requesting media playback, and required unit functions may be changed according to a media type.

The media server may have extensibility, for example, a new type of pipeline may be added without affecting the existing implementation. For example, the media server may accommodate a camera pipeline, a video conference (Skype) pipeline, a third-party pipeline, and the like.

The media server can process normal media playback and TV task execution as separate services because the interface of the TV service is different from the media playback case. In the above, the media server supports operations such as setchannel, channelup, channeldown, channeltuning, recordstart, etc. in relation to the TV service, and supports operations such as play, pause, stop, etc. in relation to general media playback, and is different for both. It supports operations and can be processed as a separate service.

The media server can control or integrate resource management functions. The hardware resource allocation and retrieval in the device are integrated in the media server, and in particular, the TV service process delivers the running task and resource allocation status to the media server. The media server secures resources each time each media is executed and the pipeline is executed, and allows execution by priority (for example, policy) when requesting media execution, based on the resource status occupied by each pipeline, and Resource recovery of other pipelines is performed. Here, predefined execution priority and required resource information for a specific request are managed by a policy manager, and the resource manager can communicate with the policy manager to process resource allocation, recovery, and the like.

The media server may have an identifier (ID) for all operations related to playback. For example, the media server may issue a command by instructing a specific pipeline based on the identifier. The media server may issue commands to the pipelines by dividing the two in order to play two or more media.

The media server may be responsible for playing HTML 5 standard media.

In addition, the media server may follow the TV restructuring scope for a separate service process of the TV pipeline. The media server may be designed and implemented regardless of the TV restructuring range. If the TV is not serviced separately, the entire TV may need to be re-run when a problem occurs in a specific task.

The media server is also called a uMS, that is, a micro media server. Here, the media player is a media client, for example, web for HTML5 video tag, camera, TV, Skype, second screen, etc. It can mean a kit (Webkit).

In the media server, management of micro resources such as a resource manager and a policy manager is a key function. In relation to this, the media server also controls a playback control role for web standard media content. In this regard, the media server can also manage pipeline controller resources.

The media server supports, for example, extensibility, reliability, and efficient resource usage.

In other words, uMS, that is, the media server is a cloud game (cloud game), MVPD (pay service, etc.), camera preview (camera preview), second screen (2nd screen), skyOS, and other resources such as web OS devices and TV resources It manages and controls resource use for proper processing in general, and functions to manage and control for efficient use. Meanwhile, each resource uses a pipeline, for example, when used, and the media server can overall manage and control the creation, deletion, and use of the pipeline for resource management.

Here, a pipeline is created when, for example, a media related to a task starts continuation of operations such as request, decoding stream, video output, and other parsing. Can be. For example, in relation to a TV service or application, watching, recording, channel tuning, etc. are individually handled under control of resource usage through a pipeline generated according to the request. .

Referring to FIG. 10, the processing structure and the like of the media server will be described in more detail as follows.

In FIG. 10, an application or service is connected to a media server 1020 and a Luna-service bus 1010, and the media server 1020 communicates with pipelines regenerated through the Luna-service bus 1010. Connected and managed.

The application or service has various clients according to its characteristics, and through which data can be exchanged with the media server 1020 or the pipeline.

The client includes, for example, a uMedia client (web kit) and a resource manager (RM) client (C/C++) for connecting with the media server 1020.

The application including the uMedia client is connected to the media server 1020 as described above. More specifically, the uMedia client corresponds to a video object to be described later, for example, and the client uses the media server 1020 for the operation of the video by request or the like.

Here, the video operation is related to a video state, and loading, unloading, play, playback, reproduce, pause, stop, etc. are all states related to the video action. Data. Each operation or state of the video can be processed through the creation of a separate pipeline. Accordingly, the uMedia client transmits state data related to the video operation to the pipeline manager 1022 in the media server.

The pipeline manager 1022 acquires information about a resource of the current device through data communication with the resource manager 1024 and requests allocation of a resource corresponding to the status data of the uMedia client. At this time, the pipeline manager 1022 or the resource manager 1024 controls the resource allocation through data communication with the policy manager 1026 when necessary. For example, when the resource manager 1024 has no or insufficient resources to allocate at the request of the pipeline manager 1022, appropriate resource allocation or the like is made according to the request according to the priority comparison of the policy manager 1026 or the like. Can.

Meanwhile, the pipeline manager 1022 requests the media pipeline controller 1028 to create a pipeline for an operation according to the request of the uMedia client for the resource allocated according to the resource allocation of the resource manager 1024. .

The media pipeline controller 1028 creates a necessary pipeline under the control of the pipeline manager 1022. As shown in the pipeline, the media pipeline and the camera pipeline, as well as pipelines related to playback, pause, pause, etc., can be created in the pipeline. Meanwhile, the pipeline may include a pipeline for HTML5, web CP, smartshare playback, thumbnail extraction, NDK, cinema, Multimedia and Hypermedia Information coding Experts Group (MHEG), and the like.

In addition, the pipeline may include, for example, a service-based pipeline (own pipeline) and a URI-based pipeline (media pipeline).

Referring to FIG. 10, an application or service including an RM client may not be directly connected to the media server 1020. This is because the application or service may directly handle the media. In other words, when an application or service directly processes media, it may not go through a media server. However, at this time, resource management is required for pipeline creation and its use. For this, the uMS connector functions. Meanwhile, when a resource management request for direct media processing of the application or service is received, the uMS connector communicates with the media server 1020 including the resource manager 1024. To this end, the media server 1020 should also be provided with a uMS connector.

Accordingly, an application or service may respond to a request of an RM client by receiving resource management of the resource manager 1024 through the uMS connector. These RM clients can handle services such as native CP, TV service, second screen, flash player, YouTube MSE (Medai Source Extensions), cloud games, and Skype. In this case, as described above, the resource manager 1024 may manage the resource through data communication with the policy manager 1026 when necessary for resource management.

On the other hand, the URI-based pipeline is made through the media server 1020, not in the case of directly processing media like the RM client described above. The URI-based pipeline may include a player factory, a Gstreamer, a streaming plug-in, and a Digital Rights Management (DRM) plugin pipeline.

Meanwhile, an interface method between an application and media services may be as follows.

This is a method of interfacing with a service in a web application. This is a method of making a Luna Call using a PSB (Palm Service Bridge) and a method of using Cordova, which extends the display to a video tag. In addition, there may be a way to use the HTML5 standard for video tags or media elements.

And, it is a method of interfacing by using a service in the PDK.

Or, it is a method of using the service in the existing CP. This can be used by extending the plug-in of the existing platform based on Luna for backward compatibility.

Lastly, in case of non-webOS, it is a way to interface. In this case, the Luna bus can be called directly to interface.

Seamless change is processed by a separate module (for example, TVWIN), which is a process for displaying and displaying the TV on the screen without webOS first and seamlessly, before or during booting of the webOS. . This is used for the purpose of first providing the basic function of the TV service for quick response to a user's Power On request because the booting time of the WebOS is late. In addition, the module supports a firmware change, factory mode, etc., which provide fast booting and basic TV functions as part of the TV service process. In addition, the module may be in charge of switching from Non-WebOS mode to WebOS mode.

11, the processing structure of the media server is illustrated.

In this case, in FIG. 11, the solid line box indicates a process processing configuration, and the dotted line box indicates an internal processing module during the process. In addition, a solid arrow indicates an inter-process call, that is, a Luna service call, and a dotted arrow indicates a notification or data flow such as register/notify.

The service or web application or PDK application (hereinafter referred to as an application) is connected to various service processing components through a Luna-service bus, through which the application operates or is controlled.

The data processing path varies depending on the type of application. For example, when the application is image data related to a camera sensor, it is transmitted to the camera processing unit 1130 for processing. At this time, the camera processor 1130 processes image data of the received application, including a gesture, a face detection module, and the like. Here, the camera processing unit 1130 may process the data by creating a pipeline through the media server processing unit 1110, for example, when the user selects or automatically uses a pipeline or the like.

Alternatively, when the application includes audio data, the corresponding audio may be processed through an audio processing unit (AudioD) 1140 and an audio module (PulseAudio) 1150. For example, the audio processing unit 1140 processes audio data received from an application and transmits it to the audio module 1150. At this time, the audio processing unit 1140 may determine the processing of audio data, including an audio policy manager. The audio data processed in this way is processed by the audio module 1160. Meanwhile, the application may notify data related to audio data processing to the audio module 1160, which may notify the audio module 1160 in the related pipeline. The audio module 1150 includes an Advanced Linux Sound Architecture (ALSA).

Alternatively, when the application includes or processes (hereinafter included) DRM-hung content, the corresponding content data is transmitted to the DRM service processing unit 1160, and the DRM service processing unit 1170 creates a DRM instance. To process DRM-hung content data. Meanwhile, the DRM service processing unit 1160 may connect and process the DRM pipeline in the media pipeline and the Luna-service bus in order to process the DRM-hung content data.

Hereinafter, processing in the case where the application is media data or TV service data (eg, broadcast data) will be described.

FIG. 12 is a view for explaining in more detail only the media server processor and the TV service processor in FIG. 11 described above.

Therefore, hereinafter, description will be given with reference to FIGS. 11 and 12 together.

First, when the application includes TV service data, it is processed by the TV service processing unit 1120/1220.

Here, the TV service processing unit 1120 includes, for example, at least one of a DVR/channel manager, a broadcasting module, a TV pipeline manager, a TV resource manager, a data broadcasting module, an audio configuration module, and a route manager. Alternatively, in FIG. 12, the TV service processing unit 1220 includes a TV broadcast handler, a TV broadcast interface, a service processing unit, a TV middleware (TV middleware), a route manager, and a BSP ( For example, NetCast). Here, the service processing unit may mean, for example, a module including a TV pipeline manager, a TV resource manager, a TV policy manager, and a USM connector.

In the present specification, the TV service processing unit may have a configuration as shown in FIG. 11 or 12 or may be implemented in a combination of both, and some components may be omitted or some components not shown may be added.

The TV service processing unit 1120/1220 transmits to the DVR/channel manager in case of digital video recorder (DVR) or channel related data based on the attribute or type of TV service data received from the application, and then back to the TV pipeline manager To the TV pipeline and process it. On the other hand, when the attribute or type of the TV service data is broadcast content data, the TV service processor 1120 generates and processes the TV pipeline through the TV pipeline manager to process the data through the broadcast module.

Alternatively, a json (Javascript standard object notation) file or a file written in c is processed by the TV broadcast handler and transmitted to the TV pipeline manager through the TV broadcast interface to generate and process the TV pipeline. In this case, the TV broadcast interface unit may transmit data or files that have passed through the TV broadcast handler to the TV pipeline manager based on the TV service policy for reference when creating the pipeline.

Meanwhile, the TV pipeline manager may be under the control of the TV resource manager in creating one or more pipelines according to a TV pipeline generation request from a processing module or a manager in the TV service. Meanwhile, the TV resource manager may be controlled by the TV policy manager to request the status and allocation of resources allocated for the TV service according to the TV pipeline manager's request to create a TV pipeline, and the media server processor 1110 /1210) and uMS connector. The resource manager in the media server processing unit 1110/1210 transmits the status and allocation of resources for the current TV service at the request of the TV resource manager. For example, as a result of checking the resource manager in the media server processing unit 1110/1210, if all the resources for the TV service have already been allocated, the TV resource manager may notify that all resources are currently allocated. At this time, the resource manager in the media server processing unit, together with the notify, removes a predetermined TV pipeline according to a priority or a predetermined criterion among TV pipelines allocated for TV service and TV pipeline for the requested TV service. You can also request or allocate creation. Alternatively, the TV resource manager may appropriately remove, add, or establish a TV pipeline in the TV resource manager according to the status report of the resource manager in the media server processor 1110/1210.

Meanwhile, the BSP supports backward compatibility with, for example, existing digital devices.

The TV pipelines thus generated can be properly operated under the control of the route manager in the process. The route manager may determine or control the processing route or process of the pipelines in consideration of not only the TV pipeline but also the operation of the pipeline generated by the media server processor 1110/1210 in the process.

Next, when the application includes media data rather than TV service data, the media server processing unit 1110/1210 processes it. Here, the media server processing unit 1110/1210 includes a resource manager, a policy manager, a media pipeline manager, a media pipeline controller, and the like. On the other hand, the pipeline created under the control of the media pipeline manager and the media pipeline controller can be variously generated, such as a camera preview pipeline, a cloud game pipeline, and a media pipeline. Meanwhile, the media pipeline may include a streaming protocol, an auto/static gstreamer, DRM, etc., and the processing flow may be determined under the control of the route manager. The detailed processing process in the media server processing unit 1110/1210 uses the above-described description of FIG. 10, and is not repeated herein.

In the present specification, the resource manager in the media server processing unit 1110/1210 may manage resources on a counter basis, for example.

13 is a diagram illustrating a control method of a remote control device that controls any one of video display devices according to embodiments of the present invention.

As illustrated in FIG. 13A, a pointer 205 corresponding to the remote control device 200 is displayed on the display unit 180.

The user can move or rotate the remote control device 200 up and down, left and right (FIG. 13 (b)), and forward and backward (FIG. 13 (c)). The pointer 205 displayed on the display unit 180 of the image display device corresponds to the movement of the remote control device 200. The remote control device 200 may be referred to as a spatial remote controller because the corresponding pointer 205 is moved and displayed according to movement in 3D space, as shown in the drawing.

13B illustrates that when the user moves the remote control device 200 to the left, the pointer 205 displayed on the display unit 180 of the image display device moves to the left correspondingly.

Information on the movement of the remote control device 200 detected through the sensor of the remote control device 200 is transmitted to the image display device. The video display device may calculate the coordinates of the pointer 205 from information related to the movement of the remote control device 200. The image display device may display the pointer 205 to correspond to the calculated coordinates.

13C illustrates a case in which a user moves the remote control device 200 away from the display unit 180 while pressing a specific button in the remote control device 200. As a result, the selected area in the display unit 180 corresponding to the pointer 205 may be zoomed in and displayed. Conversely, when the user moves the remote control device 200 to be closer to the display unit 180, the selected area in the display unit 180 corresponding to the pointer 205 may be zoomed out and reduced. On the other hand, when the remote control device 200 moves away from the display unit 180, the selection area is zoomed out, and when the remote control device 200 approaches the display unit 180, the selection area may be zoomed in.

Meanwhile, in a state in which a specific button in the remote control device 200 is pressed, recognition of up, down, left and right movement may be excluded. That is, when the remote control device 200 moves to move away from or approach the display unit 180, the up, down, left, and right movements are not recognized, and only the forward and backward movements can be recognized. When a specific button in the remote control device 200 is not pressed, only the pointer 205 moves according to the up, down, left, and right movement of the remote control device 200.

Meanwhile, the moving speed or the moving direction of the pointer 205 may correspond to the moving speed or the moving direction of the remote control device 200.

Meanwhile, the pointer in the present specification means an object displayed on the display unit 180 in response to the operation of the remote control device 200. Accordingly, various shapes of objects other than the arrow shape illustrated in the drawing are possible with the pointer 205. For example, it may be a concept including points, cursors, prompts, and thick outlines. In addition, the pointer 205 may be displayed corresponding to any one point of the horizontal axis and the vertical axis on the display unit 180, as well as a plurality of points, such as a line and a surface. Do.

14 is an internal block diagram of a remote control device controlling any one of video display devices according to embodiments of the present invention.

As shown in FIG. 14, the remote control device 200 includes a wireless communication unit 225, a user input unit 235, a sensor unit 240, an output unit 250, a power supply unit 260, and a storage unit 270. , It may include a control unit 280.

The wireless communication unit 225 transmits and receives a signal to any one of the video display devices according to the embodiments of the present invention described above. Among the video display devices according to embodiments of the present invention, one video display device 100 will be described as an example.

In this embodiment, the remote control device 200 may include an RF module 221 capable of transmitting and receiving signals to and from the video display device 100 according to the RF communication standard. Also, the remote control device 200 may include an IR module 223 capable of transmitting and receiving signals to and from the video display device 100 according to the IR communication standard.

In this embodiment, the remote control device 200 transmits a signal containing information about the movement of the remote control device 200 to the video display device 100 through the RF module 221.

In addition, the remote control device 200 may receive a signal transmitted from the video display device 100 through the RF module 221. In addition, the remote control device 200 may transmit a command for power on/off, channel change, volume change, etc. to the image display device 100 through the IR module 223 as necessary.

The user input unit 235 may include a keypad, a button, a touch pad, or a touch screen. The user may input a command related to the image display device 100 to the remote control device 200 by operating the user input unit 235. When the user input unit 235 has a hard key button, the user may input a command related to the image display device 100 to the remote control device 200 through a push operation of the hard key button. When the user input unit 235 includes a touch screen, the user may input a command related to the video display device 100 to the remote control device 200 by touching a soft key of the touch screen. In addition, the user input unit 235 may include various types of input means that the user can operate, such as a scroll key or a jog key, and this embodiment does not limit the scope of the present invention.

The sensor unit 240 may include a gyro sensor 241 or an acceleration sensor 243.

The gyro sensor 241 may sense information on the movement of the remote control device 200.

For example, the gyro sensor 241 may sense information on the operation of the remote control device 200 based on the x, y, and z axes. The acceleration sensor 243 may sense information related to the movement speed of the remote control device 200. Meanwhile, a distance measurement sensor may be further provided, whereby a distance from the display unit 180 can be sensed.

The output unit 250 may output an image or audio signal corresponding to the operation of the user input unit 235 or a signal transmitted from the image display device 100. The user may recognize whether the user input unit 235 is operated or whether the image display device 100 is controlled through the output unit 250.

For example, the output unit 250 includes a LED module 251 that is turned on when the user input unit 235 is operated or a signal is transmitted and received through the wireless communication unit 225, a vibration module generating vibration ( 253), an audio output module 255 for outputting sound, or a display module 257 for outputting an image.

The power supply unit 260 supplies power to the remote control device 200. The power supply unit 260 may reduce power waste by stopping power supply when the remote control device 200 has not moved for a predetermined time. The power supply unit 260 may resume power supply when a predetermined key provided in the remote control device 200 is operated.

The storage unit 270 may store various types of programs and application data required for control or operation of the remote control device 200. If the remote control device 200 transmits and receives signals wirelessly through the video display device 100 and the RF module 221, the remote control device 200 and the video display device 100 transmit signals through a predetermined frequency band. Send and receive. The control unit 280 of the remote control device 200 stores information on a frequency band capable of wirelessly transmitting and receiving signals to and from the video display device 100 paired with the remote control device 200 in the storage unit 270 and Can be referenced.

The control unit 280 controls all matters related to the control of the remote control device 200. The control unit 280 displays a signal corresponding to a predetermined key manipulation of the user input unit 235 or a signal corresponding to the movement of the remote control device 200 sensed by the sensor unit 240 through the wireless communication unit 225. (100).

Hereinafter, embodiments related to a control method that can be implemented in the display device configured as described above will be described with reference to the accompanying drawings. It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit and essential characteristics of the present invention.

In addition, hereinafter, the display device illustrated in FIG. 15 as a display device will be described as an example. However, it is needless to say that the display device according to an embodiment of the present invention may be implemented as the display devices shown in FIGS. 1 to 14.

In addition, hereinafter, with reference to FIGS. 15 to 26, the display device and its control method will be described in detail. However, a person skilled in the art may refer to FIGS. 1 to 14 previously described, and supplement or interpret the examples 15 to 26 or modify embodiments.

15 is a configuration diagram of a display device according to an embodiment of the present invention.

Referring to FIG. 15, the display device 1500 includes a tuner 1510, a communication unit 1520, a controller 1530, a display 1540, a speaker 1550, a microphone 1560, a memory 1570, and a camera 1580 ).

The tuner 1510 receives a broadcast signal from a broadcasting station.

The communication unit 1520 communicates with at least one of an external server and an external remote control.

The display 1540 displays content included in the received broadcast signal on the screen. Here, the content is received from an external server or stored in the internal memory.

The speaker 1550 outputs a first sound generated when playing content. For example, the first sound may be a TV sound.

The microphone 1560 receives a second sound including a trigger word from the user, and receives the first sound from the speaker 1550. For example, the second sound may be the user's voice. According to an embodiment of the present invention, the microphone 1560 may receive TV sound and a user's voice.

The memory 1570 stores a plurality of human images.

The camera 1580 captures the front image of the display device 1500 according to the control command of the controller 1530.

The controller 1530 controls at least one of a tuner 1510, a communication unit 1520, a display 1540, a speaker 1550, a microphone 1560, a memory 1570, and a camera 1580.

When not displaying content, the controller 1530 displays at least one first image in the first area and receives an input for selecting any one of the at least one first image from a user, thereby selecting the selected first image When a plurality of second images corresponding to is displayed on the second area, and an input for reproducing the first image is received from the user, the plurality of second images are displayed on the screen at predetermined time intervals.

16 is a diagram illustrating a control method of a display device according to an embodiment of the present invention. The invention is performed by a controller 1530.

Referring to FIG. 16, first, a broadcast signal is received from a broadcasting station through a tuner (S1610).

The content included in the received broadcast signal is displayed on the screen (S1620).

When the content is not displayed, at least one first image is displayed in the first area (S1630).

When an input for selecting any one of the at least one first image is received from the user, a plurality of second images corresponding to the selected first image are displayed on the second area (S1640).

When an input for reproducing the first image is received from the user, the plurality of second images are displayed on the screen at predetermined time intervals (S1650).

17 is a diagram illustrating an initial screen when a gallery application of a display device is executed according to an embodiment of the present invention. 17 includes FIGS. 17(a), 17(b), and 17(c).

Referring to FIG. 17A, when the gallery application is executed, the controller 1530 displays a first theme image 1710, a second theme image 1712, and a third theme image 1714.

The theme image consists of a built-in theme image and a downloadable theme image. For example, i) Built-in theme can be Sunny Day, Rainy Day, Art Gallery, and ii) Downloadable theme image can be Art Photos, Colorful Spring.

In the theme image area, a representative image of each theme is displayed. In addition, when the pointer 1 is over the theme image 1710, the controller 1530 updates the lower part of the theme preview 1720.

When receiving an input for selecting a specific theme image, the controller 1530 automatically plays a specific theme repeatedly. In addition, when the third theme image 1714 is selected, but the download is not possible, the controller 1530 downloads the third theme image. If the download is not successful, the controller 1530 displays an indicator 1716 indicating that the download is possible. In addition, when the mode is a store mode, the controller 1530 selects a downloadable theme, and displays a text message indicating that it is unavailable.

Next, the theme image preview 1720 will be described. The theme image preview 1720 includes the theme title, theme information, and image list. The image list includes a first image, a second image, a third image, a fourth image, and a fifth image.

The theme information includes i) a photo theme, a plurality of photo information, and ii) a video theme, a video information. The image list provides an image of the entire work included in the theme. When selecting an image in the image list, the controller 1530 plays back the selected image. If the theme image is a future image, "New theme will be updated soon." Text message 1730 may be displayed.

When an input for selecting the my theme 1718 is received from the user, the controller 1530 enters an edit screen that executes a function for editing the my theme.

17(b) means a table showing the arrangement order of the theme images.

Referring to FIG. 17(b), the arrangement of the theme images follows in principle the order of this table. For example, if the summer, autumn, and winter theme images are updated, the controller 1530 displays them first and displays a badge.

17(c) is a diagram illustrating a state in which the theme image is updated.

Referring to FIG. 17(c), when the theme image is updated, the controller 1530 displays an image including content indicating this in some areas of the theme image.

18 is a diagram illustrating downloading a downloadable gallery image according to an embodiment of the present invention. 18 includes FIGS. 18(a), 18(b), 18(c), and 18(d).

Referring to FIG. 18(a), an input for selecting a downloadable gallery image 1810 with the pointer 10 is received.

Referring to FIG. 18(b), the controller 1530 displays an image 1820 indicating that it is being downloaded on the screen.

Referring to FIG. 18(c), the controller 1530 displays a progressive bar 1830 indicating a download status on a screen.

Referring to FIG. 18(d), when the download is completed, the controller 1530 displays a toast pop-up message 1840 indicating that the download is completed. The controller 1530 changes the downloaded theme image icon to a down completed icon.

19 is a diagram illustrating editing of a user's personal gallery image according to an embodiment of the present invention.

Referring to FIG. 19, when entering the My Theme editing screen, the controller 1530 displays a description text on the top 1910 of the screen. For example, the description text may be Check out basic themes and downloaded themes.

Next, the controller 1530 displays the theme image in the center portion of the screen 1920. Here, the controller 1530 displays only playable theme images, for example, basic theme images and downloaded theme images. The theme image sort order is the same as the main screen. When a theme image is selected, the controller 1530 automatically displays the entire image in the theme image repeatedly.

Next, upon receiving an input for selecting the delete icon 1930, the controller 1530 changes the mode to the delete mode. If there is no deleteable theme image or no downloaded theme image, the controller 1580 does not display the delete icon 1930.

When an input for selecting the theme selection icon 1940 is received from the user, the controller 1530 changes the mode to the selection mode. In the store mode, the controller 1580 deactivates the theme selection icon 1940. In this case, it works only with full theme image playback.

20 is a diagram illustrating a selection mode according to an embodiment of the present invention.

Referring to FIG. 20, in the selection mode, a description text 2010 is displayed. The explanatory text includes: Theme videos will play in the selected order. The number of themes can be changed in real time when receiving input for selecting the theme.

Next, a description will be given of selecting a theme image 2020. The theme image selects all by default. Then, at the first entry, the controller 1530 selects all. Thereafter, when entering, the controller 1530 maintains the last selection value.

When an input for selecting the SELECT ALL 2030 button is received from the user, the controller 1530 selects the entire theme. Next, while the whole theme is selected, the phrase of the SELECT ALL (2030) button is changed to DESELECT ALL, and when DESELECT ALL is selected, all selections are canceled.

When an input for selecting the CANCEL 2040 button is received from the user, the controller 1530 ends the selection mode when the button is selected and returns to the previous page.

When an input for selecting the PLAY 2050 button is received from the user, the controller 1530 displays a theme image corresponding to the selected theme. When there are multiple selected themes, the theme images corresponding to the themes are displayed in the selected order. If neither theme is selected, the controller 1530 deactivates the PLAY 2050 button. After selecting a theme and selecting PLAY 2050, the theme images corresponding to the theme are played in the order checked by the user.

21 is a diagram illustrating a selection mode execution process according to an embodiment of the present invention. 21 includes FIGS. 21(a), 21(b), 21(c), and 21(d).

Referring to FIG. 21(a), if no theme is selected, the controller 1530 deactivates the play button 30.

Referring to FIG. 21(b), the controller 1530 generates the numbers 2110 in the order selected by the user.

Referring to FIG. 21(c), the controller 1530 generates numbers 1110 and 2120 in the order selected by the user, for example.

Referring to FIG. 21(d), when an input for selecting the SELECT ALL(20) button with the pointer 10 is received from the user, the controller 1530 may select a number in the list order among the themes not selected by the user, for example. , No. 3 (2130), No. 4 (2140), and No. 5 (2150) are generated.

22 is a diagram illustrating a deletion mode according to an embodiment of the present invention.

Referring to FIG. 22, the description text includes the following contents. Select Themes to Delete ({num}). Here, ({num}) means the number of selected themes, and the number can be changed in real time when the theme is selected.

Next, the theme selection 2220 will be described. The controller 1530 can delete only the downloaded theme.

When an input for selecting the SELECT ALL 2230 button is received from the user, the controller 1530 selects the entire downloaded theme. Next, while the whole theme is selected, the phrase of the SELECT ALL (2230) button is changed to DESELECT ALL, and when DESELECT ALL is selected, all selections are canceled.

When an input for selecting the CANCEL 2240 button is received from the user, the controller 1530 ends the selection mode when the button is selected, and returns to the previous page.

When an input for selecting the SELECT ALL 2230 button is received from the user, the controller 1530 selects the entire downloaded theme. Next, while the whole theme is selected, the phrase of the SELECT ALL (2230) button is changed to DESELECT ALL, and when DESELECT ALL is selected, all selections are canceled.

The DELETE 2250 button will be described. If no theme is selected, the controller 1530 deactivates the DELETE 2250 button.

When an input for selecting the DELETE 2250 button is received from the user, the controller 1530 displays an Alert pop-up and deletes the theme. After the theme is deleted, the screen moves to the main screen, and the controller 1530 displays the deletion completed text message in a Toast pop-up format. Next, the controller 1530 changes the deleted theme to a previous state (downloadable icon).

23 is a diagram illustrating a gallery app execution screen according to an embodiment of the present invention.

Referring to FIG. 23, when an input for selecting the list icon 2310 is received from a user, the controller 1530 returns the screen to the list. Here, the tool tip is GO TO LIST and returns to the previous screen when selected.

Next, the Content Info 2320 will be described. Content Info (2320) includes the author's name and the name of the work. When the user presses the remote control confirmation button, the controller 1530 displays on the Control UI screen. Here, Timeout follows a common rule. For example, the timeout can be 5 seconds. When the work is changed while the UI is called, the controller 1530 also changes the work description.

Next, the Music Player 2330 will be described. The tool tip of the music player becomes a MUSIC PLAYER. If the playback has been played even once, the "song name" is displayed on the screen. The controller 1530 displays the Music Player icon on the screen. Here, the music player icon has the same shape as the floating UI.

Next, the theme sound 2340 will be described. In the case of THEME SOUND, there are two states: ON and OFF. In the case of the icon display type, it is displayed as one without on/off distinction. The default value is set to On. When set to Off, it operates as a pause. And the setting value is maintained even after DC Off.

When entering the first playback screen, the controller 1530 plays the work without the Control UI below the screen. For example, in a specific theme, the controller 1530 displays for 10 seconds for each image.

Next, the Music Player 2330 and Theme Sound 2340 will be described.

First, the default default state is Theme Sound On. Next, when playing music in the Music Player 2330, the controller 1530 processes the Theme Sound 2340 off. Next, when the music is paused in the Music Player 2330, the controller 1530 returns to the previous Theme Sound value.

According to an embodiment of the present invention, when entering the theme while using the music player 2330, the controller 1530 plays the music player 2330 first than the theme sound 2240.

When entering Home during theme playback, the controller 1530 displays the name of the theme currently playing under the App Title.

24 is a diagram illustrating an operation sequence of an afterimage prevention image for a gallery according to an embodiment of the present invention. 24 includes FIGS. 24(a), 24(b), and 24(c).

Referring to FIG. 24(a), a gallery slide show progresses, and the controller 1530 repeatedly plays the entire theme.

Referring to FIG. 24(b), the controller 1530 adjusts the brightness of the reproduced image. For example, the controller 1530 gradually reduces the brightness of the image from the first start to 30 minutes and then 150 nit.

Referring to FIG. 24(c), after 30 minutes from the beginning, the controller 1530 plays the gallery saver. The gallery saver functions to protect the screen. When the predetermined time has elapsed, the controller controls to return to the gallery slide show again.

Next, the controller 1530 controls to terminate the gallery saver when the TV is turned on again after the TV is turned off during Gallery Saver playback.

Referring to FIG. 24(d), an operation scenario for each home mode and store mode will be described. The mode can be divided into a home mode and a store mode.

First, in the home mode, in the case of theme playback and gallery saver, when playing one theme, the theme can be played for 1 hour, and the gallery saver can be played for 2 hours.

In the picture mode, the gallery app can be fixed to the gallery picture mode not much. In this case, the OLED Light value can be varied. For example, it can be gradually reduced to 150 nit and maintained until 30 minutes.

25 is a diagram illustrating a pop-up message for each specific situation according to an embodiment of the present invention.

Referring to FIG. 25, first, when the theme is not recognized, that is, if the theme is not recognized and playback is impossible, the controller 1530 displays the first pop-up message 2510.

When the theme download is in progress, the controller 1530 displays the second pop-up message 2520.

When the theme download is in progress and the network is not connected, the controller 1530 displays a third pop-up message 2530.

If a theme download is in progress and more than three downloaded themes attempt to download, the controller 1530 displays a fourth pop-up message 2540.

When the terms pop-up proceeds, the controller 1530 displays the fifth pop-up message 2550.

26 is a diagram illustrating a pop-up message for each specific situation according to an embodiment of the present invention.

Referring to FIG. 26, first, when the theme download is completed, the controller 1530 displays a sixth pop-up message 2560.

When the theme download is in progress, if a downloadable theme is selected from the store load, the controller 1530 displays a seventh pop-up message 2570.

If it is impossible to download the theme due to insufficient internal memory, the controller 1530 displays the eighth pop-up message 2580.

When the theme download is in progress, when the stop button is pressed, the controller 1530 displays a ninth pop-up message 2590.

When the theme deletion fails, the controller 1530 displays the tenth pop-up message 2610.

When the theme deletion is completed, the controller 1530 displays the eleventh pop-up message 2611.

27 is a diagram illustrating a pop-up message for each specific situation according to an embodiment of the present invention.

Referring to FIG. 27, when attempting to delete a theme, when there is one theme, the controller 1530 displays a twelfth pop-up message 2612. When there are two or more themes, the controller 1530 displays a thirteenth pop-up message 2613.

When the new theme is updated, the controller 1530 displays a 14th pop-up message 2614.

28 is a diagram illustrating an audio guide process when a gallery app is executed according to an embodiment of the present invention.

The audio guide process means that when the display device is used by the visually impaired, guided broadcasting is provided. This is because the visually impaired is difficult to control with a remote controller while viewing the screen.

Referring to FIG. 28, when the screen is first entered, when the focus moves to the gallery area 2810, the controller 1530 outputs a voice message. The voice message can be gallery, choose your favorite gallery theme.

Next, when the pointer 10 is moved to the theme title area 2820, the controller 1530 outputs a voice message for the focus item and the theme title. Here, the focus item means a portion of the pointer 10 on the screen.

Next, when the focus moves to the photo title area 2830, the controller 1530 outputs a voice message for the photo title and the artist.

Next, when the focus moves to my theme area 2840, the controller 1530 outputs a voice message for my theme and button input.

When the focus moves to the termination area 2850, the controller 1530 outputs a voice message for the application termination and button input.

29 is a diagram illustrating an audio guide process when editing a personal gallery according to an embodiment of the present invention.

Referring to FIG. 29, when the focus is on the my theme area 2910, the controller 1530 outputs a voice message. Here, the voice message can be said to check my theme, basic theme and downloaded theme.

If the focus is on the theme title area 2920, the controller 1530 outputs a voice message. Here, the voice message may be a theme title.

When the focus is on the delete icon area 2930, the controller 1530 outputs a voice message. Here, the voice message can be Delete Gallery theme, press the button.

When the focus is on the reproduction selection area 2940, the controller 1530 outputs a voice message for the selection item and the artist.

When the focus is in the termination area 2950, the controller 1530 outputs a voice message for the application termination and button input.

30 is a diagram illustrating an audio guide process in a selection mode according to an embodiment of the present invention.

Referring to FIG. 30, when the selection mode 3010 is entered, the controller 1530 outputs a voice message. Here, select the theme to play the voice message, the theme video will play in the selected order. When entering the selection mode, the controller 1530 controls not to read the selection item.

When the focus is on the theme title area 3020, the controller 1530 outputs a voice message for check and uncheck. In addition, when the setting value is changed by selecting content, the controller 1530 outputs a voice message for this.

When the focus is on the entire selection button area 3030, the controller 1530 outputs a voice message for the entire selection and button input.

When the focus is on the delete button area 3040, the controller 1530 outputs a voice message for deletion and button input.

When the focus is on the play button area 3050, the controller 1530 outputs a voice message for play and button input.

When the focus is in the termination area 3060, the controller 1530 outputs a voice message for the application termination and button input.

31 is a diagram illustrating an audio guide process in a gallery app execution screen according to an embodiment of the present invention.

Referring to FIG. 31, upon entering the theme screen, the controller 1530 outputs a voice message for the play theme.

When the focus enters the list area 3010, the controller 1530 outputs a voice message to the list.

When the focus is in the content information area 3020, the controller 1530 outputs a voice message for content information.

When the focus is on the music player area 3030, the controller 1530 outputs a voice message for input of a music player and a button.

When the focus is in the theme sound area 3040, the controller 1530 outputs a voice message for turning on and off the theme sound and inputting a button.

32 is a diagram illustrating a front recognition utterance process in a gallery app execution screen according to an embodiment of the present invention.

Referring to FIG. 32, in the front recognition utterance process, when a user presses a voice recognition button of a remote controller in front of a screen and speaks, a specific starting word included in the user voice is recognized, and the controller 1530 recognizes the specific starting word. It means the process of executing the corresponding function.

First, for a click request, full recognition utterance is available in all clickable UIs on the screen screen.

Next, in the theme title area 3210, when the scroll request 3212 is received from the user, the controller 1530 may scroll the theme title to the right or left. Also, when the content playback request 3214 is received from the user, the controller 1530 plays the content.

Next, in the theme title sub-region 3220, when the scroll request 3222 is received from the user, the controller 1530 scrolls the photo title right or left.

33 is a diagram illustrating a full recognition speech process when editing a personal gallery according to an embodiment of the present invention. First, for a click request, full recognition utterance is available in all clickable UIs on the screen screen.

Referring to FIG. 33, the controller 1530 outputs a description text on the top 3310 of the screen. For example, the description text may be Check out basic themes and downloaded themes.

Next, the controller 1530 displays the theme image in the center portion 3320 of the screen. Here, the controller 1530 displays only playable theme images, for example, basic theme images and downloaded theme images. The theme image sort order is the same as the main screen. When the theme image is selected by voice, the controller 1530 automatically displays the entire image in the theme image repeatedly.

Next, upon receiving an input for selecting the deletion icon 3330 by voice, the controller 1530 changes the mode to the deletion mode. If there is no deleteable theme image or no downloaded theme image, the controller 1580 does not display the delete icon 1930.

When an input for selecting the theme selection icon 3340 by voice is received from the user, the controller 1530 changes the mode to the selection mode. In the store mode, the controller 1530 deactivates the theme selection icon 1940. In this case, it works only with full theme image playback.

34 is a diagram illustrating a front recognition utterance process in a selection mode according to an embodiment of the present invention.

Referring to FIG. 34, in the selection mode, the explanatory text 2010 is displayed. The explanatory text includes: Theme videos will play in the selected order. The number of themes can be changed in real time when receiving input for selecting the theme.

Next, a description will be given of selecting the theme image 3420 by voice. The theme image selects all by default. Then, at the first entry, the controller 1530 selects all. Thereafter, when entering, the controller 1530 maintains the last selection value.

When an input for selecting the SELECT ALL (3430) button by voice is received from the user, the controller 1530 selects the entire theme. Next, when the whole theme is selected, the phrase of the SELECT ALL (3430) button is changed to DESELECT ALL, and when DESELECT ALL is selected, all selections are canceled.

If an input for selecting the CANCEL (3440) button by voice is received from the user, the controller 1530 ends the selection mode when the button is selected, and returns to the previous page.

When an input for selecting the PLAY button 3450 by voice is received from the user, the controller 1530 displays a theme image corresponding to the selected theme. When there are multiple selected themes, the theme images corresponding to the themes are displayed in the selected order. If neither theme is selected, the controller 1530 deactivates the PLAY 2050 button. After selecting a theme and selecting PLAY (3450) by voice, the theme images corresponding to the theme are played in the order checked by the user.

35 is a diagram illustrating a full recognition speech process when a gallery app is executed according to an embodiment of the present invention.

Referring to FIG. 35, when an input for selecting a list icon 3510 by voice is received from a user, the controller 1530 returns the screen to the list. Here, the tool tip becomes a GO TO LIST and returns to the previous screen when selected.

Next, the Content Info 3520 will be described. Content Info (3520) includes the author's name and the name of the work. When the user presses the remote control confirmation button, the controller 1530 displays on the Control UI screen. Here, Timeout follows a common rule. For example, the timeout can be 5 seconds. When the work is changed while the UI is called, the controller 1530 also changes the work description.

Next, the Music Player 3530 will be described. The tool tip of the music player becomes a MUSIC PLAYER. If the playback has been played even once, the "song name" is displayed on the screen. The controller 1530 displays the Music Player icon on the screen. Here, the music player icon has the same shape as the floating UI.

Next, the theme sound 3540 will be described. In the case of THEME SOUND, there are two states: ON and OFF. In the case of the icon display type, it is displayed as one without on/off distinction. The default value is set to On. When set to Off, it operates as a pause. And the setting value is maintained even after DC Off.

When entering the first playback screen, the controller 1530 plays the work without the Control UI below the screen. For example, in a specific theme, the controller 1530 displays for 10 seconds for each image.

Next, the Music Player (3530) and Theme Sound (3540) will be described.

First, the default default state is Theme Sound On. Next, when playing music in the Music Player 3530, the controller 1530 processes the Theme Sound 2340 off. Next, when music is paused in the Music Player 3530, the controller 1530 returns to the previous Theme Sound value.

According to an embodiment of the present invention, when entering the theme while using the music player 3530, the controller 1530 plays the music player 3530 first than the theme sound 3540.

When entering Home during theme playback, the controller 1530 displays the name of the theme currently playing under the App Title.

36 is a diagram illustrating a screen structure when a gallery app is executed according to an embodiment of the present invention.

Referring to FIG. 36, the main page 3610 includes a theme list, a theme photo list, and a my theme.

My theme 3620 includes a theme list, a selection mode, and a deletion mode.

The selection mode 3630 includes a function of playing a selection item. The deletion mode 3640 includes a deletion function.

The theme player 3650 includes a previous step, content information, a music player, and a theme sound On/Off function.

According to an embodiment of the present invention, when the user does not watch TV, the gallery application is executed, and the theme image and video are downloaded from an external server, so that the user can view the image or video, thereby improving user convenience. have.

According to another embodiment of the present invention, when the gallery application is executed, an input for selecting a pre-stored theme image and a downloaded theme image is received from a user, and the theme image can be freely edited, thereby improving user convenience. .

According to another embodiment of the present invention, when the gallery application is executed, an input for selecting all selections, cancellations, and playbacks in a selection mode is received from a user, so that the playback order and grouping order can be adjusted, thereby improving user convenience. Can.

The image display device and the operation method according to the present invention are not limited to the configuration and method of the above-described embodiments, and the above embodiments are all or part of each embodiment so that various modifications can be made. May be selectively combined.

On the other hand, the operation method of the video display device of the present invention can be implemented as a code that can be read by the processor on a recording medium that can be read by the processor provided in the video display device. The processor-readable recording medium includes all types of recording devices in which data that can be read by the processor are stored. Examples of the recording medium that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and include those implemented in the form of carrier waves such as transmission through the Internet. . In addition, the processor-readable recording medium may be distributed over a networked computer system so that the processor-readable code is stored and executed in a distributed manner.

In addition, although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention pertains without departing from the gist of the present invention claimed in the claims. In addition, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical idea or prospect of the present invention.

1500: display device
1510: tuner 1520: communication unit
1530: controller 1540: display
1550: speaker 1560: microphone
1570: memory

Claims (13)

In the display device,
A tuner that receives a broadcast signal from a broadcasting station;
A communication unit performing communication with at least one of an external server and an external remote control;
A memory for storing data for operation of the display device;
A display for displaying content included in the received broadcast signal on a screen; -The content is received from the external server or stored in memory-and
And a controller that controls the tuner, communication unit, memory, and display,
The controller,
When the display does not display the content, controls the display to display at least one first image in a first area,
When an input for selecting any one of the at least one first image is received from a user, the display is controlled to display a plurality of second images corresponding to the selected first image in a second area,
When receiving an input for reproducing the first image from a user, controlling the display to display the plurality of second images on a screen at predetermined time intervals,
Display device. The method of claim 1, wherein the controller
The display is controlled to display a representative image among individual theme images corresponding to the first image in the first area,
If the pointer is over the theme image, the controller updates the theme preview portion of the second area,
The first image includes a first theme image, a second theme image, and a third theme image,
Display device. The method of claim 1, wherein the controller
Upon receiving an input for selecting a downloadable gallery image as a pointer, control to display an image indicating that the display is being downloaded on the screen,
The display is controlled to display a progressive bar indicating the download status on the screen,
When the download is completed, the display controls to display a toast pop-up message on the screen indicating that the download is completed.
Display device. The method of claim 1, wherein the controller
When entering the theme editing screen for editing the first image, the display controls to display a description text on the screen,
Control the display to display a theme image on the screen,
Upon receiving an input for selecting the theme image, the display is automatically controlled to repeatedly display the entire image in the theme image,
When receiving input from the user to select the theme delete icon, the mode is changed to the delete mode,
When receiving input from a user to select a theme selection icon, the mode is changed to a selection mode,
Display device. The method of claim 4, wherein the controller
When receiving input from the user to select the select all button, the entire theme is selected,
When the whole theme is selected, the phrase of the whole selection changes to deselect all.
When selecting the deselect all button, deselect all is executed,
When an input for selecting the cancel button is received from the user, the selection mode is exited and the previous page is returned,
When an input for selecting a play button is received from a user, the display controls to display a theme image corresponding to the selected theme,
Display device. The method of claim 4, wherein the controller
If none of the above theme images are selected, the controller disables the play button,
Display device. The method of claim 1, wherein the controller
When receiving input from a user selecting at least one theme and selecting a delete button, the display controls to display a warning popup,
Delete at least one selected theme,
Control the display to display the main screen,
Controlling the display to display a deleted text message in a toast pop-up format,
Display device. The method of claim 1, wherein the controller
In the state where the gallery application is executed, the display is controlled to display content information including the artist's name, artwork, and artwork description,
When the first signal corresponding to the remote control confirmation button is received from the user, the display is controlled to display a first control screen for changing the content,
If the work is changed while the first control screen is displayed, the display controls to change the work description corresponding to the work as well.
Display device. The method of claim 1, wherein the controller
Controlling the display to display while adjusting the brightness of the plurality of second images, while the display controls to display the plurality of second images on a screen at predetermined time intervals,
Display device. The method of claim 1, wherein the controller
If playback is impossible because the theme corresponding to the first image is not recognized, the display is controlled to display a first pop-up message,
If the theme download is in progress, the display controls to display a second pop-up message,
If the theme download is in progress and the network is not connected, controlling the display to display a third pop-up message,
Display device. The method of claim 10, wherein the controller
When the theme download is completed, the display is controlled to display a sixth pop-up message indicating completion,
If a downloadable theme is selected in the store mode while the theme download is in progress, the display is controlled to display a seventh pop-up message requesting to switch to home mode,
If it is impossible to download a theme due to insufficient internal memory, controlling the display to display an eighth pop-up message indicating insufficient internal memory;
Display device. According to claim 1,
Further comprising a speaker for outputting sound,
The controller
When entering the application execution screen for the first time and the pointer moves to the first area, the speaker is controlled to output a voice message requesting a selection,
When the pointer is moved to the second area, the speaker outputs a voice message for the second area,
Display device. In the control method of the display device,
Receiving a broadcast signal from a broadcasting station;
Displaying content included in the received broadcast signal on a screen;
When not displaying the content, displaying at least one first image in a first area;
When an input for selecting any one of the at least one first image is received from a user, displaying a plurality of second images corresponding to the selected first image in a second area; And
When an input for reproducing the first image is received from a user, displaying the plurality of second images on a screen at predetermined time intervals.
Control method of a display device comprising a.

Images