KR20220038179A - Digital device and digital device control method - Google Patents

Abstract

Disclosed herein are a digital device and a digital device control method. Here, the digital device according to an embodiment of the present invention includes a receiver for receiving content and signaling data for the content, a user input receiver for receiving a first user input for calling a menu, and decoding the content and signaling data A decoder, a control unit that controls to output the decoded content to a screen, and a control unit that overlays a menu screen on a predetermined area on a screen on which the content is being output according to the first user input and outputs the overlaid menu screen, and outputs the content and the menu screen and an output unit, wherein the control unit controls the menu screen to be output, including an application list or a content list including at least one content related to the content being output on the screen, and the pointer within the menu screen is a predetermined area When positioned in or hovered, a GUI for switching the menu screen configuration of the output menu screen is controlled to be output, and the menu screen configuration is controlled to be switched according to a user's selection from the outputted GUI.

Description

DIGITAL DEVICE AND DIGITAL DEVICE CONTROL METHOD

The present invention relates to a digital device, and more particularly, to a digital device and method for maximizing user convenience through more intuitive and quick content access by configuring a content-oriented menu.

A rapid transition from analog systems to digital systems is taking place. Here, the digital system is more resistant to external noise than the conventional analog system, so data loss is small, it is advantageous in error correction, and it can provide an interactive service.

A conventional digital TV receives a control signal through a control means such as a remote controller provided as a pair at the time of manufacture, or key buttons provided on an external frame or front panel of the digital TV, and operates corresponding thereto. was performed and the results were printed. For example, in a conventional digital TV, when a menu is requested through the control means or a key button, a menu screen including a format or content predetermined at the time of manufacture is provided on the screen. In other words, the conventional digital TV outputs only the content preset to the requested menu screen. Therefore, in order to search and use a desired function or content through the digital TV, the user has to go through several depths or use a separate operation or function button, etc. It is inconvenient, and there is a problem in that it is difficult to quickly access and use a desired function or content in such an environment.

In order to solve the above problems, the digital device according to the present invention has a task to enable users to access and use data such as functions and contents more easily and quickly than in the prior art.

The digital device according to the present invention enables easy and quick access and use of desired data through a minimum depth or screen change on a called menu while minimizing interference with the content being output on the main screen, that is, the content currently being viewed. make it a different task.

The digital device according to the present invention easily provides a multi-view screen based on a Web OS platform in a state that main content or/and a menu launcher is being output, and Getting him under control is another task.

Another task of the digital device according to the present invention is to allow anyone to use the digital device easily and conveniently by configuring and providing a menu screen that is more intuitive and maximized usability compared to the prior art.

The technical problem to be achieved in the present invention is not limited to the above-described technical problem(s), and other technical problems not mentioned are clear to those of ordinary skill in the art to which the present invention belongs from the content disclosed in this specification can be understood clearly.

In order to solve the above problem, a digital device according to an embodiment of the present invention includes a receiving unit for receiving content and signaling data for the content, a user input receiving unit for receiving a first user input requesting multi-view content output, A decoder that decodes the content and signaling data, controls to output the decoded content to the screen, and overlays a window for outputting the multi-view content to a predetermined area on the screen where the content is being output according to the first user input and an output unit for outputting the content and the window, wherein the control unit controls to output an interface for outputting one or more sub-windows when the pointer is positioned within the window, and to select the interface Accordingly, the one or more sub-windows are controlled to be output together with the window in a screen area.

A method for processing multi-view data in a digital device according to an embodiment of the present invention includes receiving content and signaling data for the content, decoding the content and signaling data, and displaying the decoded content on a screen The step of outputting, receiving a first user input requesting output of the multi-view content, and overlaying a window for outputting the multi-view content on a predetermined area on the screen on which the content is being output according to the first user input and outputting the output step, wherein the output window includes an interface for outputting one or more sub-windows, and outputting the one or more sub-windows together with the window to a screen area according to selection of the interface .

Solution(s) disclosed through this specification are not limited by the above, and other solutions not mentioned are clear to those of ordinary skill in the art to which the present invention belongs from the content disclosed in this specification can be understood clearly.

According to the present invention, there are the following effects.

According to one of the various embodiments of the present invention, a digital device has an effect of enabling a user to access and use data such as a desired function and content more easily and quickly than in the related art.

According to another embodiment of various embodiments of the present invention, the digital device provides desired data through the minimum depth or screen change on the called menu while minimizing the interruption to the content being output on the main screen, that is, the content currently being viewed. It has the effect of making it easy and quick to access and use.

According to another embodiment of the present invention, the digital device provides a multi-view screen easily and controls the webOS platform while the main content and/or the menu launcher is being output. It works.

According to another embodiment of various embodiments of the present invention, the digital device has an effect that anyone can use the digital device easily and conveniently by configuring and providing a menu screen that is more intuitive and maximized usability compared to the prior art.

The effect of the present invention is not limited to the above-described effect(s), and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the content disclosed in this specification. .

1 is a diagram schematically illustrating a service system including a digital device according to an embodiment of the present invention;
2 is a block diagram illustrating a digital device according to an embodiment of the present invention;
3 is a block diagram illustrating a digital device according to another embodiment of the present invention;
4 is a diagram illustrating a WebOS architecture according to an embodiment of the present invention;
5 is a diagram illustrating an architecture of a webOS device according to an embodiment of the present invention;
6 is a diagram illustrating a graphic composition flow in a webOS device according to an embodiment of the present invention;
7 is a view for explaining a media server according to an embodiment of the present invention;
8 is a view for explaining the constituent blocks of a media server according to an embodiment of the present invention;
9 is a diagram illustrating a relationship between a media server and a TV service according to an embodiment of the present invention;
10 is a diagram illustrating a screen of a digital device outputting content;
11 is a diagram illustrating a digital device for outputting a requested menu to which a menu is output according to an embodiment of the present invention;
12 to 21 are diagrams for explaining the current mode menu configuration and control according to the present invention;
22 to 27 are diagrams for explaining the configuration and control of the past mode menu according to the present invention;
28 to 35 are diagrams for explaining the future mode menu configuration and control according to the present invention;
36 is a diagram for explaining an example of a digital service system according to an embodiment of the present invention;
37 is a diagram for explaining an embodiment of outputting a menu requested by the digital device 3710 from the mobile device according to the present invention;
38 is a view for explaining an embodiment of the configuration of a menu launcher in the form of a mobile device according to the present invention;
39 is a flowchart illustrating a menu data processing method in a digital device according to the present invention;
Fig. 40 is a diagram for explaining a sound control method when a screen is divided in relation to multi-view mode data processing;
41 and 42 are diagrams for explaining a method for entering a multi-view mode according to the present invention;
43 is a diagram illustrating a multi-view entry screen according to an embodiment of the present invention;
44 is a view for explaining a multi-view window control method according to another embodiment of the present invention;
45 is a diagram illustrating a detailed control method of a multi-view window according to an embodiment of the present invention;
46 and 47 are diagrams for explaining a detailed reproduction control method of a multi-view window according to an embodiment of the present invention;
48 is a view for explaining a method for ending a multi-view mode according to the present invention;
49 to 58 are diagrams for explaining a sub-multi-view operation control method according to the present invention; and
59 is a flowchart illustrating a method of controlling a digital device supporting a multi-view mode according to an embodiment of the present invention.

Hereinafter, various embodiment(s) of a digital device and a method for controlling the digital device according to the present invention will be described in detail with reference to the drawing(s).

The suffixes "module", "part", etc. for components used in this specification are given only in consideration of the ease of writing the specification, and both may be used interchangeably as needed. In addition, even when described as an ordinal number such as "first-", "second-", etc., it is only for convenience of explanation of the term rather than an order, and is not limited to such terms or ordinal numbers.

In addition, the terms used in the present specification are also selected as widely used general terms as possible in consideration of the function according to the technical idea of the present invention, but this is according to the intention or customs of those of ordinary skill in the art or the emergence of new technologies, etc. may vary. However, in certain cases, there are terms arbitrarily selected by the applicant, but the meaning will be described in the related description. Therefore, it is clear that the term should be interpreted based on the actual meaning it has and the content described throughout this specification, not just the name.

On the other hand, the content described in this specification or / and drawings, as a preferred embodiment according to the present invention is not limited thereto, and the scope of the rights should be determined through the claims.

Hereinafter, the term “digital device” described in this specification is used to include all devices capable of transmitting, receiving, processing, and outputting data, content, service, application, etc., for example. The digital device can pair or connect (hereinafter referred to as 'pairing') to another digital device, an external server, and the like through a wired/wireless network, and transmit/receive predetermined data through it. can receive The predetermined data also includes control data. If necessary, the data may be appropriately converted prior to its transmission/reception. The digital device includes, for example, a network TV (Network TV), a Hybrid Broadcast Broadband TV (HBBTV), a smart TV (Smart TV), an Internet Protocol TV (IPTV), a standing device such as a PC, and a PDA ( A mobile device or handheld device such as a personal digital assistant), a smart phone, a tablet PC, a notebook, and a wearable device may all be included. In the present specification, digital TV and a mobile device are illustrated and described as an embodiment of a digital device in FIGS. 2 and 3, which will be described later for better understanding of the present invention and for convenience of explanation by the applicant. In addition, the digital device described herein may be a digital signage, a monitor, or a display device composed only of a panel, and a set-top box (STB). It may include or be a part of one service system in combination with a server and the like.

Meanwhile, the term “wired/wireless network” described in this specification refers to a communication network that supports various communication standards or protocols for pairing and/or data transmission/reception between digital devices or an external server. These wired/wireless networks include all communication networks to be supported now or in the future according to the standard, and all of one or more communication protocols for it can be supported. Such wired / wireless networks include, for example, Universal Serial Bus (USB), Composite Video Banking Sync (CVBS), Component, S-Video (analog), Digital Visual Interface (DVI), High Definition Multimedia Interface (HDMI), Networks for wired connection such as RGB and D-SUB and communication standards and protocols for them, Bluetooth, RFID (Radio Frequency Identification), infrared data association (IrDA), UWB (Ultra Wideband), Zigbee (ZigBee), DLNA (Digital Living Network Alliance), WLAN (Wireless LAN) (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), LTE/LTE -A (Long Term Evolution/LTE-Advanced) or Wi-Fi Direct may be formed by a network for wireless connection and a communication standard or protocol for the same.

In the present specification, when only a digital device is named, the meaning may mean a fixed or mobile device depending on the context, and unless otherwise specified, it may be used in a meaning including both.

On the other hand, a digital device is an intelligent device that supports, for example, a broadcast reception function, a computer function or support, and at least one external input, and is used for e-mail and web browsing through the above-described wired/wireless network. , banking, games, applications, and the like. In addition, the digital device may include an interface for supporting at least one input or control means such as a handwritten input device, a touch-screen, a spatial remote control, and the like. there is.

In addition, the digital device may use a standardized general-purpose operating system (OS), and in particular, a Linux OS may also be included therein. Meanwhile, the digital device described in this specification processes data, service, application, etc. based on a Web OS (Web OS) platform as an embodiment. Therefore, digital devices can add, delete, modify, update various services or applications on the general-purpose OS kernel or Linux kernel, and configure a more user-friendly environment through it. can be provided.

On the other hand, the above-mentioned digital device may receive and process an external input. In this case, the external input is connected to the external input device, that is, the digital device described above through a wired/wireless network to transmit/receive data. All input means or digital devices that can be processed by For example, as the external input, HDMI (High-Definition Multimedia Interface), a game device such as a playstation or an X-Box, a smart phone, a tablet PC, a pocket photo, etc. It includes all digital devices such as a printing device, a smart TV, and a Blu-ray device.

In addition, the term “server” as used herein refers to a digital device or system that supplies data to or receives data from the above-mentioned digital device, that is, a client, and may also be called a processor. do. As the server, for example, a portal server that provides a web page, web content or web service, an advertisement server that provides advertising data, A content server that provides content, an SNS server that provides SNS (Social Network Service), a service server provided by a manufacturer, and a video on demand (VoD) or streaming service are provided. MVPD (Multichannel Video Programming Distributor) for MVPD, a service server that provides a pay service, and the like, a cloud server, etc. may be included.

In addition, in the case where only the application is described for convenience of description in the present specification, the meaning may include not only the application but also the service based on the context, etc., and the web application on the webOS platform according to the present invention ( web application) can also be included.

A digital device according to an embodiment of the present invention includes a receiver for receiving content and signaling data for the content, a user input receiver for receiving a first user input requesting multi-view content output, and decoding the content and signaling data a decoder that controls to output the decoded content to a screen, and controls to output a window for outputting the multi-view content by overlaying a window outputting the multi-view content on a predetermined area on the screen on which the content is being output according to the first user input, and the an output unit for outputting content and a window, wherein the control unit controls an interface for outputting one or more sub-windows to be output when the pointer is positioned within the window, and according to the selection of the interface, the one or more sub-windows Controls windows to be output together with the window in a screen area.

In the above, the content output through one or more sub-windows may be content related to at least one of the content and multi-view content output through the window. Meanwhile, the user input receiving unit may receive a user input requesting to output a menu launcher, and the control unit may control to output one or more menu items on the screen according to the menu launcher output requesting user input. .

In the above, the first user input may be a multi-view function icon selection signal provided around the selected menu item when any one of one or more menu items belonging to the output menu launcher is selected.

The control unit may control to output a window for outputting the multi-view content to an area that does not overlap with the output menu launcher, and control to output the one or more sub-windows in different sizes compared to the window Also, by dragging and dropping the window, it is possible to control to interchange with the content being provided on the screen. Meanwhile, the controller may control the one or more sub-windows to be output to a screen area adjacent to the window or an edge of the screen, and when the output window is changed from an initial position within the screen area to another position, , it is possible to control to output one or more sub-windows in addition to the sub-windows output according to the selection of the interface based on the changed location, and display at least one of the output window and the sub-windows with content of the same category. It is configured in groups including, and when the pointer is located in the corresponding window or sub-window, it is possible to control to output the group-based content list in the form of a window or an icon on the corresponding window. Referring to the accompanying drawings, The present invention will be described in more detail as follows.

1 is a diagram schematically illustrating a service system including a digital device according to an embodiment of the present invention.

Referring to FIG. 1 , the service system includes a content provider 10 , a service provider 20 , a network provider 30 and a Home Network End User (HNED). (Customer) (40). Here, the HNED 40 includes, for example, the client 100, that is, a digital device according to the present invention.

The content provider 10 produces and provides various types of content. As shown in FIG. 1 , as a content provider 10 , a terrestrial broadcaster, a cable broadcaster (cable SO (System Operator)) or MSO (Multiple SO), a satellite broadcaster, various Internet broadcasters, individuals Content providers and the like may be exemplified. Meanwhile, the content provider 10 may produce and provide various services or applications in addition to broadcast content.

The service provider 20 provides the HNED 40 by service packetizing the content produced by the content provider 10 . For example, the service provider 20 packages for a service at least one of contents produced by first terrestrial broadcasting, second terrestrial broadcasting, cable MSO, satellite broadcasting, various Internet broadcasting, applications, and the like, and the HNED (40) is provided.

The service provider 20 provides a service to the client 100 in a uni-cast or multi-cast method. Meanwhile, the service provider 20 may transmit data to a plurality of pre-registered clients 100 at once. For this purpose, an Internet Group Management Protocol (IGMP) protocol or the like may be used.

The above-described content provider 10 and service provider 20 may be the same entity. For example, by packaging the content produced by the content provider 10 and providing it to the HNED 40 , the function of the service provider 20 may also be performed or vice versa.

The network provider 30 provides a network for exchanging data between the content provider 10 or/and the service provider 20 and the client 100 .

The client 100, as a consumer belonging to the HNED 40, receives data by, for example, building a home network through the network provider 30, and receives data related to various services or applications such as VoD and streaming. can also be sent/received.

Meanwhile, the content provider 10 and/or the service provider 20 in the service system may use conditional access or content protection means to protect transmitted content. Accordingly, the client 100 may use a processing means such as a CableCARD (or POD: Point of Deployment), DCAS (Downloadable CAS), etc. in response to the conditional access or content protection.

In addition, the client 100 may also use an interactive service through a network. Accordingly, the client 100 may rather perform the role or function of the content provider, and the service provider 20 may receive it and transmit it to another client or the like.

In FIG. 1 , the content provider 10 and/or the service provider 20 may be a server providing a service to be described later in this specification. In this case, the server may mean owning or including the network provider 30 as necessary. Hereinafter, even if not specifically mentioned, the service or service data includes an internal service or application as well as the service or application received from the outside described above, and these services or applications are webOS (Web OS)-based services for the client 100 . or application data.

2 is a block diagram illustrating a digital device according to an embodiment of the present invention.

Hereinafter, the digital device described herein corresponds to the client 100 of FIG. 1 described above.

The digital device 200 includes a network interface 201 , a TCP/IP manager 202 , a service delivery manager 203 , an SI decoder 204 , Demultiplexer (demux or demultiplexer) 205, audio decoder (audio decoder) 206, video decoder (video decoder) 207, display unit (display A/V and OSD module) 208, service control manager (service control manager) 209, service discovery manager (210), SI & metadata database (SI&metadata DB) (211), metadata manager (metadata manager) (212), service manager (213), It is configured to include a UI manager 214 and the like.

The network interface unit 201 is configured to access IP packet(s) (Internet Protocol (IP) packet(s)) or IP datagram(s) (IP datagram(s)) (hereinafter referred to as ' IP packet(s)') are transmitted/received. For example, the network interface unit 201 may receive a service, application, content, additional information, etc. from the service provider 20 of FIG. 1 through a network network. The additional information may include SI (System Information) information. Meanwhile, the network interface unit 201 may coexist with a tuner, a demodulator, or the like, or may be replaced by the tuner or demodulator. The network interface unit 201 may include, for example, an Ethernet terminal for connection with a wired network, and for connection with a wireless network, for example, a WLAN (Wireless LAN) (Wi- Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) communication standards, etc. can be used. Meanwhile, the network interface unit 201 may access a predetermined web page through the connected network or another network linked to the connected network. That is, by accessing a predetermined web page through a network, it is possible to transmit or receive data with a corresponding server. In addition, content or data provided by a content provider or network operator may be received. That is, content such as movies, advertisements, games, VODs, broadcast signals, etc. and information related thereto may be received from a content provider or a network provider through a network. In addition, it is possible to receive firmware update information and an update file provided by a network operator. It may also transmit data to the Internet or to a content provider or network operator. Also, the network interface unit 201 may select and receive a desired application from among applications opened through a network.

The TCP/IP manager 202, for IP packets received to the digital device 200 and IP packets transmitted from the digital device 200, that is, packet transfer between a source (source) and a destination (destination) ( involved in packet delivery). The TCP/IP manager 202 classifies the received packet(s) to correspond to an appropriate protocol, and divides the classified packet(s) into a service delivery manager 205, a service discovery manager 210, and a service control manager ( 209), the metadata manager 212, and the like.

The service delivery manager 203 is in charge of controlling service data. For example, the service delivery manager 203 may use RTP/RTCP when controlling real-time streaming data. When the real-time streaming data is transmitted using RTP, the service delivery manager 203 parses the received data packet according to the RTP and transmits it to the demultiplexer 205 or the control of the service manager 213 according to the SI&metadata database 211. Meanwhile, the service delivery manager 203 feeds back the network reception information to the server providing the service using RTCP.

The demultiplexer 205 demultiplexes the received packet into audio, video, SI data, and the like, and transmits it to the audio/video decoders 206/207 and SI decoder 204, respectively.

The SI decoder 204 decodes demultiplexed SI data. The SI data may include Program Specific Information (PSI), Program and System Information Protocol (PSIP), Digital Video Broadcasting-Service Information (DVB-SI), Digital Television Terrestrial Multimedia Broadcasting/Coding Mobile Multimedia Broadcasting (DTMB/CMMB), etc. It can be any one. Also, the SI decoder 204 may store the decoded SI data in the SI&metadata database 211 . SI data stored in the SI&database 211 may be read and used according to a user's request, for example.

The audio/video decoder 206/207 decodes each demultiplexed audio data and video data. The decoded audio data and video data are provided to the user through the display unit 208 .

The application manager includes, for example, a UI manager 214 and a service manager 213 and may perform a control function of the digital device 200 . In other words, the application manager may manage the overall state of the digital device 200 , provide a user interface (UI), and manage other managers.

The UI manager 214 provides a graphical user interface (GUI)/UI for a user using an on screen display (OSD), etc., receives a key input from the user, and performs device operations according to the input. For example, the UI manager 214 transmits the key input signal to the service manager 213 upon receiving a key input for channel selection from the user.

The service manager 213 controls a manager associated with a service, such as the service delivery manager 203 , the service discovery manager 210 , the service control manager 209 , and the metadata manager 212 .

In addition, the service manager 213 generates a channel map and controls selection of a channel using the generated channel map according to a key input received from the UI manager 214 . The service manager 213 receives service information from the SI decoder 204 and sets an audio/video PID (Packet Identifier) of the selected channel in the demultiplexer 205 . The PID set in this way can be used in the above-described demultiplexing process. Accordingly, the demultiplexer 205 filters audio data, video data, and SI data (PID or section filtering) using the PID.

The service discovery manager 210 provides information necessary for selecting a service provider that provides a service. When receiving a channel selection signal from the service manager 213, the service discovery manager 210 uses the information to find a service.

The service control manager 209 is in charge of service selection and control. For example, the service control manager 209 uses IGMP or RTSP when the user selects a live broadcasting service such as the existing broadcasting method, and selects a service such as VOD (Video on Demand). RTSP is used to select and control services. The RTSP protocol may provide a trick mode for real-time streaming. Also, the service control manager 209 may initialize and manage a session through the IMS gateway 250 using an IP Multimedia Subsystem (IMS) or a Session Initiation Protocol (SIP). The above protocols are only an example, and other protocols may be used according to implementation examples.

The metadata manager 212 manages metadata associated with a service and stores the metadata in the SI&metadata database 211 .

The SI&metadata database 211 includes service information decoded by the SI decoder 204 , metadata managed by the metadata manager 212 , and information required to select a service provider provided by the service discovery manager 210 . Save. In addition, the SI&metadata database 211 may store set-up data for the system, and the like.

The SI&metadata database 211 may be implemented using a non-volatile memory (NVRAM), a flash memory, or the like.

On the other hand, the IMS gateway 250 is a gateway that collects functions necessary to access an IMS-based IPTV service.

The storage unit (not shown) may store a program for each signal processing and control, or may store a signal-processed image, audio, or data signal. Also, the storage unit may perform a function for temporarily storing an image, audio, or data signal input from the external device interface unit or the network interface unit 201 . The storage unit may store information about a predetermined broadcast channel through a channel storage function. The storage unit may store an application or an application list input from the external device interface unit or the network interface unit 201 . Also, the storage unit may store webOS platform data, which will be described later. The storage unit includes, for example, a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg SD or XD memory, etc.); It may include at least one type of storage medium among RAM (RAM) and ROM (EEPROM, etc.). The digital device 200 may reproduce content files (movie files, still image files, music files, document files, application files, etc.) stored in the storage unit and provide them to the user.

The above-described digital device 200 may be a digital broadcast receiver capable of processing a digital broadcast signal of a fixed or mobile ATSC method or a DVB method. In addition, the digital device according to the present invention may omit some of the illustrated components as needed, or may further include non-illustrated components. Meanwhile, unlike the above, the digital device may receive and reproduce content through a network interface unit or an external device interface unit without a tuner and a demodulator.

3 is a block diagram illustrating a digital device according to another embodiment of the present invention.

In FIG. 2, a digital TV, which is a fixed device, has been described as an example of a digital device, and in FIG. 3, a mobile device will be described as another example of a digital device.

Referring to FIG. 3 , the mobile device 300 includes a wireless communication unit 310 , an audio/video (A/V) input unit 320 , a user input unit 330 , a sensing unit 340 , an output unit 350 , It may include a memory 360 , an interface unit 370 , a control unit 380 , and a power supply unit 390 .

Hereinafter, each component will be described in detail as follows.

The wireless communication unit 310 may include one or more modules that enable wireless communication between the mobile device 300 and a wireless communication system or between the mobile device and a network in which the mobile device is located. For example, the wireless communication unit 310 may include a broadcast reception module 311 , a mobile communication module 312 , a wireless Internet module 313 , a short-range communication module 314 , and a location information module 315 , etc. .

The broadcast reception module 311 receives a broadcast signal and/or broadcast related information from an external broadcast management server through a broadcast channel. Here, the broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may mean a server that generates and transmits a broadcast signal and/or broadcast-related information or a server that receives a previously generated broadcast signal and/or broadcast-related information and transmits the received broadcast signal and/or broadcast-related information to the terminal. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, as well as a TV broadcast signal or a broadcast signal in which a data broadcast signal is combined with a radio broadcast signal. The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 312 . Broadcast-related information may exist in various forms, for example, in the form of an Electronic Program Guide (EPG) or an Electronic Service Guide (ESG). The broadcast reception module 311 is, for example, ATSC, Digital Video Broadcasting-Terrestrial (DVB-T), DVB-S (Satellite), MediaFLO (Media Forward Link Only), DVB-H (Handheld), ISDB-T ( A digital broadcast signal can be received using a digital broadcast system such as Integrated Services Digital Broadcast-Terrestrial). Of course, the broadcast receiving module 311 may be configured to be suitable for other broadcasting systems as well as the above-described digital broadcasting system. The broadcast signal and/or broadcast related information received through the broadcast reception module 311 may be stored in the memory 360 .

The mobile communication module 312 transmits/receives wireless signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission/reception of a voice signal, a video call signal, or a text/multimedia message.

The wireless Internet module 313 may be built-in or external to the mobile device 300 , including a module for wireless Internet access. As wireless Internet technologies, wireless LAN (WLAN) (Wi-Fi), wireless broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), etc. may be used.

The short-range communication module 314 refers to a module for short-range communication. Short range communication technology such as Bluetooth, RFID (Radio Frequency Identification), infrared (IrDA), Ultra Wideband (UWB), ZigBee, RS-232, RS-485, etc. This can be used.

The location information module 315 is a module for obtaining location information of the mobile device 300 , and may be a global positioning system (GPS) module as an example.

The A/V input unit 320 is for inputting audio and/or video signals, and may include a camera 321 , a microphone 322 , and the like. The camera 321 processes an image frame such as a still image or a moving image obtained by an image sensor in a video call mode or a shooting mode. The processed image frame may be displayed on the display unit 351 .

The image frame processed by the camera 321 may be stored in the memory 360 or transmitted to the outside through the wireless communication unit 310 . Two or more cameras 321 may be provided according to the use environment.

The microphone 322 receives an external sound signal by a microphone in a call mode, a recording mode, a voice recognition mode, and the like, and processes it as electrical voice data. The processed voice data may be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 312 in the call mode and output. Various noise removal algorithms for removing noise generated in the process of receiving an external sound signal may be implemented in the microphone 322 .

The user input unit 330 generates input data for the user to control the operation of the terminal. The user input unit 330 may include a key pad, a dome switch, a touch pad (static pressure/capacitance), a jog wheel, a jog switch, and the like.

The sensing unit 340 detects the current state of the mobile device 300, such as the opening/closing state of the mobile device 300, the location of the mobile device 300, the presence or absence of user contact, the orientation of the mobile device, and acceleration/deceleration of the mobile device. It detects and generates a sensing signal for controlling the operation of the mobile device 300 . For example, when the mobile device 300 is moved or tilted, the position or tilt of the mobile device may be sensed. In addition, whether the power supply unit 390 is supplied with power, whether the interface unit 370 is coupled to an external device may also be sensed. Meanwhile, the sensing unit 340 may include a proximity sensor 341 including near field communication (NFC) or the like.

The output unit 350 is for generating an output related to visual, auditory or tactile sense, and may include a display unit 351 , a sound output module 352 , an alarm unit 353 , and a haptic module 354 , etc. there is.

The display unit 351 displays (outputs) information processed by the mobile device 300 . For example, when the mobile device is in a call mode, a UI or GUI related to a call is displayed. When the mobile device 300 is in the video call mode or the photographing mode, the photographed and/or received image, UI, and GUI are displayed.

The display unit 351 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display ( flexible display) and at least one of a three-dimensional display.

Some of these displays may be configured as a transparent type or a light-transmitting type so that the outside can be viewed through them. This may be referred to as a transparent display, and a representative example of the transparent display is a TOLED (Transparant OLED). The rear structure of the display unit 351 may also be configured as a light transmission type structure. With this structure, the user can see an object located at the rear of the terminal body through the area occupied by the display unit 351 of the terminal body.

Two or more display units 351 may exist according to an implementation form of the mobile device 300 . For example, in the mobile device 300 , a plurality of display units may be spaced apart or disposed integrally on one surface, or may be respectively disposed on different surfaces.

When the display unit 351 and the sensor for sensing a touch operation (hereinafter referred to as a 'touch sensor') form a layer structure (hereinafter referred to as a 'touch screen'), the display unit 351 is an input device other than an output device. It can also be used as a device. The touch sensor may have the form of, for example, a touch film, a touch sheet, and a touch pad.

The touch sensor may be configured to convert a change in pressure applied to a specific part of the display unit 351 or a change in capacitance occurring in a specific part of the display unit 351 into an electrical input signal. The touch sensor may be configured to detect not only the touched position and area, but also the pressure at the time of the touch.

When there is a touch input to the touch sensor, a corresponding signal(s) is sent to the touch controller. The touch controller processes the signal(s) and then transmits corresponding data to the controller 380 . Accordingly, the control unit 380 can know which area of the display unit 351 has been touched, and the like.

The proximity sensor 341 may be disposed in an inner region of the mobile device covered by the touch screen or in the vicinity of the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object existing in the vicinity without mechanical contact using the force of an electromagnetic field or infrared rays. Proximity sensors have a longer lifespan than contact sensors and their utility is also high.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, and an infrared proximity sensor. When the touch screen is of a capacitive type, it is configured to detect the proximity of the pointer by a change in an electric field according to the proximity of the pointer. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of description, the act of bringing the pointer closer to the touch screen without making contact so that the pointer is recognized as being positioned on the touch screen is referred to as a “proximity touch”, and the touch The act of actually touching the pointer on the screen is called "contact touch". The position at which a proximity touch is performed with the pointer on the touch screen means a position at which the pointer vertically corresponds to the touch screen when the pointer is touched in proximity.

The proximity sensor detects a proximity touch and a proximity touch pattern (eg, proximity touch distance, proximity touch direction, proximity touch speed, proximity touch time, proximity touch position, proximity touch movement state, etc.). Information corresponding to the sensed proximity touch operation and the proximity touch pattern may be output on the touch screen.

The sound output module 352 may output audio data received from the wireless communication unit 310 or stored in the memory 360 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output module 352 also outputs a sound signal related to a function (eg, a call signal reception sound, a message reception sound, etc.) performed by the mobile device 300 . The sound output module 352 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 353 outputs a signal for notifying the occurrence of an event in the mobile device 300 . Examples of events generated in the mobile device include call signal reception, message reception, key signal input, and touch input. The alarm unit 353 may output a signal for notifying the occurrence of an event in a form other than a video signal or an audio signal, for example, vibration. The video signal or audio signal may also be output through the display unit 351 or the audio output module 352 , so that they 351 and 352 may be classified as a part of the alarm unit 353 .

The haptic module 354 generates various tactile effects that the user can feel. A representative example of the tactile effect generated by the haptic module 354 is vibration. The intensity and pattern of vibration generated by the haptic module 354 are controllable. For example, different vibrations may be synthesized and outputted or output sequentially. In addition to vibration, the haptic module 354 is configured to respond to stimuli such as a pin arrangement that moves vertically with respect to the contact skin surface, a jet force or suction force of air through a nozzle or an inlet, a brush against the skin surface, contact of an electrode, electrostatic force, etc. Various tactile effects can be generated, such as an effect caused by heat absorption and an effect by reproducing a feeling of cool/warm using an element capable of absorbing heat or generating heat. The haptic module 354 may not only transmit the tactile effect through direct contact, but may also be implemented so that the user can feel the tactile effect through a muscle sense of a finger or arm. Two or more haptic modules 354 may be provided according to a configuration aspect of the mobile device 300 .

The memory 360 may store a program for the operation of the controller 380 , and may temporarily store input/output data (eg, a phone book, a message, a still image, a moving image, etc.). The memory 360 may store data related to vibrations and sounds of various patterns output when a touch input on the touch screen is input.

Memory 360 is a flash memory type (flash memory type), hard disk type (hard disk type), multimedia card micro type (multimedia card micro type), card type memory (for example, SD or XD memory, etc.), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM), Magnetic Memory, It may include at least one type of storage medium among a magnetic disk and an optical disk. The mobile device 300 may operate in relation to a web storage that performs a storage function of the memory 360 on the Internet.

The interface unit 370 serves as a passage with all external devices connected to the mobile device 300 . The interface unit 370 receives data from an external device, receives power and transmits it to each component inside the mobile device 300 , or allows data inside the mobile device 300 to be transmitted to an external device. For example, a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, a port for connecting a device equipped with an identification module, an audio input/output (I/O) port, A video I/O port, an earphone port, etc. may be included in the interface unit 370 .

The identification module is a chip storing various information for authenticating the usage right of the mobile device 300 , and includes a User Identify Module (UIM), a Subscriber Identify Module (SIM), a universal user authentication module ( Universal Subscriber Identity Module (USIM), and the like. A device equipped with an identification module (hereinafter, 'identification device') may be manufactured in the form of a smart card. Accordingly, the identification device may be connected to the terminal through the port.

The interface unit 370, when the mobile device 300 is connected to an external cradle, is a passage through which power from the cradle is supplied to the mobile device 300, or various kinds of input from the cradle by a user. It may be a path through which a command signal is transmitted to the mobile device. Various command signals or the power input from the cradle may be operated as signals for recognizing that the mobile device is correctly mounted on the cradle.

The controller 380 generally controls the overall operation of the mobile device 300 . The controller 380 performs related control and processing for, for example, a voice call, data communication, video call, and the like. The controller 380 may include a multimedia module 381 for playing multimedia. The multimedia module 381 may be implemented within the control unit 380 or may be implemented separately from the control unit 380 . The controller 380 may perform a pattern recognition process capable of recognizing a handwriting input or a drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 390 receives external power and internal power under the control of the control unit 380 to supply power required for operation of each component.

Various embodiments described herein may be implemented in a computer-readable recording medium using, for example, software, hardware, or a combination thereof.

According to the hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), It may be implemented using at least one of a processor, a controller, micro-controllers, microprocessors, and other electrical units for performing functions. Examples may be implemented by the control unit 380 itself.

According to the software implementation, embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein. Software code may be implemented as a software application written in an appropriate programming language. Here, the software code may be stored in the memory 360 and executed by the controller 380 .

On the other hand, the mobile terminal can be extended to a wearable device that can be worn on the body beyond the dimension that the user mainly holds in the hand. Such wearable devices include smart watches, smart glasses, and head mounted displays (HMDs). Hereinafter, examples of mobile terminals extended to wearable devices will be described.

The wearable device may be configured to be able to exchange (or interwork) data with another mobile terminal 300 . The short-range communication module 314 may detect (or recognize) a communicable wearable device in the vicinity of the mobile terminal 300 . Furthermore, when the detected wearable device is a device authenticated to communicate with the mobile terminal 300 , the controller 380 transmits at least a portion of data processed by the mobile terminal 300 to the wearable device through the short-range communication module 314 . can be sent to Accordingly, the user may use data processed by the mobile terminal 300 through the wearable device. For example, when a call is received in the mobile terminal 300, a phone call is performed through the wearable device, or when a message is received in the mobile terminal 300, it is possible to check the received message through the wearable device .

The digital device described in this specification may be operated by a webOS platform. Hereinafter, webOS-based configuration or processing processes such as algorithms may be performed in the control unit of the digital device, etc. described above. Here, the control unit is used in a broad sense including the above-described control unit. Therefore, hereinafter, for processing of webOS-based or related services, applications, and contents in digital devices, hardware or components including related software and firmware are referred to as controllers. It can be named and explained.

This webOS-based platform is, for example, to improve development independence and function scalability by integrating services and applications based on the Luna-service bus, and application development based on the web application framework. It can also increase productivity. In addition, it is possible to support multi-tasking by efficiently utilizing system resources and the like through webOS processes and resource management.

On the other hand, the webOS platform described in this specification can be used not only in fixed devices such as PCs, TVs, and set-top boxes (STBs), but also in mobile devices such as mobile phones, smart phones, tablet PCs, notebooks, and wearable devices. .

The software structure for digital devices is a monolithic structure that is dependent on conventional problem solving and the market. A single process and a closed product based on multi-threading technology. As a result, there were difficulties in external application, and after that, we pursued a new platform-based development, and pursued cost innovation through chip-set replacement and efficiency in UI application and external application development, leading to layering & componentization (layering & componentization). ) was formed to have a 3-layer structure and an Add-on structure for add-on, single source product, and open application. In recent years, the software structure has further developed a modular architecture of functional units, a Web Open API (Application Programming Interface) provision for an eco-system, and a game engine for a game engine. A modular design for a native open API, etc. is being made, and accordingly, a multi-process structure based on a service structure is being created.

4 is a diagram illustrating a webOS architecture according to an embodiment of the present invention.

Referring to FIG. 4 , the architecture of the webOS platform will be described as follows.

The platform may be largely divided into a kernel, a system library-based WebOS core platform, an application, a service, and the like.

The architecture of the webOS platform is a layered structure, with the OS at the lowest layer, system library(s) at the next layer, and applications at the top. First, the lowest layer may include Linux as an OS of the digital device by including a Linux kernel as an OS layer. Above the OS layer, BSP (Board Support Package) / HAL (Hardware Abstraction Layer) layer, WebOS core module layer (Web OS core modules layer), service layer (service layer), Luna-service bus layer (Luna-Service) Bus layer), Anyo framework/NDK (Native Developer's Kit)/QT layer (Enyo framework/NDK/QT layer), and an application layer at the top layer sequentially exist. Meanwhile, some layers of the above-described webOS layer structure may be omitted, and a plurality of layers may be formed into one layer or, conversely, one layer may become a plurality of layer structures. The webOS core module layer is based on LSM (Luna Surface Manager) that manages a surface window, etc., SAM (System & Application Manager) that manages the execution and execution status of applications, and WebKit (WebKit) Thus, it may include a Web Application Manager (WAM) that manages web applications and the like.

The LSM manages an application window displayed on the screen. The LSM manages display hardware, provides a buffer capable of rendering necessary content to applications, and synthesizes results rendered by a plurality of applications to display on the screen. can be printed out.

The SAM manages execution policies for various conditions of systems and applications.

On the other hand, WAM is based on the Enyo Framework, as webOS can view a web application as a basic application.

The service use of the application is made through the Luna-service bus, a new service can be registered on the bus, and the application can find and use the service it needs.

The service layer may include services of various service levels, such as a TV service and a WebOS service. Meanwhile, the webOS service may include a media server, Node.JS, and the like. In particular, the Node.JS service supports, for example, JavaScript.

A webOS service can communicate through a bus to a Linux process that implements function logic. This can be largely divided into four parts, the TV process and services that are migrated from the existing TV to the webOS or are manufacturer-differentiated services, the common webOS service and JavaScript developed and used through Node.js It consists of Node.js services.

The application layer may include all applications supported by the digital device, such as a TV application, a showcase application, a native application, and a web application.

An application on the webOS may be classified into a web application, a Palm Development Kit (PDK) application, a Qt Meta Language or Qt Modeling Language (QML) application, and the like according to an implementation method. The web application is based on the WebKit engine, and is executed on the WAM runtime. Such a web application may be developed and executed based on the Anyo framework, or based on general HTML5, Cascading Style Sheets (CSS), or JavaScript. The PDK application includes, for example, a native application developed in C/C++ based on a PDK provided for a third-party or external developer. The PDK refers to a set of development libraries and tools provided so that a third party such as a game can develop a native application (C/C++). For example, a PDK application may be used for development of an application whose performance is important. The QML application is a Qt-based native application, and includes a basic application provided with the webOS platform, such as a card view, a home dashboard, and a virtual keyboard. Here, QML is a mark-up language in the form of a script instead of C++. Meanwhile, in the above, the native application refers to an application developed in C/C++, compiled, and executed in a binary form, and such a native application has an advantage in that the execution speed is fast.

5 is a diagram illustrating an architecture of a webOS device according to an embodiment of the present invention.

5 is a block diagram based on a runtime of a webOS device, which can be understood with reference to the layered structure of FIG. 4 .

Hereinafter, it will be described with reference to FIGS. 4 and 5 . Referring to FIG. 5 , services, applications, and webOS core modules are included on the system OS (Linux) and system libraries, and communication between them may be made through a Luna-service bus.

Node.js services based on HTML5, CSS, java script, such as e-mail, contact, calendar, Logging, backup, file Notify notify), database (DB), activity manager (activity manager), system policy (system policy), audio daemon (AudioD: Audio Daemon), update (update), webOS services such as media server (media server), EPG TV services such as (Electronic Program Guide), PVR (Personal Video Recorder), data broadcasting, voice recognition, Now on, Notification, search CP services such as , ACR (Auto Content Recognition), CBOX (Contents List Broswer), wfdd, DMR, Remote Application, download, SDPIF (Sony Philips Digital Interface Format), PDK applications, browser Native applications such as , QML applications, and UI-related TV applications and web applications based on the Anyo framework are processed through webOS core modules such as SAM, WAM, and LSM as described above through the Luna-service bus. Meanwhile, in the above, TV applications and web applications may not necessarily be Anyo framework-based or UI related.

CBOX can manage a list of contents and metadata of external devices such as USB, DLNA, and cloud connected to the TV. On the other hand, the CBOX may output the content listing of various content containers such as USB, DMS, DVR, cloud, etc. as an integrated view. In addition, the CBOX may display various types of content listings, such as pictures, music, and videos, and manage the metadata. In addition, the CBOX may output the contents of the attached storage device in real-time. For example, the CBOX should be able to immediately output a content list of the storage device when a storage device such as a USB is plugged-in. In this case, a standardized method for processing the content listing may be defined. In addition, CBOX can accommodate various connection protocols.

SAM is intended to improve module complexity and enhance scalability. For example, the existing system manager handles multiple functions such as system UI, window management, web application runtime, and UX constraint processing in one process, and the implementation complexity is high, so to solve the problem, major functions are separated and functions It lowers the implementation complexity by clarifying the interface between them.

LSM supports independent development and integration of system UX implementations such as card view and launcher, and supports easy response to changes in product requirements. On the other hand, LSM enables multi-tasking by maximizing hardware resources (HW resource) when synthesizing a plurality of application screens such as App on App. A window management mechanism for 21:9 or the like may be provided. LSM supports the implementation of system UI based on QML and improves the development productivity. Based on MVC, QML UX can easily compose views of screen layouts and UI components, and can easily develop codes for handling user input. On the other hand, the interface between QML and the webOS component is made through the QML extension plug-in, and the graphic operation of the application can be based on wayland protocol, luna-service call, etc. there is. As described above, LSM is an abbreviation of Luna Surface Manager, and functions as an application window compositor. LSM composes and outputs independently developed applications and UI components on the screen. In relation to this, when components such as a Recents application, a showcase application, and a launcher application each render their own content, the LSM defines an output area, an interworking method, etc. as a compositor. In other words, the LSM, which is a compositor, processes graphic composition, focus management, input events, and the like. In this case, the LSM receives an event, focus, etc. from an input manager, and the input manager may include a remote controller, HID such as a mouse & keyboard, a joystick, a game pad, an application remote, and a pen touch. As such, the LSM supports a multiple window model, and it can be executed simultaneously in all applications due to the nature of the system UI. In this regard, launcher, request, setting, notification, system keyboard, volume UI, search, finger gesture, Voice Recognition (STT (Speech to Text), TTS (Text to) Speech), NLP (Natural Language Processing), etc.), pattern gestures (camera, MRCU (Mobile Radio Control Unit)), Live menu, ACR (Auto Content Recognition), etc. can be supported by LSM .

6 is a diagram illustrating a graphic composition flow in a webOS device according to an embodiment of the present invention.

Referring to FIG. 6 , the graphic composition processing includes a web application manager 610 in charge of a UI process, a web kit 620 in charge of a web process, an LSM 630 and a graphic manager (GM: Graphic Manager) It can be done through (640).

When web application-based graphic data (or application) is generated as a UI process in the web application manager 610 , if the generated graphic data is not a full-screen application, it is transmitted to the LSM 630 . On the other hand, the web application manager 610 receives the application generated in the web kit 620 for sharing a GPU (Graphic Processing Unit) memory for graphic management between the UI process and the web process, and displays it in full-screen as described above. If it is not an application, it is transferred to the LSM 630 . In the case of a full-screen application, the LSM 630 may be bypassed, and in this case, it may be directly transferred to the graphic manager 640 .

The LSM 630 transmits the received UI application to the Wayland Compositor through the Wayland surface, processes it appropriately in the Wayland Compositor, and delivers it to the graphic manager. In this way, the graphic data transmitted from the LSM 630 is transmitted to the graphic manager compositor through the LSM GM surface of the graphic manager 640, for example.

On the other hand, as described above, the full-screen application is directly transferred to the graphic manager 640 without going through the LSM 630, and this application is processed in the graphic manager compositor through the WAM GM surface. The graphic manager processes all graphic data in the webOS device. In addition to the data passed through the LSM GM surface and the data passed through the WAM GM surface, GM surface such as data broadcasting application, caption application, etc. It receives all the graphic data that have passed through the , and processes it so that it is properly output on the screen. Here, the function of the GM compositor is the same as or similar to that of the aforementioned compositor.

7 is a diagram illustrating a media server according to an embodiment of the present invention, FIG. 8 is a diagram illustrating a configuration block diagram of a media server according to an embodiment of the present invention, and FIG. 9 is a diagram illustrating a relationship between a media server and a TV service according to an embodiment of the present invention.

Referring to FIG. 7 , the media server supports execution of various multimedia in a digital device and manages necessary resources. The media server may efficiently use hardware resources required for media play. For example, the media server requires audio/video hardware resources to execute multimedia, and can efficiently utilize the resource by managing the resource usage status. In general, a fixed device having a larger screen than a mobile device requires more hardware resources to execute multimedia, and the encoding/decoding and graphic data transfer speed must also be fast due to a large amount of data. On the other hand, in addition to streaming and file-based playback, the media server performs broadcasting, recording, and tuning tasks, recording simultaneously with viewing, or simultaneously displaying the sender and receiver screens during a video call. You should be able to handle it. However, in the media server, hardware resources such as encoders, decoders, tuners, and display engines are limited in chip-set units, and it is difficult to execute multiple tasks at the same time. input and process it.

The media server can strengthen system stability. This is because, for example, the playback pipeline in which an error occurred during media playback can be removed for each pipeline and restarted, so that the error occurs as described above. Even in this case, other media playback may not be affected. Such a pipeline is a chain that connects unit functions such as decoding, analysis, and output when a media playback request is made, and required unit functions may vary according to a media type and the like.

The media server may have extensibility. For example, a new type of pipeline may be added without affecting an existing implementation method. For example, the media server may accommodate a camera pipeline, a video conferencing (Skype) pipeline, a third-party pipeline, and the like.

The media server may process general media playback and TV task execution as separate services, because the interface of the TV service is different from the media playback case. In the above, the media server supports operations such as 'setchannel', 'channelup', 'channeldown', 'channeltuning', and 'recordstart' in relation to TV service, and 'play' and 'pause' in relation to general media playback By supporting operations such as ' and 'stop', different operations are supported for both, and can be processed as separate services.

The media server may control or integrate the resource management function. Allocating and recovering hardware resources in the device is integrated in the media server, and in particular, the TV service process transmits the currently running task and resource allocation status to the media server. The media server secures resources and executes pipelines whenever each media is executed. Based on the resource status occupied by each pipeline, the media server allows execution by priority (eg, policy) when requesting media execution and Reclaim resources from other pipelines, etc. Here, predefined execution priorities and resource information required for a specific request are managed by a policy manager, and the resource manager may communicate with the policy manager to process resource allocation, retrieval, and the like.

The media server may hold an identifier (ID) for all operations related to playback. For example, the media server may issue a command by instructing a specific pipeline based on the identifier. The media server may separately issue a command to the pipelines to play two or more media. The media server may be in charge of playing HTML5 standard media.

In addition, the media server may follow the TV restructuring scope for separate service processing of the TV pipeline. The media server may be designed and implemented irrespective of the scope of TV restructuring. If the TV is not separately serviced, the entire TV may have to be re-executed when a problem occurs in a specific task.

The media server is also called uMS, that is, a micro media server. Here, a media player is a media client, which is, for example, an HTML5 video tag, a camera, a TV, Skype, a web for a second screen, etc. It may mean a kit (Webkit).

In the media server, management of micro resources such as a resource manager and a policy manager is a core function. In this regard, the media server also controls the playback control role for the web standard media content. In this regard, the media server may also manage a pipeline controller resource.

Such a media server supports, for example, extensibility, reliability, efficient resource usage, and the like.

In other words, uMS, that is, the media server, is a cloud game (cloud game), MVPD (pay service, etc.), camera preview (camera preview), second screen (2nd screen), resources such as skype, webOS devices such as TV resources It functions to manage and control the use of resources for proper processing within the overall management and control so that efficient use is possible. Meanwhile, when each resource is used, for example, a pipeline is used, and the media server may overall manage and control the creation, deletion, use, etc. of the pipeline for resource management. Here, the pipeline means, for example, when the media related to the task starts a series of tasks such as parsing of a request, a decoding stream, a video output, etc. can be For example, in relation to a TV service or application, watching, recording, channel tuning, etc. are each individually controlled and processed on resource use through a pipeline created according to the request. .

Referring to FIG. 8 , the processing structure of the media server will be described in more detail as follows.

In FIG. 8 , an application or service is connected to the media server 820 through the Luna-service bus 810 , and the media server 820 is connected to the pipelines re-created through the Luna-service bus 810 and Connected and managed

An application or service may have various clients according to its characteristics, and may exchange data with the media server 820 or a pipeline through it.

The client includes, for example, a uMedia client (webkit) and a resource manager (RM) client (C/C++) for connection with the media server 820 .

The application including the uMedia client is connected to the media server 820 as described above. More specifically, the uMedia client corresponds to, for example, a video object to be described later, and such a client uses the media server 820 for video operation by request or the like. Here, the video operation relates to a video state, and loading, unloading, play, playback, or reproduce, pause, stop, etc. are all states related to the video operation. It may contain data. Each operation or state of such a video can be processed through individual pipeline creation. Accordingly, the uMedia client transmits the state data related to the video operation to the pipeline manager 822 in the media server.

The pipeline manager 822 obtains information on the resource of the current device through data communication with the resource manager 824 , and requests allocation of a resource corresponding to the state data of the uMedia client. In this case, the pipeline manager 822 or the resource manager 824 controls resource allocation through data communication with the policy manager 826 if necessary in relation to the resource allocation. For example, when there is no or insufficient resources to be allocated according to the request of the pipeline manager 822 in the resource manager 824, according to the priority comparison of the policy manager 826, appropriate resource allocation is performed according to the request. can Meanwhile, the pipeline manager 822 requests the media pipeline controller 828 to create a pipeline for an operation according to the request of the uMedia client with respect to the resource allocated according to the resource allocation of the resource manager 824 . .

The media pipeline controller 828 creates a necessary pipeline under the control of the pipeline manager 822 . As shown in the figure, in the generated pipeline, not only a media pipeline and a camera pipeline, but also pipelines related to playback, pause, pause, and the like may be generated. Meanwhile, the pipeline may include pipelines for HTML5, web CP, smartshare playback, thumbnail extraction, NDK, cinema, Multimedia and Hypermedia Information Coding Experts Group (MHEG), and the like.

In addition, the pipeline may include, for example, a service-based pipeline (own pipeline) and a URI-based pipeline (media pipeline).

Referring to FIG. 8 , an application or service including an RM client may not be directly connected to the media server 820 . This is because applications or services may directly process media. In other words, when an application or service directly processes media, it may not go through a media server. However, at this time, resource management is required for pipeline creation and use, and for this purpose, the uMS connector functions. Meanwhile, when a resource management request for direct media processing of the application or service is received, the uMS connector communicates with the media server 820 including the resource manager 824 . To this end, the media server 820 must also be equipped with a uMS connector.

Accordingly, the application or service may respond to the request of the RM client by receiving the resource management of the resource manager 824 through the uMS connector. These RM clients can handle services such as native CP, TV service, second screen, flash player, YouTube MSE (Media Source Extensions), cloud game, and skype. In this case, as described above, the resource manager 824 may appropriately manage the resource through data communication with the policy manager 826 when necessary for resource management.

On the other hand, the URI-based pipeline is performed through the media server 820 rather than the case of directly processing media like the aforementioned RM client. Such a URI-based pipeline may include a player factory, a Gstreamer, a streaming plug-in, a Digital Rights Management (DRM) plug-in pipeline, and the like.

Meanwhile, an interface method between an application and media services may be as follows.

It is a method to interface using a service in a web application. This is a method of making a Luna call using PSB (Palm Service Bridge) and a method of using Cordova, which extends the display to a video tag. In addition, there may be a method of using the HTML5 standard for video tags or media elements. And, it is a method of interfacing using a service in the PDK. Alternatively, it is a method of using a service in an existing CP. This can be used by extending the plug-in of the existing platform based on Luna for backward compatibility.

Finally, it is an interface method in case of non-webOS. In this case, you can interface by calling the Luna bus directly.

Seamless change is processed by a separate module (eg, TVWIN), which is a process for showing TV on the screen first and seamlessly without webOS before or during booting of webOS am. This is used for the purpose of providing the basic functions of the TV service first in order to quickly respond to the user's power-on request because the booting time of the webOS is slow. In addition, as part of the TV service process, the module supports seamless change, factory mode, etc. that provide fast booting and basic TV functions. In addition, the module may be in charge of switching from the Non-WebOS mode to the WebOS mode.

Referring to Fig. 9, the processing structure of the media server is shown.

At this time, in FIG. 9 , a solid line box may indicate a process processing configuration, and a dotted line box may indicate an internal processing module during a process. In addition, a solid arrow may indicate an inter-process call, that is, a Luna service call, and a dotted arrow may indicate a notification such as register/notify or a data flow.

A service or web application or PDK application (hereinafter, 'application') is connected to various service processing components through a Luna-service bus, through which the application operates or operation is controlled.

Depending on the type of application, the data processing path is different. For example, when the application is image data related to a camera sensor, it is transmitted to the camera processing unit 930 and processed. In this case, the camera processing unit 930 processes the image data of the received application including a gesture, a face detection module, and the like. Here, the camera processing unit 930 may generate a pipeline through the media server processing unit 910 and process the data, for example, in the case of data requiring a user's selection or automatic use of a pipeline.

Alternatively, when the application includes audio data, the audio may be processed through the audio processing unit (AudioD) 940 and the audio module (PulseAudio) 950 . For example, the audio processing unit 940 processes audio data received from the application and transmits it to the audio module 950 . In this case, the audio processing unit 940 may include an audio policy manager to determine processing of audio data. The processed audio data is processed in the audio module 960 . Meanwhile, the application may notify data related to audio data processing to the audio module 960 , which may also be notified to the audio module 960 in a related pipeline. The audio module 950 includes an Advanced Linux Sound Architecture (ALSA).

Alternatively, when the application includes or processes (hereinafter included) DRM-applied content, the corresponding content data is transmitted to the DRM service processing unit 1160, and the DRM service processing unit 1170 creates a DRM instance (instance). to process the DRM-applied content data. Meanwhile, the DRM service processing unit 960 may be connected to the DRM pipeline in the media pipeline through the Luna-service bus to process the DRM-applied content data.

Hereinafter, processing in the case where the application is media data or TV service data (eg, broadcast data) will be described.

FIG. 9 is a diagram for explaining in more detail only the media server processing unit and the TV service processing unit in FIG. 8 described above.

Therefore, hereinafter, it will be described with reference to FIGS. 8 and 9 together.

First, when the application includes TV service data, it is processed by the TV service processing unit 820/920.

Here, the TV service processing unit 820 includes, for example, at least one of a DVR/channel manager, a broadcast module, a TV pipeline manager, a TV resource manager, a data broadcast module, an audio setting module, and a path manager. Alternatively, in FIG. 9, the TV service processing unit 920 includes a TV broadcast handler, a TV broadcast interface, a service processing unit, a TV middleware (TV MW (middleware)), a path manager, and a BSP ( For example, NetCast). Here, the service processing unit may refer to a module including, for example, a TV pipeline manager, a TV resource manager, a TV policy manager, and a USM connector.

In this specification, the TV service processing unit may have the same configuration as in FIG. 8 or 9 or may be implemented as a combination of both, and some components may be omitted or some components not shown may be added.

Based on the attribute or type of TV service data received from the application, the TV service processing unit 820/920 transmits it to a DVR/channel manager in case of a digital video recorder (DVR) or channel-related data, and then back to the TV pipeline manager to create a TV pipeline and process it. On the other hand, when the attribute or type of the TV service data is broadcast content data, the TV service processing unit 820 generates and processes a TV pipeline through the TV pipeline manager in order to process the corresponding data through the broadcast module.

Alternatively, a json (Javascript standard object notation) file or a file written in c is processed in the TV broadcast handler, transmitted to the TV pipeline manager through the TV broadcast interface unit, and a TV pipeline is generated and processed. In this case, the TV broadcast interface unit may transmit data or files that have passed through the TV broadcast handler to the TV pipeline manager based on the TV service policy and refer to the data or files when creating a pipeline.

Hereinafter, the processing process in the TV service processing unit 920 of FIG. 9 , in particular, in the TV broadcast interface and below will be described in more detail as follows.

The TV broadcast interface may also perform a controller function of the TV service processing unit 920 . The TV broadcast interface requests pipeline creation from the TV pipeline manager, and the TV pipeline manager creates a TV pipeline and requests resources from the TV resource manager. The TV resource manager makes a request for a resource to the media server through the UMS connector, and when it is obtained, it returns it back to the TV pipeline manager.

The TV pipeline manager arranges the returned resources in the created TV pipeline, and registers pipeline information in the path manager. Thereafter, the TV pipeline manager returns the result to the TV pipeline manager, and the TV pipeline manager returns the pipeline to the TV broadcast interface.

Thereafter, the TV broadcast interface communicates with TV middleware (MW) to request a channel change, and the TV middleware returns a result.

The TV service may be processed through the above-described process.

The TV pipeline manager may receive control from the TV resource manager when generating one or more pipelines according to a TV pipeline creation request from a processing module or manager within the TV service. Meanwhile, the TV resource manager may be controlled by the TV policy manager to request the status and allocation of resources allocated for a TV service according to the TV pipeline creation request of the TV pipeline manager, and the media server processing unit 810 . /910) and data communication through the uMS connector. The resource manager in the media server processing unit 810/910 transmits the status of a resource for the current TV service and whether or not to allocate it according to the request of the TV resource manager. For example, as a result of checking the resource manager in the media server processing unit 810/910, if all the resources for the TV service have already been allocated, the TV resource manager may notify that all the resources are currently allocated. In this case, the resource manager in the media server processing unit removes a predetermined TV pipeline according to a priority or a predetermined criterion among TV pipelines previously allocated for the TV service together with the notify, and creates a TV pipeline for the requested TV service. may be requested or allocated. Alternatively, the TV resource manager may appropriately control removal, addition, establishment, etc. of TV pipelines in the TV resource manager according to the status report of the resource manager in the media server processing unit 810/910.

Meanwhile, BSP supports compatibility with, for example, existing digital devices.

The TV pipelines generated in this way may be appropriately operated under the control of the path manager during the processing process. The path manager may determine or control processing paths or processes of pipelines by considering not only the TV pipeline but also the operation of the pipeline generated by the media server processing unit 810/910 in the processing process.

Next, when the application includes media data rather than TV service data, it is processed by the media server processing unit 810/910. Here, the media server processing unit 810/910 includes a resource manager, a policy manager, a media pipeline manager, a media pipeline controller, and the like. On the other hand, pipelines generated under the control of the media pipeline manager and the media pipeline controller can be created in various ways, such as a camera preview pipeline, a cloud game pipeline, and a media pipeline. Meanwhile, the media pipeline may include a streaming protocol, auto/static gstreamer, DRM, and the like, whose processing flow may be determined under the control of the path manager. The detailed processing procedure in the media server processing unit 810/910 uses the description of FIG. 10 described above, and will not be repeated herein.

In the present specification, the resource manager in the media server processing unit 810/910 may perform resource management on a counter-based basis, for example.

The design of the media server on the above-described webOS platform will be described in more detail as follows.

The Media Server is a media framework that supports third-party multimedia pipeline(s) to interface with the WebOS platform. The media server may control, manage, isolate, deconflict, etc. resources so that the third-party multimedia pipeline(s) may be compliant. Such a media server can be viewed as a platform module that provides a generalized API so that applications can play media, and manages hardware resources and policies consistently. On the other hand, the design of the media server is to reduce complexity through generalization of media processing and separation of related modules.

The core of such a media server is to provide, for example, integration between a service interface and a webOS UI. To this end, the media server controls the resource manager, policy manager, and pipeline manager, and provides API access according to resource manager queries.

The uMS connector is the main API and SDK to interface the client media pipeline processes with the media server. Ums connectors are events and messages about interfaces. The client media pipelines implement client media pipeline state events to enable load, play, pause, seek, stop, unload, release resource (release_resource), acquire resource (acquire_resource), etc. .

uMedia API provides C, C++ API as media server.

The media resource manager provides a method of describing the use of media hardware resources and pipeline client resources with one simple configuration file. The Media Resource Manager provides all the capabilities and information needed to implement default or third-party media policy management.

The media policy manager functions when the resource manager rejects the media pipeline because of a resource conflict. The policy manager may provide a consistent API and SDK to enable third-party policy manager implementations. The policy manager supports media pipelines matching LRU (Least Recently Used), and may be used for one or more conflicted resources.

The pipeline manager tracks and maintains client media pipelines. The pipeline controller provides a consistent API to the pipeline manager to control and manage client media pipelines.

The media server communicates with the resource manager through library calls, and the resource manager can communicate with TV services, media pipeline, and luna service bus.

The media resource manager configures the entire configurable configuration file for disk drive media hardware and media client pipelines, detects resource conflicts, and collects all information necessary to implement media policy management. .

The media policy manager reads the policy_select and policy_action fields of the resource configuration file, the resource content tries to select the active pipeline described by the policy_select field, and issues with outgoing/selected pipelines based on the policy_action field issue (issue). The selection function may be a parameter supported by a pipeline configuration setting entry. The policy actions are unload and release. All pipelines support the unload command to release all allocated resources. The pipeline may additionally support the release command to release specific resources. In the above, the release command is for fast switch pipelines competing with common resources, and the unload command of all resources may not be required for the incoming pipeline and deconflict.

The pipeline manager manages the pipeline controller. The pipeline manager maintains a running queue of the pipeline controller and provides unique indexing for incoming messages from the application(s) via the media server.

The pipeline controller maintains the relationship of the associated media client pipeline process. The pipeline controller maintains all relevant state and provides the media client pipeline control interface to the pipeline manager. The pipelined client process is an individual process using the uMS connector to provide a control interface to a media server or the like. Pipeline (client) media technology (Gstreamer, Stage Fright) can be separate and completely decoupled from media server management and services.

Meanwhile, “image data” disclosed in this specification includes, for example, still image data such as a picture or photograph, a thumbnail image, and a capture image as well as a video It may also be used to include moving picture data such as (video).

In the present specification, a menu is an application that is output through the entire area or main area of the screen through a web launcher or menu launcher (hereinafter referred to as 'menu launcher') in a digital device equipped with a webOS. It is provided overlaid on the execution screen.

The menu includes a first part including historical data for previously viewed or executed applications, and a second part in which one or more applications available or executable on the digital device are listed. is composed by Here, the second part may not only provide the application list, but may also provide a content list related to an application currently provided through the main area. In other words, the second part may provide at least one of the first mode and the second mode. Hereinafter, it will be described that the first mode of the second part provides an application list, and the second mode of the second part provides a content list. When providing the menu launcher, whether to provide the first mode or the second mode as the second part may be determined according to a setting or an application being provided in the main area. The determination may be determined and provided according to various criteria, such as an application attribute of a main area, according to a user, in addition to the above-mentioned criteria. Meanwhile, the first part may be called a receive part and the second part may be called a list part.

Each of the receive part and the list part may include at least one or more menu items. The menu item may be a unit for classifying, identifying, and accessing content, application, data, and the like in each part. In addition, each of the menu items may be provided through a window (window0).

Meanwhile, in the present specification, a menu, a return part, a list part, and a menu item are illustrated and described in a rectangular or trapezoidal shape, respectively, but the present disclosure is not limited thereto.

10 is a diagram illustrating a screen of a digital device outputting an application.

When the application execution request signal is received, the digital device 1000 provides an execution screen of the application execution requested through the entire screen or the main area of the screen.

Here, the application includes all applications of a digital device equipped with a webOS, and the digital device equipped with the webOS is also called a web application. The application may include various applications such as a TV (broadcast) application for a broadcast service such as a broadcast program, an application for external input, and an application for image data. For convenience, FIG. 10 illustrates an embodiment in which broadcast content of a specific channel is provided through execution of a TV application.

Meanwhile, the application may be executed through data pre-stored in a storage medium such as a memory of the digital device 1000 or application data downloaded or streamed from various external servers such as a broadcasting station. In this specification, application and application data may be used interchangeably in some cases, and in that case, the meaning may be determined through context in a corresponding sentence or paragraph. In addition, in the case of preferred applications and frequently used applications, the digital device receives application data using a preload function manually or automatically, and performs quick access or switching/interchanging (switching/interchanging) of the corresponding application. interchanging) can be made possible.

The application execution screen of FIG. 10 is provided according to a user's request through an input means such as a remote control connected or paired with the digital device 1000 or a mobile device. Alternatively, the digital device 1000 may provide the application execution screen according to a predetermined key button signal provided on the front panel of the digital device even without using the input means as described above. In addition, the digital device 1000 may provide various functions, menus, etc. as well as the application execution screen as shown in FIG. 10 based on various input signals such as voice commands and gesture commands through or without the input means. . However, for convenience, when an input means is used below, a signal to a digital device such as a key button on the front panel, a voice command, a gesture, facial recognition, fingerprint recognition, etc. execution request and control command, as well as when using the remote control or a mobile device It has a meaning including at least one of all means for inputting a command.

On the other hand, if the application is a TV application, it may be processed through the component(s) of the digital device shown in FIG. 2 or 3 . In other words, by receiving a broadcast program or/and IP packet(s) or a broadcast signal including signaling data for the broadcast program through a network interface or a tuner, a demodulator, a demultiplexer, an SI decoder, and an audio/video It is finally output to the screen through a decoder or the like. The processing process refers to the contents of FIGS. 2 to 3 described above, and detailed descriptions thereof are omitted herein.

11 is a diagram illustrating a method of providing a menu in a digital device according to an embodiment of the present invention.

When a signal for requesting a menu is received through an input means while using an application as shown in FIG. 10 , the digital device outputs a menu on a current screen according to the received signal. This may be provided by accessing or selecting an icon or function button provided through the screen through a pointer, even if the output of the menu is not necessarily a menu key signal of the input means.

In the conventional digital device, even if the screen is completely switched or not switched to output the menu, many parts or main parts of the currently used application execution screen are covered or overlapped, which makes it inconvenient for the user to use the content. there was.

The digital device equipped with the webOS platform according to the present invention provides the requested menu through a menu launcher as shown in FIGS. 11A to 11C . The provided menu is provided in a region or area on the application execution screen that is being provided through the entire screen, as shown in FIG. 11A . Here, as shown in the figure, the one area is a part of the lower part of the screen as an example, but the present invention is not limited thereto. Meanwhile, the size (hereinafter referred to as size for convenience) or shape such as the height or width of the menu launcher may be arbitrarily adjusted according to a user's setting or an application property. For example, referring to the properties of the application being used, when certain data is output at the bottom of the screen, such as data broadcasting, the menu launcher may be output to a different area from that shown in FIG. 11A . Conversely, as illustrated, the menu launcher may output the predetermined data to another area that does not overlap with the menu output area. This is also applicable even after the menu is output. In addition, the menu may be output to any one of a partial area in the center of the display screen or an edge area of the display screen. In addition, the two parts of the menu may be provided separately in different areas instead of in one area. For example, the first part (recent part) may be provided in one area under the screen, and the second part (list part) may be provided in any one non-overlapping edge area among the screen edge areas. Alternatively, FIG. 11A may be provided at the bottom of the screen as shown, FIG. 11B may be provided on the left edge area of the screen, and FIG. 11C may be provided on the right edge area of the screen at the same time. Also, when any one part or at least one menu item of the part is selected in FIG. 11A , a function or related data related to the selected menu item may be output to at least one edge area among the edge areas.

Accordingly, as shown in FIG. 11A , if a menu is configured and provided, screen blocking can be minimized compared to a conventional digital device.

In addition, the size of the menu may be changed based on the position or movement of the pointer. For example, if the pointer is positioned on a menu, it is determined that the user's intention is to control or access the menu, and the size of the original menu launcher is provided. In this case, the size of the menu launcher may or may not be the maximum. On the other hand, if the pointer is located in an area other than the menu (eg, an application output area currently in use), the user's intention is determined as application control or access, and the currently used menu launcher is minimized or temporarily hidden and the It can be convenient to use the application. In the case where the menu launcher providing the menu is temporarily hidden in the above, it may be provided so that the user can identify that the menu launcher is already being output according to a request. This is to minimize the inconvenience or inconvenience of use, such as the user calling the menu again. In summary, the menu launcher including a menu according to a user's request, a property of an application currently being used, a position/movement of a pointer, etc. can be provided in several versions or can be controlled, such as a size change. In addition, as described above, the menu launcher can be controlled automatically or manually even after outputting the menu.

11A shows a digital device in which a menu launcher including a menu is provided according to a user's request while the application 1110 is being used through the main screen. 11A may be, for example, an initial screen in which a menu launcher including a menu according to the user's request is provided on the screen.

As described above, the menu provided through the menu launcher may be largely composed of two parts. The first part 1120 outputs the receipts data, and the second part ( 1130 includes or outputs an application list or a content list.

An interface 1130 for accessing history data related to the first part 1120 or additional history data is provided on one side of the first part 1120 . The history data means data on an application that was provided on the screen before using the application currently being provided through the screen. Accordingly, when the user selects the interface 1140 , the menu launcher may be switched or changed as shown in FIG. 11B . Referring to FIG. 11B , an interface 1140-1 is provided on the rightmost side, and history data is arranged and provided on the left side of the interface 1140-1. When the user selects the interface 1140-1 through a predetermined action or key signal, the digital device is switched back to the screen of FIG. 11A. Accordingly, according to the access of the interface 1140-1, the menu launcher is provided as shown in FIGS. 11B to 11A. Here, the history data may be provided in an arrangement based on a timeline from the interface 1140-1 to the left. That is, as the history data is closer to the interface 1140-1, it may be data of a recently accessed application or vice versa. Meanwhile, historical data provided in FIG. 11B may be accessed only for a predetermined period of time according to a user's request or setting. In addition, according to the position or movement of the pointer, as for the history data provided in FIG. 11B , history data of a different time unit, that is, additional history data may be provided. In this case, the interface 1140-1 may be continuously provided at the corresponding location.

Meanwhile, an interface 1150 for accessing an application list related to the second part 1130 or an additional application list is provided on one side of the second part 1130 . When the user selects the interface 1150 , the digital device provides the menu launcher of FIG. 11C . An interface 1150-1 is provided on the left side of FIG. 11C, and menu items corresponding to applications are arranged and provided on the right side of the interface. In this case, the sorted menu items may or may not include menu items corresponding to the application that is already being output in FIG. 11A . Meanwhile, the application-responsive menu items provided in the second part 1130 of the menu launcher of FIG. 11A may or may not be preset basic menu items according to settings of the digital device or the user. Alternatively, the menu items provided in the second part 1130 of FIG. 11A or FIG. 11C may be sorted based on the user's application preference, such as the user's application use frequency, number of times, and use time, or the properties of the corresponding application. there is.

In FIGS. 11A to 11C , the interfaces 1140 , 1140-1 , 1150 , and 1150-1 may be embodied in other forms different from those shown. Also, the first part 1120 and the second part 1130 constituting the menu launcher provided in FIGS. 11A to 11C may switch/interchange output positions or regions thereof. In addition, each menu item in the corresponding part can also be moved to the corresponding part or another part.

As used herein, “access or selection” means that a pointer provided on the screen is located in a certain area or part for a predetermined time or is hovering, clicking, dragging, dropping ( drop) and the like. In addition, the selection may be made through a remote control or a separate key button(s) provided on the front panel of the digital device, or may be made through a voice, a gesture, or the like, as described above. It can also be achieved by each combination. Meanwhile, in the present specification, the remote control is described using a motion remote control as an example for convenience, but is not limited thereto, and may include other digital devices such as smartphones.

Although it is illustrated in FIG. 11A that only one menu item is provided in the first part 1120 , the present invention is not limited thereto, and a plurality of menu items may be provided in the first part 1120 . Meanwhile, each of the menu items may identify an application or content corresponding to the corresponding menu item, or may include or output one or more image data corresponding to the corresponding menu item. Each of the menu items may be image data obtained by automatically capturing a screen when an application being used is terminated or a channel or screen is switched. Accordingly, when the user selects the first part 1120 , the first application (application currently being viewed) providing screen is displayed on the main screen as a second application execution screen corresponding to the image data of the first part 1120 . is converted to Before the conversion, the captured image data of the last screen of the first application or the representative thumbnail image data of the first application is provided on the first part 1120 instead of the second application image data. On the other hand, the first application being provided on the main screen may be located in the background in a stopped or stopped state without the playback being terminated. In addition, or in this case, despite the selection, the screen is not completely switched, and playback may be performed at the corresponding position as it is, PIP (Picture in Picture), POP (Picture of Picture), PBP (Picture) by Picture) may be provided.

In the second part 1130 of the menu launcher, as described above, an application list is provided. However, as shown in FIG. 11A , all application lists provided or available in the digital device through the second part 1130 may not be listed at once due to the limitation of the device screen size. Accordingly, the application list that is not listed through the second part 1130 of FIG. 11A is provided in the same manner as in FIG. 11C through the fourth part 1130, as described above. Accordingly, the application list provided in the second part 1130 of FIG. 11A may be regarded as selected from applications available in the digital device. For example, the applications provided in the second part 1130 of FIG. 11A include, among all applications available on the device, i) a user's setting or request, ii) an application recently used by the user, iii) the user's frequency of use, iv) an application related to an application being provided through the main screen, v) may be selected and provided according to the alphabet or Hangul letters/vowels of the application title, or i) to v) may be selected and provided based on a combination of at least two or more. In the latter case, at least two or more of i) to v) may be used as priority factors, respectively, and may be selected based on the factor values.

On the other hand, when a menu item corresponding to any one of the menu items corresponding to the applications listed in the second part 1130 of FIG. 11A is selected, the application of the selected menu item is directly executed on the corresponding area or the main screen It can be provided as a replacement for the application of the whole or provided together with the previously used application through screen division. This may be due to a property of the selected application or may be implemented by providing a function or a control icon in the vicinity of the selected application. Image data of the first part 1120 may be changed according to the screen providing result. On the other hand, the location or order of the applications listed on the second part 1120 of FIG. 11A or FIG. 11C can be arbitrarily changed or controlled. This is also the same for the history data of FIG. 11B. That is, the user can arbitrarily control the position or order of the timeline-based history data.

The menu item selected in each part may have a different color from that of the unselected menu item, or may be provided in a differentiated manner, such as in an enlarged size compared to the unselected menu item. In addition, as shown, the selected menu item may be slightly raised upward, the border may be highlighted, or title data of an application corresponding to a separate corresponding menu item may be output below. In addition, at least two or more of the differentiating contents may be combined to provide various configurations so that the selected menu item is differentiated from other menu items.

11, by providing a menu launcher including a menu together with an application in use, the user can recognize the menu more intuitively as compared to the conventional digital device and access the desired application more easily and quickly. can In addition to using the menu, the user can use various functions or sub-menu items of a digital device or application through the minimum depth while minimizing screen blocking or changes of the application being used, thereby maximizing the convenience of use. In other words, in the digital device according to the present invention, applications can be accessed quickly, switching/interchanging between contents is fast, and operations such as multi-tasking can be easily and quickly performed.

12 is a diagram illustrating a method of providing a menu in a digital device according to another embodiment of the present invention.

Unlike FIG. 11 , the digital device may provide a content/application-based menu configured as shown in FIG. 12 through a menu launcher in response to a user's menu request.

Referring to FIG. 12 , one application is provided on the main screen 1210 , and a menu launcher is provided overlaid on a predetermined area of the main screen 1210 .

Unlike in FIG. 11 , the menu to be described below can be divided into menus for a present mode, a past mode, and a future mode, and the menu launcher configuration for each mode may be different can This division of menu modes is, for example, to provide more intuitive and easier menu use convenience compared to FIG. 11 .

The past mode relates to a menu configuration for previously used history data, and the future mode may include a menu configuration for recommended content/application to be used in the future. Meanwhile, as shown in FIG. 11 , the current mode means a basic menu provided when a menu is called, and may mean including all menu configurations other than the past mode and the future mode. As an example, the current mode may be, for example, a menu configured by focusing on an application/content frequently used in a digital device.

In the present specification, for convenience of description, in response to a user's menu request, the digital device may be set to provide a menu of the current mode as an initial menu through a menu launcher. In this case, in the digital device, the past mode and the future mode may be accessible from the first provided current mode. However, the present invention is not limited thereto. For example, when the user does not specifically request a menu of a specific mode, the digital device provides at least one of the present, past, and future modes through the menu launcher through the user's setting or various related factors. However, hereinafter, for convenience, the menu launcher in the current mode is provided as a basic menu or default menu, and access to the past mode and the future mode from the menu launcher will be described as an example. However, in relation to the present invention, scenarios for mode switching, mode entry, etc. are not limited to those shown and described in the present specification, and may be made in various ways.

12 to 21 show the menu launcher of the current mode menu, FIGS. 22 to 27 show the menu launcher of the past mode menu, and FIGS. 28 to 35 show the menu launcher of the future mode menu. Explain.

First, the menu launcher configuration of the current mode menu will be described in detail with reference to FIGS. 12 to 21 .

Referring to FIG. 12 , as described above, the menu launcher includes a receive part 1220 and a list part 1230 . Here, the menu launcher may further include an interface part 1250 for entering the future mode. And, although not shown, the menu launcher is an interface for accessing a history data list or related functions or data of the recent part 1220 and an additional application list or related functions or data access of the list part 1230 It may further include an interface and the like. Meanwhile, the list part 1230 is configured in the second mode, that is, the content list for convenience.

In the current mode, the menu is provided with a receive part 1220 and a list part 1230 divided based on a reference indicator 1240 , respectively.

The receive part 1220 corresponds to the first part of FIG. 11, that is, the receive part, and includes history data, and in the current mode, only the history data for the application used immediately before the application being provided on the screen in the current mode. can be printed In this case, the receive part 1220 may provide at least one of various data such as image data of the corresponding application, title data 1225 , server data, and icons as history data. The image data may be, for example, any one of a representative thumbnail image of a corresponding application, image data captured just before switching/interchanging to a currently provided application, and image data downloaded from a server or stored in a digital device. In addition, when there are a plurality of image data or history data for a corresponding application, the plurality of image data or history data may be provided in the corresponding menu item in a slide show method.

When the digital device receives the receive part 1220 selection signal, the application included in the receive part 1220, that is, a previously executed application, is played by switching/interchanging with an application provided through the main area. In this case, the playback may be started after the previous part where playback was stopped, or playback may be started again from the beginning. On the other hand, when the switched/interchanged application is, for example, a real-time application (eg, a real-time broadcast program), guide and select a point to be played among data at the previous playback stop point or the current time. may follow Alternatively, although not shown, when the receive part 1220 is selected or the pointer 1260 is located within the receive part 1220 as described above, the digital device displays a reproducing icon for reproducing the corresponding application. may be provided or detailed information about the corresponding content may be provided. Alternatively, although not shown, the digital device does not reproduce by switching/interchanging, as described above, even when the receive part 1220 is selected, but does not affect the application running in the main area. A corresponding application may be executed within the cent part 1220 . In this case, the digital device may provide a playback bar during the playback to further provide data such as a current playback position, a playback end time, or a time remaining until the playback ends.

On the other hand, when a signal for selecting title data or server data in the reward part 1220 is received through the pointer 1260 , the digital device provides an application related to the title data or server data or a server related to the application Guide data for access may be provided on-screen or via a browser. In this case, the guide data or the browser may be overlaid and provided at the highest level on the screen.

The digital device may provide in the form of a menu icon when additional or related data is provided in the process of selecting or accessing the receive part 1220 in the above description. However, it is not limited to the method of providing the icon form.

In addition, in the present specification, it is illustrated that only data for one application is provided in the receive part 1220 for convenience, but this is only an example and is not limited thereto.

On the other hand, the aforementioned portion of the recents is related to the past mode to be described later. However, a detailed description related thereto will be described later and will be omitted herein.

In the above, the reference indicator 1240 serves as a criterion that distinguishes the receive part 1220 and the list part 1230, ie, a boundary. For example, as described above, since the recent part 1220 outputs only one recent data, it is not scrolled left/right or the like. However, the list part data can be scrolled left/right according to the number. However, the list part data is made based on the reference indicator 1240, and even if the scrolling occurs, the receive part 1220 may be continuously provided at the corresponding position without being affected. In addition, if the menu launcher has a multi-level configuration rather than a single-level vertical as shown in the figure, the reference indicator 1240 may perform a scroll function.

List part 1230 provides one or more menu items for an application, as described above. In this case, the menu items are sorted for the list part of the current mode, for example, according to various criteria. The various criteria may refer to, for example, at least one of a setting of a user or a digital device, a recent execution order of an application, an execution time, the number of executions, an execution frequency, an attribute, and the like.

Each of the menu items may include icon data 1232 for a corresponding application, as shown in FIG. 12 . In this case, each of the menu items may further include image data, title data, and the like for the application. In the above, the image data may be, for example, representative thumbnail image data for a corresponding application. Meanwhile, various data provided on the menu item may be provided only when a signal for the corresponding menu item is received, or may be provided according to access of an adjacent menu item.

As described above, although the list part is not clearly shown, it is possible to access additional application menu items based on the reference indicator 1240 without changing the screen configuration by providing a scroll bar.

An embodiment of controlling the list part of the menu launcher will be described with reference to FIG. 13 . Here, the list part of FIG. 13 is configured in the second mode.

When the digital device receives a signal for selecting one menu item belonging to the list part, as shown in FIG. 13 , the digital device provides the first selected menu item to be differentiated from the unselected menu item(s). In FIG. 13 , the corresponding menu item moves up a little higher than the unselected menu item. Also, the selected menu item provides data for the application, unlike the unselected menu item(s). In FIG. 13, server data (ex, Pooq) of the corresponding application and title data (ex, it's okay, it's love) of the application are provided. However, the digital device may further provide additional data or provide other data in addition to the data shown and described above. In the above, the server data means, for example, data about a server that provides or services the corresponding application data.

Another embodiment of controlling the list part of the menu launcher will be described with reference to FIG. 14 .

Following FIG. 13, in FIG. 14, a list part is still provided in the second mode. However, as described above, according to the present invention, the list part may be configured and provided in the first mode in addition to the second mode of providing the application list or content list related to the application or content being provided in the main area. As described above, the first mode includes a list of applications that can be provided or executable by the digital device regardless of applications or content being provided in the main area. In other words, the first mode of the second part provides an application list or content list that is not related to the application or content currently running in the main area, but the second mode provides a related application list or content list.

When a digital device provides a menu launcher, it is difficult for a user to immediately identify whether to provide the second part in the first mode or in the second mode. Accordingly, it may be required to switch/interchange the mode of the second part as easily and conveniently as possible after the menu is provided. According to FIG. 14 , when the pointer 1410 is positioned in a predetermined area (eg, a lower area of the list part) within the list part, an interface 1420 for mode switching may be provided as shown in FIG. 14 .

Here, when the digital device receives the mode switching interface 1420 selection signal, the second part is provided by mode switching from the second mode of FIG. 14 to the first mode of FIG. 15 .

Referring to FIG. 15 , in the second part, a first mode, that is, a list of applications that can be provided or executable in the digital device 1510 is provided. Here, each menu item of the application list 1510 provided in the first mode includes, for example, data to identify a corresponding application. In addition, as shown in FIG. 15 , one or more menu items 1520 and 1530 of the application list 1510 provided in the first mode may be used instead of or together with the corresponding application identification data. It can contain image data for the application. For example, the image data may be image data captured at the last execution end or representative thumbnail image data of a corresponding application, which may be received from a memory of a digital device, a server or a web server that provides the corresponding application. In this case, as described above, the application list without image data may be, for example, an application that has never been executed or has not been recently executed.

FIG. 16A shows, for example, a menu structure similar to that of FIG. 11A. In other words, interfaces 1620 and 1630 for accessing additional menu items are provided on one side of each of the first part and the second part. Meanwhile, here, the second part is provided in the second mode.

Fig. 16B provides a different interface than Fig. 16A. In FIG. 16B , arrow-shaped interfaces 1640 and 1650 that perform the same function as the interface of FIG. 16A are provided. Although it is illustrated in FIG. 16B that the interfaces 1640 and 1650 are provided in an area that does not overlap with the menu launcher, they may be provided at the location of FIG. 16A according to an embodiment.

17 illustrates, for example, another embodiment of mode switching of the above-described second part. While FIG. 15 shows switching/interchanging from the second mode to the first mode, FIG. 17 is an example of switching from the first mode to the second mode. In other words, when the pointer is positioned in a predetermined area in the first mode of the second part of the menu launcher, that is, positioned at the lower edge, the digital device provides the interface 1710 for mode switching. When the mode switching interface 1710 is accessed, the digital device changes the configuration of the second part from the existing mode (first mode) to another mode (second mode).

Referring to FIG. 18 , the menu includes a first part and a second part, and the second part provides a second mode, that is, an application list or a content list. As described above, such an application list or content list includes an application or content related to the application or content being provided in the main area.

Referring to FIG. 18A , the digital device indicates that the list in the second part can be scrolled left and right. Accordingly, as shown in FIGS. 18B and 18C , when scrolling left, it can be seen that the menu item for the application is scrolled based on the reference indicator 1810 .

The scrolling in FIG. 18 may be performed by, for example, a direction key button signal provided in an input means such as a front panel or a remote control, or may be performed according to a position or movement of a pointer.

Meanwhile, although not shown, when the pointer is positioned on one menu item, the digital device may perform an operation for controlling the corresponding menu item. For example, when the pointer selects one menu item and is positioned at a corner of the selected menu item, the digital device may process the size of the corresponding menu item to be changed vertically or horizontally. For example, when the pointer is positioned at the right corner of the selected menu item, as shown in FIG. 18A , an icon for changing the left and right size is displayed or the shape of the pointer is changed and the size of the corresponding menu item can be changed according to the selection. there is. When there is a size change, the digital device may further provide various data such as more detailed data, function icons, recommended content/application, and the like, on the menu item of the changed size compared to the menu item of the original size. Alternatively, it is of course possible to enlarge only the size of the menu item.

19 is a diagram illustrating reproduction control of a menu item selected from among the above descriptions.

As shown in FIG. 19A, when the pointer 1420 is positioned on one menu item 1910 of the second part, the digital device switches/interchanges to the full screen without executing the application of the menu item 1910. As shown in 19b, it plays in PIP format within the corresponding window. In this case, referring to FIG. 19B , the digital device may provide the playback bar 1930 within the menu item to control the location or playback of the currently executed part. Although not shown in FIG. 19B , the digital device may provide various function icons for playback control together with the playback bar 1930 .

In addition, as described above, in FIG. 19, the application is played on the window providing the selected menu item. If the user wants to enlarge the playback by switching/interchanging it to the full screen, a switching/interchanging icon ( 1940) may be further provided.

20 shows the control of a selected menu item.

In FIG. 20A , if one menu item is selected and the selected menu item is moved up or down, the digital device may reproduce an application of the menu item as shown in FIG. 20B . For example, if it moves upward in FIG. 20A, the application is played as shown in FIG. 20B. Conversely, if the menu item is moved in the downward direction in FIG. 20A , the playback of the currently playing application may be terminated.

Alternatively, when an application (eg, an MP3 application) of a menu item is being played back in FIG. 20A , the menu item can be played by moving to the next or previous song directly according to the upward or downward movement of the menu item.

As shown in FIGS. 20A and 20B , for playback control of a corresponding application, after selecting a corresponding menu item, if the menu item is moved upward or downward by a predetermined height, playback control is achieved. , it is also possible to make the menu item for the corresponding application disappear from the list. Alternatively, it may be defined to remove only the playback file of the corresponding application.

As shown in FIG. 21A , when one menu item of the second part is moved in the direction of the lower edge of the screen, as shown in FIG. 21B , playback-related icons 2122 , 2124 , 2126 are displayed around the corresponding menu item (eg, the corresponding menu). on the top of the item, etc.). The playback-related icons 2122 , 2124 , and 2126 may be recommended content/application, a function icon for controlling the playback of the corresponding application, a preview image of the corresponding application, and the like.

Meanwhile, in FIG. 21C , the selected menu item may be moved to the left and right corners instead of the lower edge unlike in FIG. 21A . By moving left and right in this way, the digital device may position the corresponding menu item between other menu items.

Next, as described above, the menu launcher configuration and control of the past mode menu will be described in detail below with reference to FIGS. 22 to 27 .

With respect to the past mode, its entry can be made in a variety of ways. Entering the past mode is made by accessing, for example, an interface or part (eg, the first part 1120 or the third part 1140 in FIG. 11 ) for confirming the details of the received part or the received part. can In the above, for example, when the first part 1120 is accessed, UI data guiding entry into the past mode may be provided. Alternatively, although not shown, as shown in FIG. 11 or 12, when the first menu is provided, menu items or menu icons for entering the past mode, the present mode, and the future mode are provided, and depending on the selection, it is possible to enter the past mode. may be

Hereinafter, a scenario after entering the past mode will be described in detail.

Referring to FIG. 22A , in the past mode, that is, history data is provided in an arranged order according to a time sequence previously used. In FIG. 22A, for example, it can be seen that the application/content used is older as it moves to the left based on the right corner of the menu launcher.

Meanwhile, in FIG. 22A , sorted applications are grouped and provided, for example, by date unit (eg, yesterday 2210 ). However, it may be sorted according to other criteria according to the number of applications to be provided in the past mode. For example, in FIG. 22A , even within the date (yesterday 2210) criterion, it may be sorted by time unit, minute unit, second unit, and the like, and provided so that it can be identified. In addition, although not shown, the applications are, in addition to time criteria such as date and day of the week, application/content properties, application/content quality, application/content providing channel, person, such as a lead actor in the application/content, whether the application/content series, application/ It may be arranged and provided based on at least one of whether content use is completed, whether application/content real-time data/streaming is performed, and application/content capacity. Meanwhile, at least two or more of the above-described criteria may be combined to be used for sorting applications.

22B shows, as shown in FIG. 22A , among the sorted applications, when, for example, an application currently in use is a real-time application, an application with a fixed period of use, etc., the use is terminated. In this case, it may be removed (2220) from the application list of the past mode. In this case, the digital device may also provide guide data together to identify that the corresponding menu item is removed from the application list in the past mode due to the end of viewing upon the removal. Although it has been described above that guide data is provided when removed from the list, the guide data may be provided in advance of the removal, that is, a predetermined time before the end of application/content use of the corresponding menu item to induce additional actions of the user. . However, the guide data may be provided only when the past mode is performed. As described above, providing guide data means that it is not easy to access again after the application/content of the corresponding menu item is removed from the list. , time-shift, etc. to guide the use of functions. As described above, in the case of a real-time application, it is possible to connect a VOD server or an Internet server, check whether related data is recorded, and provide alarm data later.

Although not shown, in the present mode, past mode, and future mode, various functions such as EPG/ESG data call, immediate recording/reserved recording, and reservation viewing may be directly used through menu items.

In addition, the digital device provides historical data, that is, when there is an update, such as correction or change of some data for menu items, on the corresponding menu item or on a predetermined area of the screen while using the past mode. can be provided so that it can be identified.

23 is a diagram illustrating an embodiment of controlling a menu item in a past mode.

Referring to FIG. 23 , as described above, menu items are arranged and provided according to viewing time.

Here, the location of the menu item 2310 for the KBS2 professional baseball broadcast service application that has been previously viewed among the sorted menu items may be moved in the past mode. For example, the menu item 2310 may be positioned 2320 between the Halcyon Days application menu item and the Rain on Me application menu item, arranged as viewed earlier. Although not shown, as shown in FIG. 23 , when the location of a predetermined menu item 2310 is moved, menu items included in the past mode related to the menu item are arranged around the menu item, or a function icon related to the menu item is displayed. may be provided.

24 is a diagram illustrating an application processing method in the past mode.

Referring to FIG. 24A , in the past mode, as described above, menu items for previously viewed applications are arranged and provided. In this case, for each menu item, an image immediately before the end of the viewing is captured and provided on the menu item.

When the menu item 2420 selection signal is received through the pointer 2410, the digital device slightly raises the menu item 2420 upward to indicate that it has been selected, and displays a play icon 2430 and a play bar ( 2440) is output. Here, when the play icon 2430 selection signal is received, the digital device may play the play icon 2430 on the main area or the corresponding menu item window following the previous play part.

24B and 24C illustrate an embodiment of providing a multi-view screen through, for example, accessing a menu item for a past mode.

24A illustrates an embodiment in which, when a menu item is accessed, the application of the menu item is directly reproduced in the corresponding window or the application of the menu item is reproduced in full screen instead of the application running in the main area. In 24c, the application of the menu item is executed on the PIP window or the divided screen.

24B shows an embodiment in which the application of the selected menu item is provided through the PIP window 2450 . 24C is similar to FIG. 24B, but the positions at which the PIP windows 2450 and 2560 are provided are different. Meanwhile, in FIG. 24C , in addition to the PIP window 2460 for executing the application of the selected menu item, one or more PIP windows ( 2470,2480) are further provided.

Unlike the aforementioned FIG. 24A, in FIGS. 24B and 24C described above, other applications can be used while minimizing interference with the use of a previously used application.

25 is also a diagram illustrating an application processing method in the past mode.

Unlike FIG. 24 described above, in FIG. 25A, when a menu item corresponding to a previously used application in the past mode is selected through a pointer and then moved to a main area in which a previously used application is provided through the entire screen, as shown in FIG. 25B . , the application of the menu item is provided over the entire screen. In this case, in FIG. 25B , it is also possible to hide or end the menu launcher of the past mode that is being provided.

26 is a diagram illustrating a method of controlling the menu launcher in the past mode.

Referring to FIG. 26A , the menu launcher of the past mode is configured and provided. The menu items are grouped and provided based on usage time (2610, 2620), and each group is provided with guide data (ex, yesterday, one week) so that each group can be identified. etc.) are provided.

However, in FIG. 26A, for example, in configuring the menu of the go-go mode, only the TV service application may be configured and provided.

In this case, if the user requests past mode data for other applications such as a game, a web browser, and HDMI in addition to the TV service application, it can be provided as shown in FIG. 26B .

In FIG. 26A , if the pointer is positioned on the lower predetermined area of the menu launcher being provided, the digital device provides a menu item switching interface 2630 as shown in FIG. 26B similar to the control method in the current mode.

When the menu item switching interface 2630 of FIG. 26B is accessed, the digital device reconfigures or converts menu items for the past mode by excluding or including the menu items of the TV service application previously provided in FIG. 26A. .

26B shows that the interface 2630 only for controlling menu items between yesterday and one week ago is provided according to the location of the pointer for convenience, but as described above for all data of the past mode regardless of the location of the pointer , it can also be implemented so that conversion is possible.

Meanwhile, although not shown, in configuring the menu of the past mode, when there is a lot of historical data of the past mode, unlike the above description, only the group icon may be provided. Examples of such group icons include a week ago, yesterday, and the like in FIG. 26 . Alternatively, only one menu item for one representative application in each group icon may be provided. In the former or the latter, if a predetermined group icon selection signal is received, the digital device may expand and provide menu items belonging to the selected group.

The menu configuration method described in this specification may be, for example, according to a user's setting or may be determined by a digital device. In the latter case, various criteria such as user data, usage pattern, time, date, day of the week, and weather may be used or determined by appropriately combining the criteria.

FIG. 27 is also a diagram for explaining the menu item control in the past mode.

As described above, even in the past mode, the menu item may perform a predetermined function by moving in the up, down, left, and right directions after selection.

Referring to FIG. 27A , in the digital device, when a predetermined menu item 2710 is selected through a pointer and the selected menu item is moved in the direction of the lower edge of the screen by dragging, as shown in FIG. 27B , data related to the selected menu item ( 2722,2724,2726).

In the above, the related data 2722 , 2724 , and 2726 may be a function icon for performing a predetermined function for the application of the menu item, an application related to the application, or a preview image (including a previously viewed portion) of the application. . For example, if the selected menu item is for baseball broadcast content, the related data 2722 , 2724 , and 2726 may be baseball broadcast content of another channel.

Finally, with reference to the accompanying drawings, the menu configuration and control of the future mode will be described in detail with reference to FIGS. 28 to 35 below.

28 is a diagram illustrating, for example, a method for entering a future mode menu.

As described above, the future mode menu may be entered by accessing the interface 2820 for entering the future mode provided separately from the menu launcher via a pointer 2810, for example, as in FIG. 28A, as shown in FIG. 28b, it may be entered through the interface 2840 for entering the future mode provided together with the menu launcher of the current mode (including the past mode) provided according to the menu call.

The future mode entry interface 2820 of FIG. 28A may be provided without a current mode menu when a menu is called, for example, as in FIG. 28B , but even if there is no menu call, it will be provided directly on the screen in response to a key button signal of the remote control. may be Meanwhile, although FIG. 28A illustrates that the future mode entry interface 2820 is provided at the lower middle of the screen, the present invention is not limited thereto. Accordingly, the future mode entry interface 2820 may be provided at various locations on the screen. In addition, the shape of the future mode entry interface 2820 is not limited to the shape shown in FIG. 28A and may be configured in various ways.

As shown in FIG. 28 , when a selection signal of the future mode entry interfaces 2820 and 2830 is received, the digital device configures and provides the future mode menu as shown in FIG. 29 .

The future mode refers to a menu configuration for recommending applications/content that the user has not yet used or has high availability.

In the future mode, recommended applications/contents can be provided by sorting them by category. These categories include the Basic category, the Live category, the TV shows category, the Movies category, the Music category, the Recommended Applications, Apps category, and My Content ( My contents) category, etc. An embodiment of each of these categories is shown in FIGS. 29 to 35 with respect to the menu configuration. On the other hand, the category of the future mode menu of the present invention is not limited to the above-mentioned bar, and can be implemented in various ways according to the user's setting. Also, as will be described later, the menu configuration of each category may be implemented differently from each other in consideration of the characteristics of the corresponding category.

Referring to FIG. 29 , the future mode menu includes a search area 2910 , a category area, and a recommended content area 2930 corresponding to the selected category. In particular, in FIG. 29 , a basic category 2920 is selected from the category area, and applications/contents belonging to the basic category are listed in the recommended content area 2930 .

The basic category of FIG. 29 may list all recommended content or recommended applications/content selected according to various criteria. Therefore, in the former case, it may overlap with the recommended application/content of another category. Meanwhile, the order in which the basic categories are sorted may depend on a setting such as category order, priority order, and the like. In addition, in relation to the latter, the various criteria may include, for example, season, weather, time, identified user, recently used application/content attribute, type, etc., whether paid or not. For example, FIG. 29 is a list of applications/content related to winter and/or snow selected and recommended based on the current season (ex, winter) or weather (ex, snowy weather) in which the future mode is executed. will be. In other words, in the embodiment of FIG. 29 , applications/contents that are popular and usable in snowy winter are listed.

Meanwhile, although not shown, one or more applications/contents selected from the recommended applications/contents listed in FIGS. 29 to 35 are converted to full screen and played according to the selection, or a Picture by PBP (Picture by Contents) menu is maintained. Picture) (eg, split screen), PIP (Picture in Picture), POP (Picture of Picture), etc. may be provided in the form.

30 shows a live category 3010 and FIG. 31 lists recommended applications/contents according to a TV show category 3110 . Here, both of FIGS. 30 and 31 are categories related to, for example, a TV service or a TV service application. However, while FIG. 30 provides EPG/ESG data for broadcast contents to be broadcast based on the current time, FIG. 31 is different in that it mainly provides VOD contents.

Although not shown, if one menu item is selected from the guide provided in FIG. 30 , immediate recording/scheduled recording, etc. may be directly performed in the future mode. In addition, when a predetermined menu item is selected in FIG. 30 , a preview image may be provided, such as switching to the full screen or within a window in which the menu item is currently provided. This is also the case for reservation viewing and the like. If, in the process of using the future mode, it becomes a live broadcast time of pre-provided menu items or due to scheduled viewing, it may be removed from the future mode or executed as it is in the corresponding window. In this case, after setting the reserved viewing, if another category or menu is used, it may be played through the entire screen or after automatically executing the live category in the future mode on the corresponding window. In addition, in each of the above-described cases, guide data for convenience of use or identification may be provided.

As described above, FIG. 31 may be a VOD recommendation list according to an embodiment. In this case, free VOD content and paid VOD content may be separately provided. In addition, VOD contents may be classified and listed in units of servers that provide VOD contents, types or attributes of VOD contents.

32 is provided by a recommended application/content list belonging to the movie category 3210 . The recommended application/content list belonging to the movie category may be, for example, a movie VOD, or a list of movie content stored in a digital device but not yet used. Alternatively, the recommended application/content belonging to the movie category may be content to be provided on a movie-only channel among one or more channels belonging to a TV service. Although illustrated in FIG. 32 , it is obvious that menu items corresponding to the recommended application/content provided in the future mode can be moved in any direction, up, down, left, or right. In addition, if there is such a location movement, if the meaning of the movement is determined to, for example, increase the priority in the digital device, it is possible to rearrange or reconfigure the recommended applications/content in the list provided according to the meaning. The same goes for categories. In other words, the user may also change the position of each category menu item arranged in the category area, such as moving, and may change the configuration of the recommended application/content list to be sorted or sorted accordingly.

33 provides a recommended application/content list belonging to the music category 3310, FIG. 34 provides a recommended application/content list belonging to the application category 3410, and FIG. 35 is a recommendation belonging to the My Content category 3510 Provides application/content list.

36 is a diagram illustrating an example of a digital service system according to an embodiment of the present invention.

Referring to FIG. 36 , the digital device 3610 and the second digital device 3640 communicate with an external server (cloud server, etc.) 3650 through data communication, according to the present invention, menu configuration data for menu configuration, the menu It is possible to receive application/content data for the application/content of the My Menu item. Meanwhile, in addition to the digital device 3610 and the second digital device 3640 , the mobile device 3620 or the repeater 3630 may also perform data communication with the above-described external server 3650 . In addition, in the above, the mobile device 3620 , the repeater 3630 and the second digital device 3640 transmit data received through data communication with the external server 3650 for the menu configuration of the digital device 3610 , etc. can transmit In this case, if necessary, the data to be transmitted may be processed and transmitted, such as by changing the format.

Meanwhile, at least one of the mobile device 3620 , the repeater 3630 , and the second digital device 3640 may receive and output menu data provided by the digital device 3610 . For example, if the digital device 3610 is outputting the current mode menu, at least one of the mobile device 3620 and the second digital device 3640 may receive and output the current mode menu, and other than the current mode You can receive and output the menu of the past mode or future mode. Also, each of the mobile device 3620 and the second digital device 3640 may receive and output menu data in different modes from the digital device 3610 .

37 illustrates an embodiment in which a menu requested by the digital device 3710 is transmitted and output to the mobile device 3720 instead of the screen of the digital device 3710, for example.

Meanwhile, FIG. 38 may be an embodiment of a new type of menu launcher, that is, a configuration of a menu launcher in the form of a mobile device. As another example, FIG. 38 may be a UI implemented to execute various menus, data, applications, etc. of the mobile device in the digital device according to the execution of an application such as OSP of the mobile device.

39 is a flowchart illustrating a method for processing menu data in a digital device according to the present invention.

The digital device receives a broadcast signal including content and signaling data (S3902), decodes the received content and signaling data, and then outputs it on the screen (S3904).

When the digital device receives the first user input signal for calling the menu (S3906), the digital device configures a menu screen on a predetermined area of the screen on which the content is being output and outputs the menu screen as an overlay (S3908).

As described above, after the menu is output together with the content, when a second user input signal is received (S3910), the digital device displays a GUI for switching/interchanging the output menu screen configuration according to the second user input signal output (S3912),

When a third user input signal is received (S3914), the digital device switches/interchanges the configuration of the menu screen according to the third user input signal and outputs it (S3916).

Hereinafter, the multi-view mode data processing will be described in more detail with reference to the accompanying drawings.

40 is a diagram illustrating a sound control method when a screen is divided in relation to multi-view mode data processing.

In FIG. 40 , it is exemplified that the screen is divided into two screens for convenience. However, it should be noted in advance that the control method to be described later can be applied in the same or similar manner not only to the two split screens shown but also to the multi-view mode to be described later.

The screen of FIG. 40 includes the first split screen 4010 and the second split screen 4020 as described above. In this case, the digital device turns on the sound of the first split screen 4010 and outputs it through a speaker, but turns off the sound of the second split screen 4020 and outputs it through a speaker, etc. You can control it not to happen. In this case, for the second split screen 4020 on which no sound is output, subtitles (captions or subtitles) may be provided in the corresponding area instead of sound.

Meanwhile, the sound control method described above may be performed according to a user's setting or a basic setting of a digital device, for example, sound priority output to the split screen on the left when a screen is divided. On the other hand, as described above, even if it is not the setting of the user or the device, the sound setting can be arbitrarily controlled for any one or all split screens based on the illuminance or noise of the surrounding environment. Such control may be temporary or may be performed in units of content. On the other hand, in addition to the above-described conditions, the user's tendency based on personalized information or the attribute or type data of the content provided in each split screen may be further referred to and used for the sound setting. As described above, such a sound control method may be appropriately applied to a multi-view, which will be described later.

Hereinafter, the multi-view mode control method according to the present invention will be described in more detail. Here, the multi-view mode is described assuming the current mode for convenience, but it is obvious that the past mode and the future mode are also controllable. In addition, although not shown, a separate multi-view mode exists in addition to the above modes, and the multi-view mode control method according to the present invention may be described through it.

41 and 42 are diagrams illustrating a method for entering a multi-view mode according to the present invention.

Referring to FIG. 41 , an application execution screen for one content is provided on the main screen, and a menu launcher of the current mode is overlaid on it.

In FIG. 41 , the menu launcher in the current mode provides items for a plurality of applications. Here, when the pointer is located on any one of the provided items 4110, the digital device provides the item 4110 corresponding to the pointer position in an enlarged manner compared to other items.

Meanwhile, as described above, when the pointer is positioned on the predetermined item 4110, the digital device may process as shown in FIG. 42 .

Referring to FIG. 42A , in the digital device, if the pointer 4212 continues to exist on the corresponding item 4110 for a predetermined time or longer, an icon for multi-view entry (hereinafter referred to as multi-view) is placed on the top of the item 4110 . -view icon) 4216 or play bar 4214. The provided play bar 4214 may be displayed and provided so as to be reproducible following the immediately preceding execution point of the application of the corresponding item, for example. Meanwhile, although the playback bar 4214 is illustrated in FIG. 42A , a playback icon is provided on an item or around the multi-view icon 4216 instead of the playback bar 4214, and according to the selection of the playback icon A play bar 4214 may be provided. Here, as described above, playback control for the application of the corresponding item is possible through the playback icon or the playback bar. Upon receiving a playback command, the digital device may provide a corresponding application execution screen on the main screen, or You can also play it directly on Windows. In relation to the present invention, if a separate multi-view mode is entered or a multi-view icon is provided, the multi-view mode may be entered according to the playback command.

Figure 42b shows that the multi-view icon 4222 provided in Figure 42a can be selected through a pointer, and when the multi-view icon 4222 is selected as above, the digital device remains as shown in Figure 42c, only the corresponding item The remaining one or more items are removed so that the menu launcher is not terminated or the menu launcher itself is not terminated. As will be described later, the multi-view window may be provided by directly entering the multi-view mode.

In addition, although not shown, the multi-view mode may be simply executed by a multi-view key button provided in an input means such as a remote control, a gesture, a voice, or the like. The gesture, voice, or the like may be recognized or inputted from a corresponding digital device or through another digital device without necessarily using an input means. Although the multi-view entry has been described above, it is possible to enter by various methods other than the entry method described in this specification, and this is also included in the present invention.

43 shows and describes a multi-view mode providing screen and a multi-view control method after entering the multi-view mode.

43 is a diagram illustrating a multi-view entry screen according to an embodiment of the present invention.

Referring to FIG. 43A , the multi-view window 4310 is provided in the form of a PIP at the lower left of the main screen. However, the location of the multi-view window 4310 may be changed depending on whether a menu launcher exists, whether caption data for an application being provided on the main screen exists, whether data broadcasting is provided, and the like. Meanwhile, the multi-view window 4310 may be fixed at a corresponding position or may be moved according to a user's control. Meanwhile, the size of the multi-view window 4310 may be arbitrarily adjusted differently from that illustrated. In addition, the shape of the multi-view window 4310 does not necessarily have to be a rectangle, and a user can arbitrarily change the setting. When necessary to change the shape setting of the multi-view window 4310, the digital device may provide a template for the convenience of selection. Meanwhile, although not shown in FIG. 43A , the outline of the multi-view window may have a specific color or be highlighted in order to distinguish it from the application screen provided on the main screen and to identify the multi-view mode execution. In addition, guide data may be separately provided in the form of text or icons on the main screen or around a window to identify that it is multi-view execution.

FIG. 43B is an embodiment in which function icon(s) are provided around the multi-view window, unlike FIG. 43A. FIG. 43B may be a multi-view configuration when the first multi-view window is provided, or may be a multi-view configuration provided according to a predetermined action of a user in FIG. 43A.

When the multi-view window is called according to a user's request, etc., the digital device displays an indicator 4320 provided on one side of the multi-view window when an application related to an application to be provided on the multi-view window is additionally present. provided, so that the user can identify that the related application is available and exists. Meanwhile, a delete icon for removing the multi-view window may be provided on the corresponding multi-view window. If a plurality of multi-view windows are being provided, a recycle bin icon is provided, the delete icon may be provided on each multi-view window according to the selection of the recycle bin icon, and the multi-view window is set as the recycle bin icon. You can control the multi-view window through drag & drop, etc. On the other hand, the indicator 4320 may be provided in various heights, widths, colors, shapes, etc. for the user's identification convenience. For example, when there are two and four sub-multi-view available applications, the width of the indicator 4320 may be different so that the number of additionally available applications can be estimated. In addition, although not shown, in relation to the indicator 4320, the number of sub-multi-view available applications may be provided on the multi-view window or on or around the indicator in the form of numbers so that it can be identified.

44 is a diagram illustrating a method for controlling a multi-view window according to another embodiment of the present invention.

In particular, FIG. 44 is an embodiment related to control of an application capable of providing a sub multi-view, as in FIG. 43B described above.

When the pointer is positioned on the indicator 4410 provided on one side of the main multi-view window or the indicator 4410 is selected, as shown in FIG. 44A , the digital device responds to the indicator 4410 access and displays FIG. 44B In the same way, one or more sub multi-view windows 4420 , 4430 , and 4440 are expanded and provided.

Here, the sub multi-view windows 4420 , 4430 , and 4440 may be the same application as the application provided by the main multi-view window, or may be a different application. Meanwhile, each of the sub multi-view windows 4420 , 4430 , and 4440 may be the same application or a different application.

In addition, the sub multi-view windows 4420 , 4430 , and 4440 may be provided with a size smaller or the same as that of the main multi-view window. Also, the resolution of the sub multi-view windows 4420 , 4430 , and 4440 may be lower than the resolution of the main multi-view window.

In addition, for convenience, the sub multi-view windows 4420 , 4430 , and 4440 are described as providing applications related to the initial multi-view window, but the present invention is not limited thereto and may not be related to each other.

45 is a diagram illustrating a detailed control method of a multi-view window according to an embodiment of the present invention.

In FIGS. 44 and 45 , for example, when the application of the main multi-view window provides content of a sports channel as a TV application, the sub multi-view windows provide sports that are provided in another channel.

45A and 45B , with the exception of the main multi-view window, each of the sub multi-view windows has a corresponding multi-view window in another sub multi-view window according to the pointer position or access (4510, 4520). In contrast, it indicates that the access is provided so that it can be identified. In other words, the size of the selected multi-view window is different from that of other multi-view windows, or as highlighted or shown, the selected multi-view window is provided so that it can be identified that it is selected.

Alternatively, since the resource of the digital device is limited, even if a plurality of sub multi-view windows are provided through the indicator, the corresponding application may not be reproduced in all the multi-view windows according to the resource state. In this case, the digital device provides a representative thumbnail image or a captured image for the application of the corresponding multi-view window, and as shown in FIGS. 45A and 45B, only the application corresponding to the selected multi-view window is played back. can do. In this case, when a resource is needed, the reproduction of the main multi-view window may be paused in order to use the resource for the main multi-view window.

46 and 47 are diagrams for explaining a detailed reproduction control method of a multi-view window according to an embodiment of the present invention.

Referring to FIG. 46 , when the main multi-view window is selected while the main multi-view window is provided, the digital device may provide a delete icon 4630 as described above.

When the delete icon 4630 is accessed through the pointer, the main multi-view window may be removed and the multi-view mode may be terminated. In this case, the digital device includes the application provided through the removed multi-view window in the receive list, and may store the playback end time.

In addition, when the main multi-view window is dragged and dropped through the pointer on the main area 4610, the digital device ends playback of the application being provided in the main area 4610, and the An application may be provided through the main area 4610 . At this time, when there is a multi-view window other than the dropped multi-view window, the application provided in the main area 4610 and the application provided through the multi-view window are switched/interconnected as shown in FIG. 47 , can be changed However, if the multi-view window does not exist, the multi-view mode can be automatically ended. Alternatively, as described above, the multi-view mode may be automatically terminated according to a user's drag and drop regardless of whether the multi-view mode exists.

Meanwhile, although not shown, the above-described contents may be applied to sub multi-view windows other than the main multi-view window in the same or similar manner.

48 is a diagram illustrating a method for ending a multi-view mode according to the present invention.

As described above, in FIG. 48A , the delete icon 4820 provided on the multi-view is selected through the pointer 4810, and the digital device removes the corresponding multi-view window in response to the selection. At this time, if all multi-view windows are removed from the screen, the multi-view mode is automatically ended.

FIG. 48B shows, for example, a state in which the corresponding multi-view window is removed when the delete icon of FIG. 48A is selected. Referring to FIG. 48B , according to the selection of the delete icon, the penguin 4830 comes out and disappears with the multi-view window out of the screen as in animation to remove the corresponding window, so the multi-view window is removed and the multi-view mode ends

Meanwhile, although not shown, the termination in response to entering the multi-view mode may also be conveniently performed through an input means such as a remote control, a multi-view key button provided on the front panel, a gesture, or a voice.

49 to 58 are diagrams for explaining a sub multi-view operation control method according to the present invention.

In particular, FIGS. 49 to 52 disclose various embodiments of a method for controlling a sub multi-view operation using the above-described indicator.

Unlike FIGS. 43B and 44 , FIG. 49 illustrates an embodiment in which the indicator 4920 is provided on the upper surface of the multi-view window 4910 instead of the right side.

When the indicator 4920 provided on the upper surface of the multi-view window 4910 in FIG. 49 is accessed, as shown in FIG. 50 , the digital device moves in the direction of the upper surface of the multi-view window 4910 , or More sub multi-view windows 5010 may be provided.

Meanwhile, referring to FIG. 50 , sub multi-view windows 5020 may also be provided on the right side of the main multi-view window. The sub multi-view windows 5020 may be, for example, preview data for an application provided on the main multi-view window even if a separate indicator 4920 is not accessed. The preview data may be a still image or a moving image. For example, if the application provided on the main multi-view window is video data for a total of 60 minutes and is currently playing 40 minutes, in the sub multi-view windows 5020, 10 minutes, 20 minutes, It can be preview data for 30 minutes or 50 and 60 minutes. In addition, although it has been described above that the application on the sub multi-view windows 5020 is data related to the application being provided on the main multi-view window, the present invention is not necessarily limited thereto, and may be unrelated application data. For example, in FIG. 49 , when only the sub multi-view windows 5010 provided on the upper part as the user accesses the indicator 4920 are insufficient for reasons such as screen size, the remaining sub multi-view windows 5020 may be provided by allocating to the right side of the non-overlapping main multi-view window.

In addition, in FIG. 50 , the first sub multi-view window 5010 is preview data or related to the application on the main multi-view window, and the second sub multi-view window 5020 is the application and the application on the main multi-view window. It may be for an unrelated application. For example, suppose that a TV application on the main multi-view window is providing a first baseball game with 7 innings in progress. In this case, the first sub multi-view window 5010 may be data for the first to sixth times of the first baseball game or may be a highlight image. On the other hand, the second sub multi-view window 5020 may provide data not related to a second baseball game between teams different from the first baseball game or another broadcast program of a corresponding channel.

Referring to FIG. 51 , the main multi-view window 5110 is overlaid on the main screen on which the first application is provided and provided in a predetermined area. Here, indicators for the sub multi-view window may be provided on each side of the main multi-view window 5110 . 51, the first indicator 5120 provided on the right side and the second indicator 5130 provided on the upper side are shown for convenience. However, the present invention is not limited thereto.

In the above-described embodiment, when an indicator provided on the main multi-view window 5110 is accessed, the digital device provides sub multi-view windows around the main multi-view window 5110 according to the accessed indicator. did However, in this case, in some cases, the main multi-view window 5110, as shown in FIG. 51, because it may impair concentration on the application being provided on the main screen or the application on the main multi-view window or cover important data. The sub multi-view windows 5140 may be provided at the edge of the screen rather than around the . The location or area in which the sub multi-view windows 5140 are provided is not necessarily limited to, for example, the right side of the screen shown in FIG. 51, and is automatically performed in consideration of various data such as properties or types of applications and characteristics of the current playback scene. or it may be determined and provided manually. Meanwhile, as described above in FIG. 51 , an embodiment in which the sub multi-view windows 5140 are provided at the edge of the screen rather than around the main multi-view window 5110 but are provided collectively at once is illustrated, but the present invention is limited thereto. However, when three sub multi-view windows are provided as shown in FIG. 51 , each may be provided separately at different positions.

Unlike the above-described embodiments, FIG. 52 shows that sub multi-view windows 5210, 5220, and 5230 are optimized according to the size of the main multi-view window by an indicator or automatically when the main multi-view window is provided. It may be provided on the top. In this case, the sub multi-view windows 5210 , 5220 , and 5230 are not fixed and can be accessed left and right according to the user's action, so they can be replaced with windows for other time zones or other applications, and the main multi-view windows are not fixed. Switching/interchanging with the window or main screen is also possible.

53 shows that the main multi-view window 5310 may be provided at the lower center of the screen from the time of initial provision, and on the contrary, as in the previous embodiment, when provided for the first time, the location is changed according to the user's action after being provided at the lower left of the screen It may be a case of being moved and provided.

In the former case, for example, if one or more sub multi-view windows exist at the time of providing the main multi-view window 5310, unlike the above-described embodiments, the main multi-view window 5310 may be provided at the center of the bottom of the screen. there is. In this case, sub-multi-view window(s) 5320, 5330, 5340, and 5350 are shown on the left and/or right of the main multi-view window 5310 provided in the lower center of the screen in FIG. As can be provided. Meanwhile, in this case, the sub-multi-view windows 5320 and 5330 may indicate a point in time provided through the main multi-view window 5310 , that is, previously provided data. In addition, the sub multi-view windows 5340 and 5350 may indicate data to be provided after the time point provided through the main multi-view window 5310 , that is, after the present. In the former case, a still image or a moving image may be provided through the sub multi-view windows 5320 and 5330, and in the latter case, since it is future data, the still image data is displayed in the sub multi-view window ( 5340 and 5350). However, when the data to be provided through the sub multi-view windows 5340 and 5350 is application data that has already been stored or downloaded in a digital device, it may be a moving picture. On the other hand, unlike the above, the sub multi-view windows 5320 , 5330 , 5340 , and 5350 are not the same application as the application provided on the main multi-view window 5310 , but an application related to the application or not. may be

Taking the latter case as an example, when the main multi-view window 5310 is moved from the lower left side of the screen to the center as in the above-described embodiment, the second sub multi-view window is automatically provided accordingly. am. For example, when the second sub multi-view windows 5320 and 5330 that were not provided when the main multi-view window is provided move the main multi-view window 5310 in the screen, one or more The second sub multi-view windows 5320 and 5330 are automatically provided. These second sub multi-view windows 5320 and 5330 may be provided in a different way from the first sub multi-view windows 5340 and 5350, and the movement position or movement of the main multi-view window The number or positions of the provided sub multi-view windows may vary according to the distance movement area.

Referring to FIG. 54 , it is assumed that a user is watching a baseball application through a main screen in a digital device. In this case, when the seventh episode 5410 is currently in progress, the digital device may provide the multi-view mode according to a user's request or automatically. As described above, in providing the multi-view mode, an indicator indicating that the multi-view mode can be provided or an indicator for entering the multi-view mode is provided in one area on the main screen, and is shown in FIG. 54 through the provided indicator As described above, it is possible to provide sub multi-view windows including application data for 6 times 5420 , 5 times 5430 , 4 times 5440 , 3 times 5450 , etc. before the 7 times currently in progress. . In FIG. 54 , the sub multi-view window 5420 for 6 times immediately before the current 7th is provided in the main multi-view window 5410 , and the sub multi-view windows 5420 for 5, 4, and 3 times are sequentially provided around it. The multi-view windows 5430 , 5440 , and 5450 may be arranged and provided. However, the opposite may be true. In addition, although not shown, if the user selects any one of the sub multi-view windows, the application on the selected sub multi-view window may be played at the corresponding location or switched/interchanged with application data on the main screen. . In particular, in the latter case, for example, when at least one resource is insufficient in providing the application data on the selected sub multi-view window, it is possible to switch/interchange with the application data of the main screen. In addition, even if it is not for the above reasons, when switching/interchanging is performed between the application data currently being provided and the application data being provided in the multi-view windows, the current application data is continuously switched/interfaced on the window. It can then play or stop, but also perform a time-shift function. When the time-shifted application data is switched/interchanged back to the main screen, the user is provided with a choice of whether to start playback from the time-shift starting point or from the current playback part, and according to the selection can provide

Referring to FIG. 55 , application data is output to the main screen 5510 , and multi-view windows are also provided. Here, each of the multi-view windows may perform a group function, for example. For example, if the application data being provided on the main screen 5510 is the 8th episode of season 4, the remaining episodes of the application of the season are provided to the first multi-view window 5520 , and the remaining multi-view window 5530 is provided. , 5540, 5550) provide episodes from different seasons. In this case, an indicator for accessing an episode of a corresponding season is provided on each multi-view window, and when the provided indicator is accessed, a list of episodes of the corresponding season is provided. Meanwhile, each multi-view window may output data for identifying a corresponding season, image data, and data on the number of episodes belonging to the corresponding season together. Meanwhile, when a predetermined multi-view window itself is selected, application data of the corresponding season may be automatically reproduced on the corresponding window or on the main screen sequentially in the order of the episodes. On the other hand, when a predetermined time elapses after positioning the pointer on the corresponding multi-view window, an episode list is automatically provided, and the corresponding episode may be automatically played on the corresponding window or on the main screen according to the selection.

Referring to FIG. 56 , similarly to FIG. 55 , each multi-view window is configured in a group unit, and when a predetermined multi-view window 5610 is selected, episodes 5620 and 5630 belonging to the corresponding season are displayed in the corresponding window. may be provided as shown in the form of a sub-window of

Referring to FIG. 57 , each multi-view window provides an application being provided on the main screen, that is, when episode 8 of season 4 is provided, previous episodes of the season and episodes of the season that can be provided later are provided. At this time, in the first multi-view window, the previous episode of the episode of the application currently being provided on the main screen is provided. In this case, the first multi-view window may provide an indicator for accessing the plurality of season episodes of the application being provided on the main screen. The remaining multi-view windows provide episodes that can be provided later than the season episodes of the application being provided on the main screen. At this time, when the episodes have never been watched or recorded, the function icons for reservation recording/viewing reservation (5712/5714) according to access to the multi-view window 5710 are displayed in the multi-view window ( 5710), so you can immediately perform a function or make a reservation for execution.

58 is similar to the above-described embodiments, but for other contents, in the previous embodiments, when an additional season or additional episode exists, it is provided through a multi-view window, so there are many areas that cover the screen, so the application on the main screen It may cause inconvenience to viewing. Accordingly, in FIG. 58 , accessible seasons or episodes are provided in the form of icons around the main multi-view window, so that screen blocking is minimized, but function execution can be more easily controlled.

Although not shown in FIG. 56 , the sub multi-view window may provide an indicator as an interface for providing an additional sub multi-view window like the main multi-view window. For example, the one or more second sub multi-view windows 5620 and 5630 provided in FIG. 56 may be provided according to the access of an indicator provided on the first sub multi-view window 5610. .

59 is a flowchart illustrating a method of controlling a digital device supporting a multi-view mode according to an embodiment of the present invention.

A method of processing multi-view data in a digital device may be as follows.

The digital device receives the content and the signaling data for the content (S5902), decodes the content and the signaling data (S5904), and outputs the decoded content to the screen (S5906).

The digital device receives a user input requesting output of the menu launcher (S5908), and outputs one or more menu items on the screen according to the user input requesting output of the menu launcher (S5910).

The digital device receives a user input requesting to output the multi-view content (S5912), and according to the first user input, overlays a window for outputting the multi-view content on a predetermined area on the screen on which the content is being output and outputs it (S5914).

The output window may include an interface for outputting one or more sub-windows, and according to selection of the interface, the one or more sub-windows may be output together with the window on a screen area.

In the above, the content output through one or more sub-windows may be content related to at least one of the content and multi-view content output through the window.

In the above, the first user input may be a multi-view function icon selection signal provided around the selected menu item when any one of one or more menu items belonging to the output menu launcher is selected.

Meanwhile, the window for outputting the multi-view content may be output to an area that does not overlap with the output menu launcher, and the one or more sub-windows may be output in a size different from that of the window.

In the above, the window may interact with the content being provided on the screen according to drag and drop, and the one or more sub-windows may be output to a screen area adjacent to the window or an edge of the screen .

In addition, when the output window is changed from the initial position in the screen area to another position, one or more sub-windows may be additionally output in addition to the sub-windows output according to the interface selection based on the changed position. .

Meanwhile, at least one of the output window and sub-windows is configured in a group unit including content of the same category. In the present specification, various data that can be processed by a digital device may be provided in the main multi-view and sub multi-view windows. In addition, external input devices such as DVI/HDMI or game devices or web browser formats can also be supported. Meanwhile, when a web browser is provided on the main multi-view window, the sub multi-view window may or may not be recommended or provided in the form of a web browser.

In addition, when the application being provided on the main multi-view window is switched/interchanged with the application being provided on the main screen or the sub multi-view window, the multi-view windows other than the corresponding multi-view window do not change. Alternatively, the sub multi-view window may be entirely changed according to an application on the switched/interchanged main multi-view window.

In this specification, the description of each drawing is limited to the description of the drawing for convenience, but the technical spirit of the present invention is not limited thereto. Therefore, even if it is not shown or described, the contents that do not conflict with each other among the contents shown in the drawings or mentioned in the description for the drawings are applied to the related drawings and the descriptions of the related drawings as they are or combined appropriately. Applicable, this is also included in the technical idea of the present invention.

According to various embodiments of the present disclosure, a digital device enables a user to access and use data such as functions and contents desired by a user more easily and quickly than in the prior art, and interferes with the viewing of content being output on the main screen, that is, content being viewed previously. While minimizing By providing a menu screen that is more intuitive and user-friendly than in the past, as well as providing and controlling it easily, digital devices can be used easily and conveniently by anyone.

In the digital device and digital device control method according to the present invention, the configuration and method of the embodiments described above are not limitedly applicable, but the embodiments are all or part of each embodiment so that various modifications can be made. may be selectively combined.

Meanwhile, the method of operating a digital device of the present invention can be implemented as processor-readable codes on a processor-readable recording medium provided in the digital device. The processor-readable recording medium includes all types of recording devices in which data readable by the processor is stored. Examples of the processor-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and is implemented in the form of a carrier wave such as transmission through the Internet. It also includes being In addition, the processor-readable recording medium is distributed in a computer system connected to a network, so that the processor-readable code can be stored and executed in a distributed manner.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention pertains without departing from the gist of the present invention as claimed in the claims In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

The present invention relates to a digital device, and is applicable to various types of digital devices, and thus has industrial applicability.

4410: indicator
4420, 4430, 4440: sub multi-view window

Claims (8)

In a digital device,
a receiver for receiving content and signaling data for the content;
a user input receiving unit for receiving a first user input requesting output of multi-view content;
a decoder for decoding the content and signaling data;
a control unit that controls to output the decoded content to a screen, and controls to overlay and output a window for outputting the multi-view content on a predetermined area on the screen where the content is being output according to the first user input; and
An output unit for outputting the content and the window,
The control unit is
When the pointer is positioned within the window, control an interface for outputting one or more sub-windows to output,
controlling the one or more sub-windows to be output to a screen area together with the window according to the selection of the interface;
The user input receiving unit,
receive user input requesting menu launcher output;
The control unit is
controlling to output one or more menu items on the screen according to a user input of the menu launcher output request;
The first user input is
and a multi-view function icon selection signal provided around the selected menu item when any one of one or more menu items belonging to the output menu launcher is selected. According to claim 1,
The content output through the one or more sub-windows,
The digital device, characterized in that the content is related to at least one of the content and the multi-view content output through the window. According to claim 1,
The control unit is
The digital device according to claim 1, wherein the window for outputting the multi-view content is controlled to be output to an area that does not overlap with the output menu launcher. According to claim 1,
The control unit is
The digital device, characterized in that the one or more sub-windows are controlled to output a size different from that of the window. According to claim 1,
The control unit is
A digital device, characterized in that by dragging and dropping the window, controlling to be exchanged with the content being provided on the screen. According to claim 1,
The control unit is
The digital device of claim 1, wherein the one or more sub-windows are output to a screen area adjacent to the window or an edge of the screen. According to claim 1,
The control unit is
At least one of the output window and sub-windows is configured in a group unit including content of the same category, and when the pointer is positioned on the corresponding window or sub-window, the group-unit content list is displayed in the form of a window or an icon Digital device, characterized in that the control to output on the window. In a digital device,
a receiver for receiving content and signaling data for the content;
a user input receiving unit for receiving a first user input requesting output of multi-view content;
a decoder for decoding the content and signaling data;
a control unit that controls to output the decoded content to a screen, and controls to overlay and output a window for outputting the multi-view content on a predetermined area on the screen where the content is being output according to the first user input; and
An output unit for outputting the content and the window,
The control unit is
When the pointer is positioned within the window, control an interface for outputting one or more sub-windows to output,
controlling the one or more sub-windows to be output to a screen area together with the window according to the selection of the interface;
When the output window is changed from the initial position in the screen area to another position, control to output one or more sub-windows in addition to the sub-windows output according to the interface selection based on the changed position digital device.

Images