WO2016178442A1

WO2016178442A1 - Digital device and digital device control method

Info

Publication number: WO2016178442A1
Application number: PCT/KR2015/004506
Authority: WO
Inventors: 임세준; 정영경; 임형진; 이기호
Original assignee: 엘지전자 주식회사
Priority date: 2015-05-06
Filing date: 2015-05-06
Publication date: 2016-11-10
Also published as: KR102418149B1; KR20220038179A; KR20180002599A

Abstract

A digital device and a digital device control method are disclosed, wherein the digital device according to one embodiment of the present invention comprises: a reception unit for receiving content and signaling data of the content; a user input reception unit for receiving a first user input for a menu call; a decoder for decoding the content and the signaling data; a control unit for controlling the decoded content such that the decoded content is output on a screen and controlling a menu screen such that the menu screen is output, in an overlay manner, in a predetermined region on the screen on which the content is outputted, according to the first user input; and an output unit for outputting the content and the menu screen, wherein the control unit controls the menu screen such that the menu screen is output, the menu screen including an application list or a content list including at least one piece of content related to the content being outputted onto the screen, controls a GUI such that the GUI, for switching a menu screen configuration of the outputted menu screen when a pointer in the menu screen is located or hovers at a predetermined region, is output, and controls the menu screen configuration such that the menu screen configuration is switched according to a user's selection from the outputted GUI.

Description

Digital Devices and Digital Device Control Methods

The present invention relates to a digital device, and more particularly, to a digital device and a method for maximizing user's convenience through a more intuitive and quick content access by constructing a content-oriented menu.

There is a rapid transition from analog to digital systems. In this case, the digital system is more resistant to external noise than conventional analog systems, and thus has low data loss, is advantageous in error correction, and can provide bidirectional services.

Conventional digital TVs receive a control signal through control means such as a remote controller provided in pairs at the time of manufacture, or through key buttons provided on an external frame or front panel of the digital TV. Was performed and the result was output. For example, in the conventional digital TV, when a menu is requested through the control means or the key button, a menu screen including a predetermined format or contents at the time of manufacture is provided on the screen. In other words, the conventional digital TV outputs only the preset contents to the requested menu screen. Therefore, in order to search and use a function or content desired by a user through a digital TV, the user must go through various depths or through separate operations or function buttons. Inconvenience and difficulty in quickly accessing and using a desired function or content in such an environment.

In order to solve the above problems, the digital device according to the present invention is to make it easier and faster to access and use data, such as a function, content, etc. desired by the user than one conventional.

The digital device according to the present invention provides a method for easily and quickly accessing and using desired data through a minimum depth or screen change on a called menu while minimizing interruption in viewing content being displayed on a main screen, that is, content being previously viewed. Make it another task.

The digital device according to the present invention can easily provide a multi-view screen based on a Web OS platform while the main content or / and a menu launcher is being output. Letting him control is another task.

Another object of the digital device according to the present invention is to allow anyone to easily and conveniently use the digital device by configuring and providing a menu screen that is more intuitive and maximizing convenience than in the related art.

The technical problem to be achieved in the present invention is not limited to the above technical problem (s), and other technical problems which are not mentioned are obvious to those skilled in the art from the contents disclosed herein. Can be understood.

In order to solve the above problems, a digital device according to an embodiment of the present invention, the receiving unit for receiving the content and the signaling data for the content, a user input receiving unit for receiving a first user input for requesting the multi-view content output, A decoder for decoding the content and signaling data, controlling to output the decoded content on a screen, and overlaying a window for outputting the multi-view content on a predetermined area on the screen where the content is output according to the first user input And a control unit for controlling the output and an output unit for outputting the content and the window, wherein the control unit controls to output an interface for outputting one or more sub-windows when the pointer is located in the window, and to select the interface. According to the above one or Controls such that the output of one sub-window in the screen area with the window.

A method of processing multi-view data in a digital device according to an embodiment of the present invention may include receiving content and signaling data for the content, decoding the content and signaling data, and displaying the decoded content on a screen. Outputting, receiving a first user input for requesting multi-view content output, and overlaying and outputting a window for outputting the multi-view content on a predetermined area on a screen where the content is output according to the first user input And the output window includes an interface for outputting one or more subwindows, and outputs the one or more subwindows together with the window to a screen area according to the selection of the interface. .

The solution (s) disclosed through the present specification are not limited to the above, and other solution means which are not mentioned are obvious to those skilled in the art from the contents disclosed herein. Can be understood.

According to the present invention, the following effects are obtained.

According to one of various embodiments of the present disclosure, a digital device has an effect of allowing a user to easily access and use data, such as a desired function and content, more easily and faster than before.

According to another of various embodiments of the present disclosure, the digital device may display desired data through a minimum depth or screen change on a called menu while minimizing interruption in viewing content that is being output on the main screen, that is, previously viewed content. It has the effect of making it quick and easy to access and use.

According to another embodiment of the various embodiments of the present disclosure, a digital device may be configured to easily provide and control a multi-view screen based on a webOS platform while a main content or / and menu launcher is being output. It works.

According to another embodiment of the various embodiments of the present disclosure, the digital device has an effect that makes it easy and convenient for anyone to use the digital device by configuring and providing a menu screen that is more intuitive and maximized convenience than in the related art.

The effects of the present invention are not limited to the above-described effect (s), and other effects not mentioned will be clearly understood by those skilled in the art from the contents disclosed herein. .

1 is a view for schematically illustrating a service system including a digital device according to an embodiment of the present invention;

2 is a block diagram illustrating a digital device according to one embodiment of the present invention;

3 is a block diagram illustrating a digital device according to another embodiment of the present invention;

4 is a diagram illustrating a WebOS architecture according to an embodiment of the present invention;

FIG. 5 is a diagram for explaining the architecture of a WebOS device according to one embodiment of the present invention; FIG.

FIG. 6 is a diagram for explaining a graphic composition flow in a webOS device according to one embodiment of the present invention; FIG.

7 is a diagram illustrating a media server according to one embodiment of the present invention;

8 is a block diagram illustrating a configuration block of a media server according to one embodiment of the present invention;

9 is a diagram illustrating a relationship between a media server and a TV service according to an exemplary embodiment.

10 illustrates a screen of a digital device on which content is output;

11 illustrates a digital device for outputting a requested menu with a menu output according to an embodiment of the present invention;

12 to 21 are diagrams for explaining the current mode menu configuration and control according to the present invention;

22 to 27 are diagrams for explaining the past mode menu configuration and control according to the present invention;

28 to 35 are diagrams for explaining the future mode menu configuration and control according to the present invention;

36 is a diagram illustrating an example of a digital service system according to one embodiment of the present invention;

FIG. 37 is a diagram for explaining an embodiment of outputting a menu requested from a digital device 3710 on a mobile device according to the present invention; FIG.

FIG. 38 is a diagram illustrating an embodiment of a menu launcher configuration in the form of a mobile device according to the present invention; FIG.

39 is a flowchart illustrating a menu data processing method in a digital device according to the present invention;

40 is a diagram for explaining a sound control method in screen division in relation to multi-view mode data processing;

41 and 42 illustrate a method of entering a multi-view mode according to the present invention;

43 illustrates a multi-view entry screen according to an embodiment of the present invention;

44 is a diagram for explaining a multi-view window control method according to another embodiment of the present invention;

45 is a diagram illustrating a detailed control method of a multi-view window according to one embodiment of the present invention;

46 and 47 are diagrams for explaining a detailed reproduction control method of a multi-view window according to one embodiment of the present invention;

48 is a diagram to describe a method of exiting a multi-view mode according to the present invention;

49 to 58 are diagrams for explaining a sub multi-view operation control method according to the present invention; And

FIG. 59 is a flowchart illustrating a control method of a digital device supporting a multi-view mode according to an embodiment of the present invention.

Hereinafter, various embodiment (s) of a digital device and a method of controlling the digital device according to the present invention will be described in detail with reference to the drawing (s).

The suffixes "module", "part", and the like for the components used herein are given only in consideration of ease of preparation of the specification, and both may be used as needed. In addition, even when written in ordinal numbers such as "first-first", "second-second", and the like, it is not intended to mean the order, but merely for the convenience of explanation of the term, and is not limited to such terms or ordinal numbers.

In addition, the terms used in this specification, in consideration of the function according to the technical spirit of the present invention was selected as widely used general terms as possible, but this is in accordance with the intention or custom of the person skilled in the art or the emergence of new technologies, etc. Can vary. However, in some cases, the term is arbitrarily selected by the applicant, but the meaning will be described in the related description. Therefore, it is to be understood that the terminology should be interpreted not on the basis of merely the name, but on the actual meaning thereof and the contents described throughout this specification.

On the other hand, the contents described in the specification or / and the drawings are not limited thereto as a preferred embodiment according to the present invention, the scope of the rights should be determined through the claims.

As used herein, the term "digital device" is used herein to mean all devices capable of performing at least one or more of transmission, reception, processing, and output of data, content, services, applications, and the like. The digital device may be paired or connected (hereinafter referred to as 'pairing') with another digital device, an external server, or the like through a wire / wireless network, and transmit / receive predetermined data therethrough. Can be received. The predetermined data also includes control data. If necessary, the data can be properly converted before its transmission / reception. The digital device includes, for example, a standing device such as a Network TV, a Hybrid Broadcast Broadband TV (HBBTV), a Smart TV, an Internet Protocol TV (IPTV), a PC, or the like, and a PDA ( A mobile device or a handheld device, such as a personal digital assistant, a smart phone, a tablet PC, a notebook, a wearable device, and the like, may all be included. In the present specification, a digital TV and a mobile device in FIG. 2 and FIG. In addition, the digital device described herein may be a digital signage, monitor, or display device composed only of panels, and may be a set-top box (STB). Or may be part of one service system in combination with a server.

Meanwhile, the term “wired / wireless network” described herein refers to a communication network supporting various communication standards or protocols for pairing and / or transmitting and receiving data between digital devices or digital devices and external servers. Such wired / wireless networks include all communication networks that are currently or will be supported by the specification and are capable of supporting one or more communication protocols for them. Such wired and wireless networks include, for example, Universal Serial Bus (USB), Composite Video Banking Sync (CVBS), Component, S-Video (Analog), Digital Visual Interface (DVI), High Definition Multimedia Interface (HDMI), Network for wired connection such as RGB, D-SUB and communication standard or protocol therefor, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), Zigbee (ZigBee), Digital Living Network Alliance (DLNA), Wireless LAN (WLAN) (Wi-Fi), Wireless broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), LTE / LTE It may be formed by a network for a wireless connection such as Long Term Evolution / LTE-Advanced (A), Wi-Fi Direct, and a communication standard or protocol therefor.

In the present specification, when only referred to as a digital device, the meaning may mean a fixed or mobile device depending on the context, and may be used to include both unless specifically mentioned.

On the other hand, the digital device is, for example, an intelligent device that supports a broadcast receiving function, a computer function or support, at least one external input, and the like, and includes e-mail and web browsing through the wired / wireless network described above. , Banking, games, applications, and more. In addition, the digital device may include an interface for supporting at least one input or control means such as a handwritten input device, a touch-screen, a spatial remote controller, and the like. have.

In addition, the digital device may use a standard general-purpose operating system (OS), and in particular, the Linux OS may be included therein. On the other hand, the digital device described in the present specification, an example of processing data, services, applications, etc. based on the Web OS (Web OS) platform. Therefore, the digital device can add, delete, modify, and update various services or applications on a general-purpose OS kernel or Linux kernel, thereby constructing a more user-friendly environment. Can be provided.

Meanwhile, the above-described digital device may receive and process an external input, wherein the external input is connected to an external input device, that is, the above-described digital device through a wired / wireless network to transmit / receive data. And all input means or digital devices which can be processed. For example, the external input may be a gaming device such as a high-definition multimedia interface (HDMI), a playstation or an X-box, a smartphone, a tablet PC, a pocket photo, or the like. Digital devices such as printing devices, smart TVs, Blu-ray device devices and the like.

In addition, the term "server" described herein refers to a digital device or system for supplying data to or receiving data from the above-mentioned digital device, that is, a client, and also referred to as a processor. do. For example, a portal server for providing a web page, a web content or a web service, an advertising server for providing advertising data, Providing a content server providing content, an SNS server providing a social network service (SNS), a service server provided by a manufacturer, and providing a video on demand (VOD) or streaming service It may include a multi-channel video programming distributor (MVDP), a service server for providing a pay service, a cloud server, and the like.

In addition, hereinafter, even if described only as an application for convenience of description in the present specification, the meaning may be a meaning including not only an application but also a service based on the context, and the like. web applications).

According to an embodiment of the present invention, a digital device includes a receiver for receiving content and signaling data for the content, a user input receiving unit for receiving a first user input for requesting multi-view content output, and decoding the content and signaling data. A decoder configured to control the output of the decoded content on a screen, and to output an overlay of a window for outputting the multi-view content in a predetermined area on a screen on which the content is output according to the first user input; And an output unit for outputting a content and a window, wherein the control unit controls to output an interface for outputting one or more sub-windows when the pointer is located in the window, and according to the interface selection. Remind windows Controls output to the screen area with a window.

The content output through the one or more sub-windows may be content associated with at least one of the content and the multi-view content output through the window. The user input receiver may receive a user input for requesting a menu launcher output, and the controller may control to output one or more menu items on the screen according to the menu launcher output request user input. .

The first user input may be a multi-view function icon selection signal provided around the selected menu item as one of one or more menu items belonging to the output menu launcher is selected.

The controller may control to output the window for outputting the multi-view content to an area not overlapping with the output menu launcher, and to output the size of the one or more sub-windows differently from the window. As the window is dragged and dropped, it may be controlled to interact with content being provided on the screen. The controller may control the one or more sub-windows to be output to a screen area adjacent to the window or to an edge of the screen, and when the output window is changed from an initial position to another position in the screen area. In addition to the sub-windows output according to the interface selection based on the changed position, one or more sub-windows may be output. The at least one of the output window and the sub-windows may include content of the same category. If the pointer is located in the corresponding window or sub-window, the content list in the group unit can be controlled to be output on the corresponding window in the form of a window or an icon. The present invention in more detail If the description is as follows.

1 is a diagram schematically illustrating a service system including a digital device according to an embodiment of the present invention.

Referring to FIG. 1, the service system includes a content provider 10, a service provider 20, a network provider 30, and a home network end user (HNED). (Customer) 40. Here, the HNED 40 comprises, for example, a client 100, ie a digital device according to the invention.

The content provider 10 produces and provides various contents. As shown in FIG. 1, a terrestrial broadcast sender, a cable SO (cable system operator) or MSO (multiple SO), a satellite broadcast sender, various Internet broadcast senders, and an individual may be used as the content provider 10. Content providers and the like. The content provider 10 may produce and provide various services or applications in addition to broadcast content.

The service provider 20 service packetizes the content produced by the content provider 10 and provides it to the HNED 40. For example, the service provider 20 packages at least one or more of contents produced by the first terrestrial broadcast, the second terrestrial broadcast, the cable MSO, satellite broadcast, various Internet broadcasts, applications, and the like for the service, and HNED. Provide 40.

The service provider 20 provides a service to the client 100 in a uni-cast or multi-cast manner. Meanwhile, the service provider 20 may transmit data to a plurality of clients 100 registered in advance. For this, the service provider 20 may use an Internet Group Management Protocol (IGMP) protocol.

The content provider 10 and the service provider 20 described above may be the same entity. For example, the content produced by the content provider 10 may be packaged as a service and provided to the HNED 40 to perform the functions of the service provider 20 together or vice versa.

The network provider 30 provides a network for data exchange between the content provider 10 or / and the service provider 20 and the client 100.

The client 100 is a consumer belonging to the HNED 40, and for example, establishes a home network through the network provider 30 to receive data, and receives data about various services or applications such as VoD and streaming. You can also send / receive.

Meanwhile, the content provider 10 and / or the service provider 20 in the service system may use conditional access or content protection means to protect the transmitted content. Accordingly, the client 100 may use a processing means such as a cable card (or point of deployment) or a downloadable casing (DCAS) in response to the limited reception or content protection.

In addition, the client 100 may also use a bidirectional service through a network. Accordingly, the client 100 may perform a role or function of the content provider, and the service provider 20 may receive it and transmit it to another client.

In FIG. 1, the content provider 10 and / or the service provider 20 may be a server that provides a service described later herein. In this case, the server may mean to own or include the network provider 30 as necessary. Although not specifically mentioned below, the service or service data includes not only services or applications received from the outside described above but also internal services or applications, and these services or applications are services for the client 100 based on Web OS (Web OS). To application data.

2 is a block diagram illustrating a digital device according to an embodiment of the present invention.

Hereinafter, the digital device described herein corresponds to the client 100 of FIG. 1.

The digital device 200 includes a network interface 201, a TCP / IP manager 202, a service delivery manager 203, an SI decoder 204, A demux or demultiplexer 205, an audio decoder 206, a video decoder 207, a display A / V and OSD module 208, a service control manager (service control manager) 209, service discovery manager 210, SI & metadata DB 211, metadata manager 212, service manager 213, And a UI manager 214.

The network interface unit 201 is an IP packet (s) (Internet Protocol (IP) packet (s)) or IP datagram (s) (hereinafter referred to as' IP packet (s) '). For example, the network interface unit 201 may receive a service, an application, content, additional information, and the like from the service provider 20 of FIG. 1 through a network. The additional information may include SI (System Information) information. The network interface 201 may coexist with a tuner, a demodulator, or the like, or may be replaced with a tuner, a demodulator, or the like. The network interface unit 201 may include, for example, an Ethernet terminal for connection with a wired network, and for example, a wireless LAN (WLAN) for connection with a wireless network. Fi, Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), and High Speed Downlink Packet Access (HSDPA) communication standards. On the other hand, the network interface unit 201 can access a predetermined web page through a connected network or another network linked to the connected network. That is, by accessing a predetermined web page through the network, it is possible to send or receive data with the server. In addition, it may receive content or data provided by a content provider or a network operator. That is, content such as a movie, an advertisement, a game, a VOD, a broadcast signal, and related information provided from a content provider or a network provider may be received through a network. In addition, it is possible to receive the update information and the update file of the firmware (firmware) provided by the network operator. It may also transmit data to the Internet or content provider or network operator. In addition, the network interface unit 201 may select and receive a desired application from among applications that are open through the network.

The TCP / IP manager 202 may be configured to transfer packets between the source and the destination for IP packets received by the digital device 200 and IP packets transmitted by the digital device 200. involved in packet delivery). The TCP / IP manager 202 classifies the received packet (s) to correspond to an appropriate protocol, and classifies the classified packet (s) into a service delivery manager 205, a service discovery manager 210, and a service control manager ( 209, the metadata manager 212, and the like.

The service delivery manager 203 is in charge of controlling service data. For example, the service delivery manager 203 may use RTP / RTCP when controlling real-time streaming data. When transmitting the real-time streaming data using the RTP, the service delivery manager 203 parses the received data packet according to the RTP and transmits it to the demultiplexer 205 or controls the service manager 213. In accordance with the SI & metadata database 211. Meanwhile, the service delivery manager 203 feeds back the network reception information to a server that provides a service using RTCP.

The demultiplexer 205 demultiplexes the received packet into audio, video, SI data, and the like, and transmits the demultiplexer 205 to the audio / video decoders 206/207 and the SI decoder 204, respectively.

The SI decoder 204 decodes the demultiplexed SI data. The SI data includes PSI (Program Specific Information), PSIP (Program and System Information Protocol), DVB-SI (Digital Video Broadcasting-Service Information), DTMB / CMMB (Digital Television Terrestrial Multimedia Broadcasting / Coding Mobile Multimedia Broadcasting), and the like. It can be either. In addition, the SI decoder 204 may store the SI data to be decoded in the SI & metadata database 211. The SI data stored in the SI & database 211 may be read out and used as necessary, for example, by a user's request.

The audio / video decoder 206/207 decodes each demultiplexed audio data and video data. The decoded audio data and video data are provided to the user through the display unit 208.

The application manager may include, for example, the UI manager 214 and the service manager 213 and perform a control function of the digital device 200. In other words, the application manager may manage the overall state of the digital device 200, provide a user interface (UI), and manage other managers.

The UI manager 214 provides a Graphic User Interface (UI) / UI for a user using an OSD (On Screen Display), etc., and receives a key input from the user to perform a device operation according to the input. For example, the UI manager 214 transmits the key input signal to the service manager 213 when receiving a key input related to channel selection from the user.

The service manager 213 controls a manager associated with a service such as a service delivery manager 203, a service discovery manager 210, a service control manager 209, and a metadata manager 212.

In addition, the service manager 213 generates a channel map and controls the channel selection using the generated channel map according to the key input received from the UI manager 214. The service manager 213 receives service information from the SI decoder 204 and sets the audio / video packet identifier (PID) of the selected channel to the demultiplexer 205. The PID set as described above may be used in the above demultiplexing process. Accordingly, the demultiplexer 205 filters (PID or section filtering) audio data, video data, and SI data by using the PID.

The service discovery manager 210 provides information necessary for selecting a service provider that provides a service. Upon receiving a signal regarding channel selection from the service manager 213, the service discovery manager 210 searches for a service using the information.

The service control manager 209 is responsible for selecting and controlling services. For example, the service control manager 209 uses IGMP or RTSP when the user selects a live broadcasting service such as a conventional broadcasting method, and selects a service such as VOD (Video on Demand). The RTSP is used to perform service selection and control. The RTSP protocol may provide a trick mode for real time streaming. In addition, the service control manager 209 may initialize and manage a session through the IMS gateway 250 using an IP Multimedia Subsystem (IMS) or a Session Initiation Protocol (SIP). The protocols are one embodiment, and other protocols may be used depending on implementation.

The metadata manager 212 manages metadata associated with the service and stores the metadata in the SI & metadata database 211.

The SI & metadata database 211 stores service information decoded by the SI decoder 204, metadata managed by the metadata manager 212, and information necessary to select a service provider provided by the service discovery manager 210. Save it. In addition, the SI & metadata database 211 can store set-up data and the like for the system.

The SI & metadata database 211 may be implemented using non-volatile memory (NVRAM), flash memory, or the like.

Meanwhile, the IMS gateway 250 is a gateway that collects functions necessary for accessing an IMS-based IPTV service.

The storage unit (not shown) may store a program for each signal processing and control, or may store a signal-processed video, audio, or data signal. In addition, the storage unit may perform a function for temporarily storing an image, audio, or data signal input from the external device interface unit or the network interface unit 201. The storage unit may store information about a predetermined broadcast channel through a channel storage function. The storage unit may store an application or an application list input from the external device interface unit or the network interface unit 201. In addition, the storage unit may store webOS platform data described below. The storage unit may be, for example, a flash memory type, a hard disk type, a multimedia card micro type, a memory of a card type (eg SD or XD memory, etc.), The storage medium may include at least one type of storage medium (RAM), ROM (EEPROM, etc.). The digital device 200 may provide a user with a content file (video file, still image file, music file, document file, application file, etc.) stored in the storage unit.

The digital device 200 described above may be a digital broadcast receiver capable of processing a fixed or mobile ATSC or DVB digital broadcast signal. In addition, the digital device according to the present invention may omit some of the configurations of the illustrated configurations, or may further include components not shown on the contrary. On the other hand, unlike the above, the digital device does not include a tuner and a demodulator, and may receive and play content through a network interface unit or an external device interface unit.

3 is a block diagram illustrating a digital device according to another embodiment of the present invention.

In FIG. 2, the digital device, which is a fixed device, has been described as an embodiment of the digital device, and FIG.

Referring to FIG. 3, the mobile device 300 may include a wireless communication unit 310, an A / V input unit 320, a user input unit 330, a sensing unit 340, an output unit 350, The memory 360 may include an interface unit 370, a controller 380, a power supply unit 390, and the like.

Hereinafter, each component will be described in detail.

The wireless communication unit 310 may include one or more modules that enable wireless communication between the mobile device 300 and the wireless communication system or between the mobile device and the network in which the mobile device is located. For example, the wireless communication unit 310 may include a broadcast receiving module 311, a mobile communication module 312, a wireless internet module 313, a short range communication module 314, a location information module 315, and the like. .

The broadcast receiving module 311 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel. Here, the broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may mean a server that generates and transmits a broadcast signal and / or broadcast related information or a server that receives a previously generated broadcast signal and / or broadcast related information and transmits the same to a terminal. The broadcast signal may include not only a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, but also a broadcast signal having a data broadcast signal combined with a TV broadcast signal or a radio broadcast signal. The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 312. The broadcast related information may exist in various forms, for example, in the form of an electronic program guide (EPG) or an electronic service guide (ESG). The broadcast receiving module 311 may be, for example, ATSC, DVB-T (Digital Video Broadcasting-Terrestrial), DVB-S (Satellite), MediaFLO (Media Forward Link Only), DVB-H (Handheld), ISDB-T ( Digital broadcasting signals may be received using a digital broadcasting system such as Integrated Services Digital Broadcast-Terrestrial. Of course, the broadcast receiving module 311 may be configured to be suitable for not only the above-described digital broadcasting system but also other broadcasting systems. The broadcast signal and / or broadcast related information received through the broadcast receiving module 311 may be stored in the memory 360.

The mobile communication module 312 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice signal, a video call signal, or a text / multimedia message.

The wireless internet module 313 may include a module for wireless internet access and may be embedded or external to the mobile device 300. Wireless Internet technologies may include Wireless LAN (Wi-Fi), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), and the like.

The short range communication module 314 refers to a module for short range communication. Short range communication technology, such as Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, RS-232, RS-485, etc. This can be used.

The location information module 315 may be a module for acquiring location information of the mobile device 300, and may use a Global Position System (GPS) module as an example.

The A / V input unit 320 is for inputting audio or video signals, and may include a camera 321 and a microphone 322. The camera 321 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the imaging mode. The processed image frame may be displayed on the display unit 351.

The image frame processed by the camera 321 may be stored in the memory 360 or transmitted to the outside through the wireless communication unit 310. Two or more cameras 321 may be provided depending on the use environment.

The microphone 322 receives an external sound signal by a microphone in a call mode, a recording mode, a voice recognition mode, etc., and processes the external sound signal into electrical voice data. The processed voice data may be converted into a form transmittable to the mobile communication base station through the mobile communication module 312 and output in the call mode. The microphone 322 may be implemented with various noise removing algorithms for removing noise generated in the process of receiving an external sound signal.

The user input unit 330 generates input data for the user to control the operation of the terminal. The user input unit 330 may include a key pad, a dome switch, a touch pad (constant voltage / capacitance), a jog wheel, a jog switch, and the like.

The sensing unit 340 may determine the current state of the mobile device 300 such as an open / closed state of the mobile device 300, a location of the mobile device 300, presence or absence of user contact, orientation of the mobile device, acceleration / deceleration of the mobile device, and the like. The sensing unit generates a sensing signal for controlling the operation of the mobile device 300. For example, when the mobile device 300 is moved or tilted, the position or tilt of the mobile device may be sensed. Also, whether the power supply unit 390 is supplied with power or whether the interface unit 370 is coupled to an external device may be sensed. The sensing unit 340 may include a proximity sensor 341 including near field communication (NFC).

The output unit 350 is to generate an output related to visual, auditory or tactile senses, and may include a display unit 351, a sound output module 352, an alarm unit 353, a haptic module 354, and the like. have.

The display unit 351 displays (outputs) information processed by the mobile device 300. For example, when the mobile device is in the call mode, the UI or GUI related to the call is displayed. When the mobile device 300 is in a video call mode or a shooting mode, the mobile device 300 displays a captured image and / or a received image, UI, or GUI.

The display unit 351 may include a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display ( flexible display) and three-dimensional display.

Some of these displays can be configured to be transparent or light transmissive so that they can be seen from the outside. This may be referred to as a transparent display. A representative example of the transparent display is TOLED (Transparant OLED). The rear structure of the display unit 351 may also be configured as a light transmissive structure. With this structure, the user can see the object located behind the terminal body through the area occupied by the display unit 351 of the terminal body.

According to an implementation form of the mobile device 300, two or more display units 351 may exist. For example, a plurality of display units may be spaced apart or integrally disposed on one surface of the mobile device 300, or may be disposed on different surfaces.

When the display unit 351 and a sensor for detecting a touch operation (hereinafter referred to as a touch sensor) form a mutual layer structure (hereinafter referred to as a touch screen), the display unit 351 may input other than an output device. It can also be used as a device. The touch sensor may have, for example, a form of a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in pressure applied to a specific portion of the display unit 351 or capacitance generated at a specific portion of the display unit 351 into an electrical input signal. The touch sensor may be configured to detect not only the position and area of the touch but also the pressure at the touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and then transmits the corresponding data to the controller 380. As a result, the controller 380 may determine which area of the display unit 351 is touched.

The proximity sensor 341 may be disposed in an inner region of the mobile device surrounded by the touch screen or near the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object present in the vicinity without using a mechanical contact by using an electromagnetic force or infrared rays. Proximity sensors have a longer life and higher utilization than touch sensors.

Examples of the proximity sensor include a transmission photoelectric sensor, a direct reflection photoelectric sensor, a mirror reflection photoelectric sensor, a high frequency oscillation proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. When the touch screen is capacitive, the touch screen is configured to detect the proximity of the pointer by the change of the electric field according to the proximity of the pointer. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of allowing the pointer to be recognized without being in contact with the touch screen so that the pointer is located on the touch screen is referred to as a "proximity touch", and the touch The act of actually touching the pointer on the screen is called "contact touch." The position where the proximity touch is performed by the pointer on the touch screen refers to a position where the pointer is perpendicular to the touch screen when the pointer is in proximity proximity.

The proximity sensor detects a proximity touch and a proximity touch pattern (for example, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, and a proximity touch movement state). Information corresponding to the sensed proximity touch operation and proximity touch pattern may be output on the touch screen.

The sound output module 352 may output audio data received from the wireless communication unit 310 or stored in the memory 360 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output module 352 may output a sound signal related to a function (eg, a call signal reception sound, a message reception sound, etc.) performed by the mobile device 300. The sound output module 352 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 353 outputs a signal for notifying occurrence of an event of the mobile device 300. Examples of events occurring in the mobile device include call signal reception, message reception, key signal input, and touch input. The alarm unit 353 may output a signal for notifying the occurrence of an event by vibration, in addition to a video signal or an audio signal. The video signal or the audio signal may also be output through the display unit 351 or the audio output module 352, so that they 351 and 352 may be classified as part of the alarm unit 353.

The haptic module 354 generates various tactile effects that a user can feel. Vibration is a representative example of the haptic effect generated by the haptic module 354. The intensity and pattern of vibration generated by the haptic module 354 can be controlled. For example, different vibrations may be synthesized and output or may be sequentially output. In addition to the vibration, the haptic module 354 may be configured to provide a pin array that vertically moves with respect to the contact skin surface, a jetting force or suction force of air through the jetting or suction port, grazing to the skin surface, contact of the electrode, electrostatic force, and the like. Various tactile effects can be generated, such as effects due to the effects of cold / warm reproduction using an element that can absorb heat or generate heat. The haptic module 354 may not only deliver the haptic effect through direct contact, but also may implement the user to feel the haptic effect through muscle sensation such as a finger or an arm. The haptic module 354 may be provided with two or more according to the configuration aspect of the mobile device 300.

The memory 360 may store a program for the operation of the controller 380 and may temporarily store input / output data (for example, a phone book, a message, a still image, a video, etc.). The memory 360 may store data regarding vibration and sound of various patterns output when a touch input is input on the touch screen.

The memory 360 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM), Magnetic Memory, It may include a storage medium of at least one type of magnetic disk, optical disk. The mobile device 300 may operate in association with web storage that performs a storage function of the memory 360 on the Internet.

The interface unit 370 serves as a path to all external devices connected to the mobile device 300. The interface unit 370 receives data from an external device, receives power, transfers the power to each component inside the mobile device 300, or transmits data within the mobile device 300 to the external device. For example, wired / wireless headset port, external charger port, wired / wireless data port, memory card port, port for connecting a device with an identification module, audio input / output (I / O) port, The video I / O port, the earphone port, and the like may be included in the interface unit 370.

The identification module is a chip that stores various types of information for authenticating the usage rights of the mobile device 300, and includes a user identification module (UIM), a subscriber identification module (SIM), and a universal user authentication module (UI). Universal Subscriber Identity Module (USIM), and the like. A device equipped with an identification module (hereinafter referred to as an “identification device”) may be manufactured in the form of a smart card. Thus, the identification device can be connected with the terminal via a port.

When the mobile device 300 is connected to an external cradle, the interface unit 370 may be a path through which power from the cradle is supplied to the mobile device 300 or may be input by the user from the cradle. It may be a passage through which a command signal is transmitted to the mobile device. Various command signals or power input from the cradle may be operated as signals for recognizing that the mobile device is correctly mounted in the cradle.

The controller 380 typically controls the overall operation of the mobile device 300. The controller 380 performs, for example, related control and processing for voice call, data communication, video call, and the like. The controller 380 may include a multimedia module 381 for multimedia playback. The multimedia module 381 may be implemented in the controller 380 or may be implemented separately from the controller 380. The controller 380 may perform a pattern recognition process for recognizing a writing input or a drawing input performed on a touch-screen as a character and an image, respectively.

The power supply unit 390 receives an external power source and an internal power source under the control of the controller 380 to supply power for operation of each component.

Various embodiments described herein may be implemented in a recording medium readable by a computer or similar device using, for example, software, hardware or a combination thereof.

According to a hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and the like. It may be implemented using at least one of a processor, a controller, micro-controllers, microprocessors, and other electrical units for performing other functions. Examples may be implemented by the controller 380 itself.

According to the software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein. Software code may be implemented in software applications written in a suitable programming language. Here, the software code may be stored in the memory 360 and executed by the controller 380.

On the other hand, the mobile terminal can be extended to a wearable device that can be worn on the body beyond the user mainly holding in the hand. Such wearable devices include a smart watch, a smart glass, a head mounted display (HMD), and the like. Hereinafter, examples of the mobile terminal extended to the wearable device will be described.

The wearable device may be configured to exchange (or interlock) data with another mobile terminal 300. The short range communication module 314 may detect (or recognize) a wearable device that can communicate around the mobile terminal 300. Further, when the detected wearable device is a device that is authenticated to communicate with the mobile terminal 300, the controller 380 transmits at least a part of data processed by the mobile terminal 300 to the wearable device through the short range communication module 314. Can be sent to. Accordingly, the user may use data processed by the mobile terminal 300 through the wearable device. For example, when a call is received by the mobile terminal 300, a phone call may be performed through the wearable device, or when the message is received by the mobile terminal 300, the received message may be confirmed through the wearable device. .

The digital device described herein may be operated by a webOS platform. Hereinafter, a webOS-based configuration or algorithm or the like may be performed by the controller of the above-described digital device. Here, the control unit includes the above-described control unit and uses the concept broadly. Therefore, hereinafter, the configuration for the processing of WebOS-based or related services, applications, content, etc. in the digital device, the hardware or components including the related software (software), firmware (firmware), etc. to the controller (controller) Can be explained by naming.

The webOS based platform is designed to enhance development independence and function scalability by integrating services and applications based on the Luna-service bus, for example, and to develop applications based on the web application framework. Productivity can also be increased. In addition, multi-tasking may be supported by efficiently utilizing system resources through webOS processes and resource management.

Meanwhile, the webOS platform described in the present specification may be used in mobile devices such as mobile phones, smart phones, tablet pcs, notebooks, and wearable devices, as well as fixed devices such as PCs, TVs, and STBs. .

The architecture of software for digital devices is a monolithic structure that is based on conventional problem solving and market-dependent monolithic structures, and is a single process and closed product based on multi-threading technology. Afterwards, there was a difficulty in external application, and after that, we aimed for new platform-based development, and then layered and componentized by pursuing cost innovation and efficient UI and external application development through chip-set replacement. ), Which had a three-layered structure, add-on, single source product, and add-on structure for open applications. In recent years, the software architecture has been further developed to provide a modular architecture of functional units, to provide a Web Open API (Application Programming Interface) for the eco-system, and to provide a game engine. Modular design for the native open API (Native Open API), etc. is being made, and accordingly, it is generated as a multi-process structure based on the service structure.

4 is a diagram illustrating a webOS architecture according to an embodiment of the present invention.

Referring to Figure 4, the architecture of the WebOS platform will be described.

The platform may be largely classified into a kernel, a system library based web OS core platform, an application, a service, and the like.

The architecture of the WebOS platform is a layered structure, with the OS at the bottom layer, system library (s) at the next layer, and applications at the top. First, the lowest layer may include a Linux kernel as an OS layer and include Linux as an OS of the digital device. Above the OS layer, Board Support Package (BSP) / Hardware Abstraction Layer (HAL) layer, Web OS core modules layer, Service layer, Luna-Service bus layer In the bus layer, the Enyo framework / NDK / QT layer, and the top layer, the application layer is sequentially present. Meanwhile, some layers of the above-described webOS layer structure may be omitted, and a plurality of layers may be one layer or conversely, one layer may have a plurality of layer structures. The webOS core module layer is based on a Luna Surface Manager (LSM) that manages surface windows, etc., a System & Application Manager (SAM), a WebKit (WebKit) that manages execution and execution states of applications, etc. It may include a WAM (Web Application Manager) for managing a web application.

The LSM manages an application window displayed on the screen. The LSM manages the display hardware, provides a buffer for rendering the contents required by the applications, and synthesizes the rendering results of the plurality of applications on the screen. You can print

The SAM manages performance policies for various conditions of systems and applications.

WAM, on the other hand, is based on the Enyo Framework, which can be viewed as a web application as a basic application.

The service use of the application is made through the Luna-service bus, and the service can be newly registered on the bus, and the application can find and use the service that it needs.

The service layer may include services of various service levels, such as a TV service and a WebOS service. The webOS service may include a media server, Node.JS, and the like. In particular, the Node.JS service supports, for example, JavaScript.

WebOS services can communicate over the bus to Linux processes that implement function logic. It can be divided into four parts, which are migrated from the TV process and the existing TV to webOS or services that are differentiated from manufacturers, webOS common service and JavaScript developed and used through Node.js. It consists of a Node.js service.

The application layer may include all applications that can be supported in a digital device, such as a TV application, a showcase application, a native application, and a web application.

The application on the web OS may be classified into a web application, a Palm Development Kit (PDK) application, a Qt Meta Language or Qt Modeling Language (QML) application, and the like according to an implementation method. The web application is based on the WebKit engine and runs on the WAM Runtime. Such web applications may be based on the Enyo framework, or may be developed and executed based on general HTML5, Cascading Style Sheets (CSS), or JavaScript. The PDK application includes a third-party or native application developed in C / C ++ based on a PDK provided for an external developer. The PDK refers to a development library and a set of tools provided to enable a third party such as a game to develop a native application (C / C ++). For example, a PDK application can be used for the development of applications whose performance is important. The QML application is a Qt-based native application, and includes a basic application provided with a web OS platform such as a card view, a home dashboard, a virtual keyboard, and the like. Here, QML is a mark-up language in script form instead of C ++. Meanwhile, in the above, the native application refers to an application that is developed in C / C ++, compiled, and executed in a binary form. Such a native application has an advantage in that its execution speed is fast.

5 is a diagram illustrating an architecture of a webOS device according to an embodiment of the present invention.

FIG. 5 is a block diagram based on the runtime of a WebOS device, which can be understood with reference to the layered structure of FIG. 4.

Hereinafter, a description will be given with reference to FIGS. 4 and 5. Referring to FIG. 5, services and applications and WebOS core modules are included on a system OS (Linux) and system libraries, and communication therebetween may be via a luna-service bus.

Node.js services based on HTML5, CSS, JavaScript, e-mail, contacts, calendar, logging, backup, file notifier WebOS services such as notify, database (DB), activity manager, system policy, audio daemon (AudioD), update, media server, etc. TV services such as Electronic Program Guide (PVR), Personal Video Recorder (PVR), data broadcasting, etc., voice recognition, Now on, Notification, search CP services such as ACR (Auto Content Recognition), CBOX (Contents List Broswer), wfdd, DMR, Remote Application, Download, SDPIF (Sony Philips Digital Interface Format), PDK applications, browser , Native applications such as QML applications And the enyo framework based on the TV UI-related applications and web applications, Luna - made the process via the Web OS core modules, such as the aforementioned SAM, WAM, LSM via the service bus. Meanwhile, in the above, TV applications and web applications may not necessarily be Enyo framework based or UI related.

CBOX can manage the list and metadata of the content of external devices such as USB, DLNA, cloud, etc. connected to the TV. Meanwhile, the CBOX may output content listings of various content containers such as USB, DMS, DVR, cloud, etc. in an integrated view. In addition, the CBOX can display various types of content listings such as pictures, music, and videos, and manage its metadata. In addition, the CBOX may output the contents of the attached storage in real-time. For example, when a storage device such as USB is plugged in, the CBOX should be able to immediately output a content list of the storage device. In this case, a standardized method for processing the content listing may be defined. In addition, CBOX can accommodate a variety of connection protocols.

SAM is intended to improve module complexity and enhance scalability. For example, the existing System Manager processes multiple functions such as system UI, window management, web application runtime, and handling constraints on UX in one process, so that the complexity of implementation is large. Clear implementation interfaces reduce implementation complexity.

LSM supports the development and integration of system UX implementations, such as card views and launcher, independently, and makes it easy to respond to changes in product requirements. On the other hand, LSM, in the case of synthesizing a plurality of application screens such as an App on App (multiple window) to enable the multi-tasking to maximize the hardware resources (HW resources), multi-window (multi-window) and It can provide a window management mechanism (21: 9, etc.). LSM supports the implementation of system UI based on QML and improves its development productivity. QML UX is based on MVC, which makes it easy to compose views of layouts and UI components, and to easily develop code to handle user input. On the other hand, the interface between QML and WebOS components is made through QML extension plug-in, and the graphic operation of the application may be based on the wayland protocol, luna-service call, and so on. have. As mentioned above, LSM stands for Luna Surface Manager and functions as an application window compositor. LSM allows you to synthesize independently developed applications, UI components, etc. on the screen. In this regard, when components such as a Recents application, a showcase application, a launcher application, and the like, each render their own content, the LSM defines a output area, an interworking method, and the like as a compositor. In other words, the compositor, LSM, handles graphics compositing, focus management, input events, and so on. In this case, the LSM receives an event, focus, and the like from an input manager. The input manager may include a remote controller, a HID such as a mouse & keyboard, a joystick, a game pad, an application remote, a pen touch, and the like. As such, LSM supports multiple window models, which can be executed simultaneously in all applications due to the system UI. Related, Launcher, Recents, Settings, Notification, System Keyboard, Volume UI, Search, Finger Gesture, Voice Recognition (STT (Speech to Text), TTS (Text to LSM can support Speech, Natural Language Processing (NLP), pattern gestures (camera, Mobile Radio Control Unit (MRCU)), Live menu, Auto Content Recognition (ACR), etc. .

FIG. 6 is a diagram illustrating a graphic composition flow in a webOS device according to one embodiment of the present invention.

Referring to FIG. 6, the graphic composition process includes a web application manager 610 in charge of a UI process, a webkit 620 in charge of a web process, a LSM 630, and a graphic manager (GM). 640.

When the web application-based graphic data (or application) is generated as a UI process in the web application manager 610, the generated graphic data is transferred to the LSM 630 if the graphic data is not a full-screen application. Meanwhile, the web application manager 610 receives an application generated in the web kit 620 for sharing a GPU (Graphic Processing Unit) memory for graphic management between the UI process and the web process, and as a full-screen as described above. If it is not the application passes to the LSM (630). In the case of the full-screen application, the LSM 630 may be bypassed, and in this case, the LSM 630 may be directly transferred to the graphic manager 640.

The LSM 630 transmits the received UI application to the Wayland Compositor via the Wayland surface, and processes the received UI application to the graphic manager. The graphic data delivered from the LSM 630 is delivered to the graphic manager compositor via, for example, the LSM GM surface of the graphic manager 640.

On the other hand, as described above, the full-screen application is delivered directly to the graphic manager 640 without passing through the LSM 630, which is processed by the graphic manager compositor via the WAM GM surface. The graphics manager handles all graphic data in the webOS device, including GM surfaces such as data broadcasting applications, caption applications, as well as data via the LSM GM surfaces and WAM GM surfaces described above. Receives all the graphic data that has passed through and processes it to be properly displayed on the screen. Here, the function of the GM compositor is the same as or similar to that of the compositor described above.

FIG. 7 is a diagram illustrating a media server according to an embodiment of the present invention. FIG. 8 is a diagram illustrating a configuration block diagram of a media server according to an embodiment of the present invention. Is a diagram illustrating a relationship between a media server and a TV service according to an embodiment of the present invention.

Referring to FIG. 7, the media server supports the execution of various multimedia in the digital device and manages necessary resources. The media server can efficiently use hardware resources required for media play. For example, the media server requires audio / video hardware resources in order to execute multimedia, and can efficiently utilize the resource usage status. In general, fixed devices with larger screens than mobile devices require more hardware resources to run multimedia and require faster encoding / decoding and graphics data delivery due to the large amount of data. Meanwhile, in addition to streaming and file-based playback, the media server may perform broadcasting, recording, and tuning tasks, simultaneously record a video, or simultaneously display a sender and a receiver screen during a video call. It should be able to handle However, the media server has limited hardware resources such as encoders, decoders, tuners, and display engines on a chip-set basis, making it difficult to execute multiple tasks at the same time. Input is processed.

The media server can enhance the system stability, for example, by removing and restarting a playback pipeline in which an error occurred during media playback by pipeline and restarting the error. Even if it does not affect other media play. Such a pipeline is a chain connecting the respective unit functions such as decoding, analysis, and output when a media play request is requested, and required unit functions may vary according to a media type.

Media servers can have extensibility, for example, adding new types of pipelines without affecting existing implementations. For example, the media server may accommodate a camera pipeline, a video conference pipeline, a third-party pipeline, and the like.

The media server can handle normal media playback and TV task execution as separate services because the interface of the TV service is different from the media playback case. In the above, the media server supports operations such as' setchannel ',' channelup ',' channeldown ',' channeltuning 'and' recordstart 'in relation to TV service, and' play 'and' pause in relation to general media playback. By supporting operations such as' and 'stop', different operations can be supported for both, and they can be treated as separate services.

The media server may control or integrate management of resource management functions. The allocation and retrieval of hardware resources in the device are integrated in the media server. In particular, the TV service process transmits the running task and resource allocation status to the media server. The media server frees resources and executes pipelines as each media runs, allowing execution by priority (e.g., policy) upon request for media execution based on the resource status occupied by each pipeline. Recall resources of other pipelines. Here, predefined execution priority and required resource information for a specific request are managed by a policy manager, and the resource manager may communicate with the policy manager to process resource allocation and retrieval.

The media server may hold an identifier (ID) for all operations related to playback. For example, the media server may direct and direct a particular pipeline based on the identifier. The media server may issue separate commands to the pipelines for more than one media playback. The media server can be responsible for playback of HTML5 standard media.

In addition, the media server may follow the TV restructuring scope of the separate service processing of the TV pipeline. The media server may be designed and implemented regardless of the TV restructuring scope. If the TV is not serviced separately, the media server may need to be re-executed when there is a problem with a specific task.

The media server is also referred to as uMS, or micro media server. Here, the media player is a media client, which is, for example, a web for HTML5 video tag, camera, TV, Skype, 2nd Screen, etc. It can mean a kit.

In the media server, management of micro resources such as a resource manager, a policy manager, and the like is a core function. In this regard, the media server also controls the playback control role for the web standard media content. In this regard, the media server may also manage pipeline controller resources.

Such media servers support, for example, extensibility, reliability, efficient resource usage, and the like.

In other words, the uMS, that is, the media server, is a webOS device such as a TV game and resources such as cloud game, MVPD (pay service, etc.), camera preview, second screen, skype, and the like. It manages and controls the overall use and control of resource for proper processing within the system. Meanwhile, each resource uses, for example, a pipeline when the resource is used, and the media server can manage and control the creation, deletion, and use of the pipeline for resource management. Herein, a pipeline is created when a media associated with a task starts a continuation of tasks such as parsing a request, a decoding stream, a video output, and the like. Can be. For example, in relation to a TV service or application, watching, recording, channel tuning, and the like are each processed under control of resource usage through a pipeline generated according to the request. .

Referring to FIG. 8, the processing structure of the media server and the like will be described in more detail as follows.

In FIG. 8, an application or service is connected to a media server 820 via a luna-service bus 810, and the media server 820 is connected to pipelines re-created via the luna-service bus 810. Connect and manage

An application or service may have various clients according to its characteristics and may exchange data with the media server 820 or pipeline through it.

The client includes, for example, a uMedia client (web kit) and a resource manager (RM) client (C / C ++) for connecting to the media server 820.

The application including the uMedia client is connected to the media server 820 as described above. More specifically, the uMedia client corresponds to, for example, a video object to be described later, and the client uses the media server 820 for operation of the video by request. In this case, the video operation relates to a video state, and loading, unloading, play, playback, or reproduce, pause, stop, and the like are all states related to the video operation. May contain data. Each operation or state of such video can be handled through the creation of a separate pipeline. Accordingly, the uMedia client sends the state data related to the video operation to the pipeline manager 822 in the media server.

The pipeline manager 822 obtains information about a resource of the current device through data communication with the resource manager 824 and requests allocation of a resource corresponding to the state data of the uMedia client. In this case, the pipeline manager 822 or the resource manager 824 controls resource allocation through data communication with the policy manager 826 when necessary in relation to the resource allocation. For example, when the resource manager 824 does not have enough or insufficient resources to allocate according to the request of the pipeline manager 822, appropriate resource allocation may be performed according to the request according to the priority comparison of the policy manager 826. Can be. Meanwhile, the pipeline manager 822 requests the media pipeline controller 828 to generate a pipeline for an operation according to the request of the uMedia client for the allocated resource according to the resource allocation of the resource manager 824. .

The media pipeline controller 828 generates the necessary pipeline under the control of the pipeline manager 822. In this generated pipeline, as shown, not only a media pipeline, a camera pipeline, but also a pipeline related to play, pause, and pause may be generated. The pipeline may include a pipeline for HTML5, web CP, smartshare playback, thumbnail extraction, NDK, cinema, multimedia and hypermedia information coding expert groups (MHEG), and the like.

In addition, the pipeline may include, for example, a service-based pipeline (own pipeline) and a URI-based pipeline (media pipeline).

Referring to FIG. 8, an application or service including an RM client may not be directly connected to the media server 820. This is because an application or service may handle media directly. In other words, when an application or service directly processes media, it may not go through a media server. However, at this time, resource management is required for pipeline creation and its use. For this, the uMS connector functions. Meanwhile, when the uMS connector receives a resource management request for direct media processing of the application or service, the uMS connector communicates with the media server 820 including the resource manager 824. To this end, the media server 820 should also be provided with a uMS connector.

Accordingly, the application or service may respond to a request of the RM client by receiving resource management of the resource manager 824 through the uMS connector. These RM clients can handle services such as native CP, TV services, second screens, Flash players, YouTube Source Source Extensions (MSE), cloud games, and Skype. In this case, as described above, the resource manager 824 may manage the resource through the data communication with the policy manager 826 appropriately when necessary for resource management.

On the other hand, the URI-based pipeline is made through the media server 820, rather than directly processing the media as described above. Such a URI-based pipeline may include a player factory, a Gstreamer, a streaming plug-in, a digital rights management plug-in pipeline, and the like.

Meanwhile, an interface method between an application and media services may be as follows.

How to interface using services in a web application. This is a method of making a Luna Call using a Palm Service Bridge (PSB) or a Cordova, which extends the display to video tags. There may also be ways to use the HTML5 standard for video tags or media elements. In addition, the PDK interface using a service. Or, a method of using a service in the existing CP. It can be used to extend existing platform plug-ins based on Luna for backward compatibility.

Finally, the interface is a non-WebOS. In this case, you can call the Luna bus directly to interface.

Seamless change is handled by a separate module (e.g. TVWIN), which is the process of first displaying and seamlessly displaying the TV on the screen without webOS before or during the webOS boot. to be. It is used for the purpose of providing the basic functions of TV service for quick response to the user's power on request because the webOS boot time is slow. The module also supports seamless change, factory mode, and the like, which provide fast boot and basic TV functions as part of the TV service process. In addition, the module may be responsible for switching from the non-webOS mode to the webOS mode.

Referring to Fig. 9, a processing structure of the media server is shown.

In this case, in FIG. 9, the solid line box may indicate a process processing configuration, and the dotted line box may indicate an internal processing module during the process. In addition, the solid arrow may indicate an inter-process call, that is, a Luna service call, and the dashed arrow may indicate a notification or data flow such as register / notify.

A service or web application or PDK application (hereinafter referred to as an "application") is connected to various service processing configurations via a luna-service bus, through which the application operates or is controlled.

The data processing path depends on the type of application. For example, when the application is image data related to a camera sensor, the application is transmitted to the camera processor 930 for processing. In this case, the camera processor 930 processes image data of the received application, including a gesture, a face detection module, and the like. Here, the camera processor 930 may generate a pipeline through the media server processor 910 and process the corresponding data, for example, when the data is required to be selected by a user or to automatically use a pipeline.

Alternatively, if the application includes audio data, the audio may be processed through the audio processor 940 and the audio module 950. For example, the audio processor 940 processes audio data received from an application and transmits the audio data to the audio module 950. In this case, the audio processor 940 may include an audio policy manager to determine the processing of the audio data. The audio data thus processed is processed by the audio module 960. Meanwhile, the application may notify data related to audio data processing to the audio module 960, which may also notify the audio module 960 in a related pipeline. The audio module 950 includes an advanced Linux sound architecture (ALSA).

Alternatively, when the application includes or processes (hereinafter referred to as DRM) content, the corresponding content data is transmitted to the DRM service processor 1160, and the DRM service processor 1170 generates a DRM instance. To process the content data that is subject to DRM. Meanwhile, the DRM service processor 960 may connect and process the DRM pipeline in the media pipeline through the Luna-service bus to process the content data on which the DRM is applied.

The following describes processing when the application is media data or TV service data (e.g., broadcast data).

FIG. 9 illustrates only the media server processing unit and the TV service processing unit in FIG. 8 described above in more detail.

Therefore, below, it demonstrates with reference to FIGS. 8 and 9. FIG.

First, when an application includes TV service data, it is processed by the TV service processor 820/920.

Here, the TV service processor 820 includes at least one of, for example, a DVR / channel manager, a broadcasting module, a TV pipeline manager, a TV resource manager, a data broadcasting module, an audio setting module, a path manager, and the like. Alternatively, in FIG. 9, the TV service processor 920 may include a TV broadcast handler, a TV broadcast interface, a service processor, a TV middleware, a path manager, and a BSP. For example, it may include NetCast. Here, the service processor may mean, for example, a module including a TV pipeline manager, a TV resource manager, a TV policy manager, a USM connector, and the like.

In the present specification, the TV service processing unit may have a configuration as shown in FIG. 8 or 9 or a combination thereof, and some configurations may be omitted or some components not shown may be added.

The TV service processing unit 820/920 transmits the DVR (Digital Video Recorder) or channel related data to the DVR / channel manager based on the property or type of the TV service data received from the application, and then the TV pipeline manager. To create and process the TV pipeline. Meanwhile, when the attribute or type of the TV service data is broadcast content data, the TV service processor 820 generates and processes a TV pipeline through a TV pipeline manager to process the corresponding data through a broadcast module.

Alternatively, a json (Javascript standard object notation) file or a file written in c is processed by the TV broadcast handler and transmitted to the TV pipeline manager through the TV broadcast interface to generate and process a TV pipeline. In this case, the TV broadcast interface unit may transmit data or files that have passed through the TV broadcast handler to the TV pipeline manager based on the TV service policy and refer to them when generating the pipeline.

Hereinafter, a detailed description will be given of the processing steps of the inside of the TV service processor 920 of FIG. 9, in particular, the stages below the TV broadcast interface.

The TV broadcast interface may also perform a controller function of the TV service processor 920. The TV broadcast interface requests the pipeline creation to the TV pipeline manager, which creates the TV pipeline and requests resources to the TV resource manager. The TV resource manager makes a resource request to the media server through the UMS connector and acquires it, and returns it to the TV pipeline manager.

The TV pipeline manager arranges the returned resources in the generated TV pipeline and registers pipeline information in the path manager. The TV pipeline manager then returns the results to the TV pipeline manager, which returns the pipeline to the TV broadcast interface.

Thereafter, the TV broadcast interface communicates with the TV middleware (MW) to request a channel change and the like, and returns the result from the TV middleware.

Through the above-described process, the TV service may be processed.

The TV pipeline manager may be controlled by the TV resource manager in generating one or more pipelines in response to a TV pipeline generation request from a processing module or manager in a TV service. Meanwhile, the TV resource manager may be controlled by the TV policy manager to request the status and allocation of resources allocated for the TV service according to the TV pipeline creation request of the TV pipeline manager, and the media server processor 810. / 910) and uMS connector to communicate data. The resource manager in the media server processor 810/910 transmits the status of the current resource for the TV service and whether or not it is allocated according to the request of the TV resource manager. For example, as a result of checking the resource manager in the media server processor 810/910, if all resources for the TV service are already allocated, the TV resource manager may notify that all resources are currently allocated. At this time, the resource manager in the media server processing unit removes a predetermined TV pipeline according to a priority or a predetermined criterion among the TV pipelines pre-allocated for the TV service along with the notification and generates a TV pipeline for the requested TV service. May be requested or assigned. Alternatively, the TV resource manager may appropriately remove, add, or establish a TV pipeline in accordance with the status report of the resource manager in the media server processor 810/910.

On the other hand, the BSP, for example, supports compatibility with existing digital devices.

The TV pipelines thus generated may be properly operated under the control of the path manager during the processing. The path manager may determine or control the processing path or process of the pipelines in consideration of not only the TV pipeline but also the operation of the pipeline generated by the media server processor 810/910.

Next, when the application includes media data instead of TV service data, the media server processor 810/910 processes the data. Here, the media server processor 810/910 includes a resource manager, a policy manager, a media pipeline manager, a media pipeline controller, and the like. Meanwhile, a pipeline generated under the control of the media pipeline manager and the media pipeline controller can be variously created such as a camera preview pipeline, a cloud game pipeline, and a media pipeline. Meanwhile, the media pipeline may include a streaming protocol, an auto / static gstreamer, a DRM, and the like, which may be determined according to a path manager's control. The detailed processing procedure in the media server processing unit 810/910 uses the description of FIG. 10 described above, and will not be repeated herein.

In the present specification, the resource manager in the media server processor 810/910 may manage resources on a counter base, for example.

The media server design on the WebOS platform described above will be described in more detail as follows.

Media Server is a media framework that allows third-party multimedia pipeline (s) to interface with the WebOS platform. The media server may control, manage, isolate, deconflict, and the like, such that the third-party multimedia pipeline (s) are compliant. These media servers provide a generalized API for applications to play media and can be viewed as platform modules that consistently manage hardware resources and policies. On the other hand, the design of the media server is to reduce the complexity through generalization of media processing and separation of related modules.

The key to such a media server is to provide integration of, for example, the service interface with the WebOS UI. To this end, the media server controls the resource manager, policy manager, pipeline manager, and provides API access according to resource manager queries.

The uMS Connector is the main API and SDK to interface client media pipeline processes with the media server. Ums connectors are events and messages about the interface. The client media pipelines implement client media pipeline state events for enabling load, play, pose, seek, stop, unload, release resource (release_resource), aquire resource (acquire_resource), and the like. .

uMedia API provides C and C ++ API to media server.

The media resource manager provides a method of describing the use of media hardware resources and pipeline client resources in a simple configuration file. The Media Resource Manager provides all the performance and information needed to implement default or third-party media policy management.

The media policy manager works when the resource manager rejects the media pipeline because of a resource conflict. The policy manager can provide a consistent API and SDK to enable third-party policy manager implementations. The policy manager supports media pipelines that match Least Recently Used (LRU), and may be used for one or more conflicted resources.

The pipeline manager tracks and maintains client media pipelines. The pipeline controller provides a consistent API for the pipeline manager to control and manage client media pipelines.

The media server communicates with the resource manager via a library call, which can communicate via the TV services, media pipeline and Luna service bus.

The media resource manager can configure the entire configurable configuration file for disking the media hardware and media client pipelines, detect the resource conflict, and collect all the information needed to implement media policy management. .

The media policy manager reads the policy_select and policy_action fields in the resource configuration file, the resource contention attempts to select the active pipeline described by the policy_select field, and issues with outgoing / selected pipelines based on the policy_action field. Issue. The selection function can be a parameter supported by the pipeline configuration setting entry. Policy actions are unload and release. All pipelines support the unload command to release all allocated resources. Pipelines can additionally support the release command to release specific resources. In the above, the release command is for fast switch pipelines competing with common resources, and the unload command of all resources may not be required for the incoming pipeline and deconflict.

The pipeline manager manages pipeline controllers. The pipeline manager maintains a running queue of pipeline controllers and provides unique indexing for incoming messages from application (s) via a media server.

The pipeline controller maintains a relationship of related media client pipeline processes. The pipeline controller maintains all relevant states and provides the media manager pipeline control interface to the pipeline manager. The pipeline client process is a separate process that uses the uMS connector to provide a control interface to a media server or the like. Pipeline (client) media technologies (Gstreamer, Stage Fright) can be individually and completely decoupled from media server management and services.

On the other hand, the "image data" disclosed herein, for example, video as well as still image data (picture or photograph), thumbnail image (thumbnail image), capture image (capture image), as well as video It may be used to include moving picture data such as video.

In the present specification, the menu is an application that is output through the entire area or the main area of the screen through a web launcher or a web launcher (hereinafter referred to as a 'menu launcher') in a digital device equipped with a web OS. Overlaid on the execution screen.

The menu includes a first part containing history data about previously viewed or executed applications, and a second part listing one or more applications available or executable on the digital device. It is configured by. Here, the second part may not only provide the application list but also provide a content list related to an application currently provided through the main area. In other words, the second part may provide at least one of the first mode and the second mode. Hereinafter, the first mode of the second part provides an application list, and the second mode of the second part provides a content list. Whether to provide the first mode or the second mode to the second part when providing the menu launcher may be determined according to a setting or an application being provided to the main area. In addition to the above-described criteria, the determination may be determined and provided according to various criteria, such as an application attribute of the main area, according to the user. Meanwhile, the first part may be referred to as a recess part and the second part may be referred to as a list part.

The accent part and the list part may each include at least one menu item. The menu item may be a unit for identifying, identifying, and accessing content, applications, data, and the like in each part. In addition, each menu item may be provided through a window.

On the other hand, in the present specification, a menu, a recess part, a list part, a menu item, and the like are respectively illustrated and described in a rectangular or trapezoidal shape, but are not necessarily limited thereto.

10 is a diagram illustrating a screen of a digital device that an application is outputting.

When the application execution request signal is received, the digital device 1000 provides an execution screen of the application for which execution is requested through the entire screen or the main area of the screen.

In this case, the application includes all applications of the digital device on which the web OS is mounted, and the digital device on which the web OS is mounted is also called a web application. The application may include various applications such as a TV (broadcast) application for a broadcast service such as a broadcast program, an application for external input, and an application for image data. For convenience, FIG. 10 illustrates an embodiment in which broadcast content of a specific channel is provided through execution of a TV application.

The application may be executed through data previously stored in a storage medium such as a memory of the digital device 1000 or application data downloaded or streamed from various external servers such as a broadcasting station. In the present specification, the application and the application data may be used interchangeably, and in that case, the meaning may be determined through the context in the sentence or paragraph. In addition, in the case of a favorite application, a frequently used application, or the like, the digital device receives application data using a preload function manually or automatically, so that the application can be quickly accessed or switched / interchanged. interchanging).

The application execution screen of FIG. 10 is provided according to a user's request through an input means such as a remote controller or a mobile device connected or paired with the digital device 1000. Alternatively, the digital device 1000 may provide the application execution screen according to a predetermined key button signal provided on the front panel of the digital device even without using the above input means. In addition, the digital device 1000 may provide various functions, menus, etc. as well as an application execution screen as shown in FIG. 10 based on various input signals such as voice commands and gesture commands, even through the input means or not. . However, for the sake of convenience, an input means will be used as a signal to a digital device such as a key button, a voice command, a gesture, a face recognition, a fingerprint recognition, a fingerprint command, a control command, as well as a case where the remote control or a mobile device is used. B has the meaning including at least one of all means for command input.

On the other hand, if the application is a TV application, it may be processed through the component (s) of the digital device shown in FIGS. In other words, through a network interface or tuner, a broadcast program and / or an IP packet (s) or broadcast signal including signaling data for the broadcast program are received, and a demodulator, a demultiplexer, an SI decoder, and an audio / video are received. It is finally output on the screen via a decoder or the like. For the above process, refer to the aforementioned contents of FIGS. 2 to 3, and a detailed description thereof will be omitted.

11 is a diagram illustrating a method of providing a menu in a digital device according to one embodiment of the present invention.

When the digital device receives a signal for requesting a menu through an input means while using an application as shown in FIG. 10, the digital device outputs a menu on the current screen according to the received signal. This can be provided by accessing or selecting an icon or a function button provided through the screen even though the menu output is not necessarily a menu key signal of the input means.

Conventional digital devices may be inconvenient to use a user's content by covering or overlapping a large portion or a main portion of an application execution screen that is being used even if the screen is not switched or switched entirely to output a menu. there was.

The digital device equipped with the webOS platform according to the present invention provides the requested menu through the menu launcher as shown in Figs. 11A to 11C. The provided menu is provided in a region or area on the application execution screen being provided through the full screen as shown in FIG. 11A. Here, as shown in the drawing, one portion of the lower portion of the screen is an example, but is not limited thereto. Meanwhile, a size (hereinafter referred to as a size for convenience) or a shape such as the height or width of the menu launcher may be arbitrarily adjusted according to a user's setting or an attribute of an application. For example, when referring to the property of the application being used, when certain data is output at the bottom of the screen such as data broadcasting, the menu launcher may be output to an area different from that shown in FIG. 11A. On the contrary, the menu launcher may be output as shown, but the constant data may be output to another area not overlapping with the menu output area. This is also applicable after the menu is printed. In addition, the menu may be output to one of a partial area of the center of the display screen or one of the display screen edge areas. In addition, the menu may be provided by separating the two parts into different areas instead of one area. For example, the first part (recent part) may be provided in one area at the bottom of the screen, and the second part (list part) may be provided in one of the non-overlapping edge areas of the screen edge areas. Alternatively, FIG. 11A may be provided at the bottom of the screen as shown, FIG. 11B may be simultaneously provided in the left edge region of the screen and FIG. 11C in the right edge region of the screen. In addition, when any part or at least one menu item of the part is selected in FIG. 11A, a function or related data related to the selected menu item may be output to at least one edge area of the edge areas.

Accordingly, as illustrated in FIG. 11A, when a menu is configured and provided, screen obstruction may be minimized as compared with a conventional digital device.

In addition, the menu may change size based on the position or movement of the pointer. For example, if the pointer is located on a menu, the user's intention is determined as a menu control or access and provided in the size of the original menu launcher. In this case, the size of the menu launcher may or may not be maximum. On the other hand, if the pointer is located in an area other than a menu (for example, an application output area that is being used), the user's intention is determined to be application control or access, so that the menu launcher being used is minimized or temporarily hidden. It can be convenient to use the application. In the case of temporarily hiding a menu launcher provided with a menu, the user may be able to identify that the menu launcher is already being output upon request. This is to minimize the inconvenience or inconvenience in using the user to call the menu again. In summary, a menu launcher including a menu according to a user's request, an attribute of an application being used, a location / movement of a pointer, etc. may be provided in various versions, or the size may be controlled. In addition, such a menu launcher can be controlled as described above automatically or manually after the output.

FIG. 11A illustrates a digital device in which a menu launcher including a menu is provided according to a user's request while using the application 1110 through the main screen. 11A may be, for example, an initial screen on which a menu launcher including a menu is provided on a screen according to a request of the user.

As described above, the menu provided through the menu launcher may be largely composed of two parts, and the first part 1120 outputs percentage data and the second part ( 1130 includes or outputs an application list or a content list.

One side of the first part 1120 is provided with an interface 1130 for accessing history data or additional history data associated with the first part 1120. The historical data means data about an application provided on the screen before using an application currently provided through the screen. Therefore, when the user selects the interface 1140, the menu launcher may be switched or changed as shown in FIG. 11B. Referring to FIG. 11B, an interface 1140-1 is provided at the rightmost side, and history data is arranged and provided at the left side of the interface 1140-1. When the user selects the interface 1140-1 through a predetermined action or key signal, the digital device switches back to the screen of FIG. 11A. Accordingly, in accordance with access of the interface 1140-1, a menu launcher is provided as shown in FIG. 11B to FIG. 11A. In this case, the history data may be arranged in a timeline basis from the interface 1140-1 to the left. That is, the closer to the interface 1140-1, the historical data may be data of a recently accessed application or vice versa. Meanwhile, the history data provided in FIG. 11B may only access history data for a predetermined time according to a user's request or setting. In addition, the history data provided in FIG. 11B according to the position or the movement of the pointer may be provided with history data of another time unit, that is, additional history data. In this case, the interface 1140-1 may be continuously provided at the corresponding position.

Meanwhile, one side of the second part 1130 is provided with an interface 1150 for accessing an application list or an additional application list related to the second part 1130. When the user selects the interface 1150, the digital device provides the menu launcher of FIG. 11C. An interface 1150-1 is provided on the left side of FIG. 11C, and menu items corresponding to an application are arranged on the right side of the interface. In this case, the aligned menu items may or may not include menu items corresponding to an application that is already output in FIG. 11A. Meanwhile, the application-corresponding menu items provided in the second part 1130 of the menu launcher of FIG. 11A may or may not be a basic menu item preset according to the setting of the digital device or the user. Alternatively, the second part 1130 of FIG. 11A or the menu items provided in FIG. 11C may be arranged based on the user's application preference, such as the frequency of use of the application, the number of times of use, the time of use, or the like, or the attributes of the corresponding application. have.

In FIGS. 11A through 11C, the interfaces 1140, 1140-1, 1150, and 1150-1 may be implemented in other forms, as illustrated. In addition, the first part 1120 and the second part 1130 constituting the menu launcher provided in FIGS. 11A through 11C may switch / interchange their output positions and regions. In addition, each menu item in the corresponding part may also be moved to the corresponding part or another part.

As used herein, the term “access or selection” means that a pointer or the like provided on the screen is positioned, hovered, clicked, dragged, or dropped on an area, part, etc. for a predetermined time. drop) or the like. In addition, the selection may be made through separate key button (s) provided on the front panel of the remote controller or the digital device, or may be made through voice, gesture, and the like. It can also be done by each combination. In the present specification, the remote controller is described as an example of a motion remote controller for convenience, but is not limited thereto, and may include other digital devices such as a smartphone.

In FIG. 11A, although only one menu item is provided in the first part 1120, the present invention is not limited thereto, and a plurality of menu items may be provided in the first part 1120. Meanwhile, each menu item may identify an application or content corresponding to the menu item or include or output one or more image data corresponding to the menu item. Each menu item may be image data that automatically captures a screen when the application is in use or when a channel or a screen is switched. Therefore, when the user selects the first part 1120, the main screen may be used as the first application (the application being watched) on the main screen to the second application execution screen corresponding to the image data of the first part 1120. Is switched to. Capture image data of the last screen of the first application or the representative thumbnail image data of the first application before the switching is provided on the first part 1120 in place of the second application image data. On the other hand, the first application being provided on the main screen may be located in the background in a stopped or stopped state without being terminated. In addition, or in this case, despite the above selection, the screen may not be switched to the whole, but may be played as it is at the corresponding position, and PIP (Picture in Picture), POP (Picture of Picture), PBP (Picture) by Picture).

As described above, the second part 1130 of the menu launcher is provided with an application list. However, as shown in FIG. 11A, all application lists provided or available on the digital device through the second part 1130 may not be listed at a time due to the limitation of the device screen size. Therefore, the application list not listed through the second part 1130 of FIG. 11A is provided in the same manner as that of FIG. 11C through the fourth part 1130 as described above. Accordingly, the application list provided to the second part 1130 of FIG. 11A may be viewed as selected from applications available in the digital device. For example, the applications provided in the second part 1130 of FIG. 11A include, among all the applications available on the device, i) a user's setting or request, ii) a user's recently used application, iii) a user's frequency of use, iv) an application related to the application being provided through the main screen, and v) selected and provided based on alphabets or Korean characters / vowels of the application title, or based on i) to v) at least two or more combinations. In the latter case, at least two or more of i) to v) may be selected as priority factors and selected based on factor values.

Meanwhile, when a menu item corresponding to any one of the menu items corresponding to the applications listed in the second part 1130 of FIG. 11A is selected, the application of the selected menu item is directly executed on the corresponding area or the main screen. It can be provided together with the application being used by providing a full replacement of the application of the or through screen division. This may be due to the nature of the selected application or may be implemented through providing a function or control icon around the selected application. Image data of the first part 1120 may be changed according to the screen provision result. Meanwhile, the position or order of the applications listed on the second part 1120 of FIG. 11A or in FIG. 11C may be arbitrarily changed or controlled. The same applies to the history data of FIG. 11B. That is, the user may arbitrarily control the position or order of the timeline-based history data.

The menu item selected in each part may be differently provided in a manner different from that of the non-selected menu item or enlarged in size compared to the non-selected menu item. In addition, as shown, the selected menu item may be slightly raised upward, the border is highlighted, or title data of an application corresponding to a separate corresponding menu item may be output below. In addition, the differentiated content may be combined in at least two or more so that the selected menu item may be variously configured and differentiated from the menu item that is not.

As described above and as shown in FIG. 11, by providing a menu launcher including a menu together with an existing application, a user may not only recognize the menu more intuitively but also access the desired application more easily and quickly than the conventional digital device. Can be. In addition, the user may use various functions or sub-menu items of the digital device or the application with a minimum depth in addition to using the menu while minimizing screen obstruction or change of the application being used, thereby maximizing convenience of use. In other words, in the digital device according to the present invention, an application can be quickly accessed, fast switching / interchange between contents can be performed, and operations such as multi-tasking can be easily and quickly performed.

12 is a diagram illustrating a method of providing a menu in a digital device according to another embodiment of the present invention.

In response to a user's menu request, the digital device may provide a content / application based menu configured as shown in FIG. 12 through the menu launcher unlike FIG. 11.

Referring to FIG. 12, one application is provided on the main screen 1210, and a menu launcher is overlaid on a predetermined area of the main screen 1210.

Unlike in FIG. 11, the menu described below may be divided into menus for a current mode, a past mode, and a future mode, and the menu launcher configuration for each mode may be different. Can be. This menu mode division is, for example, to provide a more intuitive and easy menu using convenience as compared to FIG.

The past mode may relate to a menu configuration regarding historical data that has been used previously, and the future mode may include a menu configuration regarding recommended content / application to be used in the future. Meanwhile, as shown in FIG. 11, the current mode means a basic menu provided when a menu is called, and may include a menu configuration other than the past mode and the future mode. For example, the current mode may be a menu configured by focusing on an application / content frequently used in a digital device.

In the present specification, for convenience of description, in response to a user's menu request, the digital device may be set to provide a menu of a current mode as an initial menu through a menu launcher. In this case, in the digital device, the past mode and the future mode may be accessible from the first provided current mode. However, it is not limited to this. For example, the digital device provides at least one of the current, past and future modes through the menu launcher through the user's setting or various related factors, especially when the user does not request a menu of a specific mode. However, hereinafter, for convenience, a menu launcher of the current mode is provided as a basic menu or a basic menu, and the past mode and the future mode will be described as an embodiment. However, in connection with the present invention, scenarios for mode switching, mode entry, and the like are not limited to those shown and described herein, but may be made in various ways.

12 to 21 show the menu launcher of the current mode menu, FIGS. 22 to 27 show the menu launcher of the past mode menu, and FIGS. 28 to 35 show the menu launcher of the future mode menu, which will be described in detail with reference to the accompanying drawings. Explain.

12 to 21, the configuration of the menu launcher of the current mode menu will be described in detail.

Referring to FIG. 12, as described above, the menu launcher includes a reward part 1220 and a list part 1230. Here, the menu launcher may further include an interface part 1250 for entering the future mode. In addition, although not shown, the menu launcher may include an interface for history data list or related functions or data access of the reward part 1220 and an additional application list or related functions or data access for the list part 1230. The interface may further include. Meanwhile, the list part 1230 is configured as a second mode, that is, a content list for convenience.

In the current mode, the menu is provided by dividing the accent part 1220 and the list part 1230 on the basis of a reference indicator 1240.

The accent part 1220 corresponds to the first part of FIG. 11, that is, the accent part of FIG. 11, includes history data, and in the current mode, only the history data for an application used immediately before an application currently provided on the screen is included. You can print In this case, the reward part 1220 may provide at least one of various data such as image data, title data 1225, server data, and icon of the corresponding application as history data. The image data may be, for example, any one of a representative thumbnail image of the corresponding application, image data captured immediately before switching / interchange to an application currently being provided, image data downloaded from a server or stored in a digital device. In addition, when there are a plurality of image data or history data for the corresponding application, it may be provided in the corresponding menu item in a slide show manner.

When the select part of the receive part 1220 is received, the digital device switches / interchanges and reproduces the application included in the receive part 1220, that is, the previously executed application, with the application provided through the main area. At this time, the playback may start after the previous playback interrupted portion or may start playback from the beginning again. On the other hand, when the application to be switched / inter-changed, for example, a real-time application (ex. Real-time broadcast program), for example, to guide and select to select a point to play out of the data at the previous playback stop time or the current time You can also follow Alternatively, although not shown, when the accent part 1220 is selected or the pointer 1260 is positioned within the accent part 1220 as described above, the digital device may display a reproducing icon for playing the corresponding application. It may be provided or detailed information about the content. Alternatively, although not shown, the digital device does not affect the application running in the main area, instead of switching / interchanging and playing, as described above, even when the recess part 1220 is selected. The application may be executed in the cent part 1220. In this case, the digital device may further provide data such as a current play position, a play end time, or a time remaining until the play end by providing a play bar during the play.

On the other hand, when the digital device receives a signal for selecting title data or server data in the receive part 1220 through the pointer 1260, the digital device provides an application related to the title data or server data or a server related to the application. Guide data for access can be provided on the screen or via a browser. In this case, the guide data or the browser may be overlaid and provided at the highest level on the screen.

In the above description, the digital device may provide a menu icon in the form of a menu icon when providing additional or related data during the selection or access of the accent part 1220. However, the present invention is not limited to the method of providing the icon form.

In addition, in the present specification, although the convenience part 1220 provides only data for one application, this is merely an example and the present invention is not limited thereto.

On the other hand, the above mentioned parts are related to the past mode to be described later. However, the detailed description thereof will be described later and omitted herein.

In the above description, the reference indicator 1240 serves as a reference, that is, a boundary between the accent part 1220 and the list part 1230. For example, as described above, the accent part 1220 outputs only one accent data so that the accent part 1220 is not scrolled to the left or right. However, the list part data may be scrolled left / right according to the number. However, the list part data is made based on the reference indicator 1240, and even though the scroll part, the receive part 1220 may be continuously provided at a corresponding position without being affected. In addition, if the menu launcher has a multi-stage configuration instead of one-stage on a vertical basis, the reference indicator 1240 may perform a scroll function.

List part 1230 provides one or more menu items for the application, as described above. In this case, the menu items are sorted, for example, for the list part of the current mode according to various criteria. The various criteria may mean, for example, at least one of a setting of a user or a digital device, a recent execution order of an application, an execution time, an execution frequency, an execution frequency, an attribute, and the like.

Each menu item may include icon data 1232 for the corresponding application, as shown in FIG. 12. In this case, each menu item may further include image data, title data, etc. for the application. In the above, the image data may be, for example, representative thumbnail image data for the corresponding application. Meanwhile, various data provided on the menu item may be provided only when a signal for the corresponding menu item is received, or may be provided according to access of an adjacent menu item.

As described above, the list part, although not clearly shown, provides a scroll bar to access additional application menu items based on the reference indicator 1240 without changing the screen configuration.

An embodiment of controlling the list part of the menu launcher will be described with reference to FIG. 13. Here, the list part of FIG. 13 is configured in the second mode.

When the digital device receives a signal for selecting one menu item belonging to the list part, as shown in FIG. 13, the digital device first provides the selected menu item to be differentiated from the unselected menu item (s). In FIG. 13, the menu item rises slightly higher than the menu item that is not selected. In addition, the selected menu item, unlike the menu item (s) not selected, provides data for the application. In FIG. 13, server data (ex, Pooq) of the application and title data (ex, okay, love) of the application are provided. However, the digital device may further provide additional data or provide other data in addition to the data shown and described above. In the above, the server data means, for example, data about a server that provides or services corresponding application data.

Referring to FIG. 14, another embodiment of controlling the list part of the menu launcher will be described below.

13, the list part is still configured and provided in the second mode. However, as described above, according to the present invention, the list part may be configured and provided in the first mode in addition to the second mode of providing the application list or the content list related to the application or the content being provided in the main area. As described above, the first mode includes a list of applications that can be provided or executable in the digital device regardless of the application or content being provided in the main area. In other words, the first mode of the second part provides an application list or content list that is not related to an application or content currently running in the main area, while the second mode provides an associated application list or content list.

In the digital device providing a menu launcher, it is difficult for a user to immediately identify whether to provide the second part in the first mode or the second mode. Therefore, it may be required to switch / interchange the mode of the second part as easily and conveniently as possible after the menu is provided. According to FIG. 14, when the pointer 1410 is located in a predetermined region (eg, a lower region of the list part) in the list part, an interface 1420 for mode switching may be provided as shown in FIG. 14.

Here, when the digital device receives the mode switching interface 1420 selection signal, the digital device switches the mode from the second mode of FIG. 14 to the first mode of FIG. 15.

Referring to FIG. 15, the second part is provided with a first mode, that is, the application list 1510 that can be provided or executed in the digital device. Here, each menu item of the application list 1510 provided in the first mode includes data for identifying the corresponding application, for example. In addition, one or

more menu items

1520 and 1530 of the application list 1510 provided in the first mode may correspond to or instead of the corresponding application identification data, as shown in FIG. 15. It may include image data for the application. For example, the image data may be image data captured at the end of the previous execution or representative thumbnail image data of the corresponding application, which may be received from a memory of the digital device, a server providing a corresponding application, or a web server. In this case, as described above, the application list without the image data may be, for example, an application that has never been executed or recently executed.

FIG. 16A shows a menu configuration similar to, for example, FIG. 11A. In other words,

interfaces

1620 and 1630 are provided for accessing additional menu items on one side of each of the first and second parts. Meanwhile, here, the second part is provided in the second mode.

FIG. 16B provides an interface different from that of FIG. 16A. FIG. 16B provides arrow-shaped

interfaces

1640 and 1650 that function the same as the interface of FIG. 16A. Although FIG. 16B illustrates that the

interfaces

1640 and 1650 are provided in an area not overlapping with the menu launcher, the

interface

1640 and 1650 may be provided in the position of FIG. 16A according to an embodiment.

FIG. 17 illustrates another embodiment of mode switching of the second part, for example. If FIG. 15 illustrates switching / interchanging from the second mode to the first mode, FIG. 17 is an example of switching from the first mode to the second mode. In other words, when the pointer is located in a predetermined area in the first mode of the second part of the menu launcher, that is, located at the bottom corner, the digital device provides an interface 1710 for mode switching. When the mode switching interface 1710 is accessed, the digital device changes the second part configuration from the existing mode (first mode) to another mode (second mode).

Referring to FIG. 18, a menu includes a first part and a second part, and the second part provides a second mode, that is, an application list or a content list. As described above, such an application list or content list includes an application or content related to the application or content being provided in the main area.

Referring to FIG. 18A, the digital device may scroll the list in the second part from side to side. Accordingly, as shown in FIGS. 18B and 18C, when the user scrolls to the left, the menu item for the application may be scrolled based on the reference indicator 1810.

The scroll in FIG. 18 may be performed by, for example, a direction key button signal provided in an input means such as a front panel or a remote controller, or may be made according to the position or movement of the pointer.

Although not shown, when the pointer is positioned on one menu item, the digital device may perform an operation for controlling the menu item. For example, when the pointer selects one menu item and is located at an edge of the selected menu item, the digital device may process the size of the corresponding menu item to be changed up and down or left and right. For example, when the pointer is located at the right corner of the selected menu item, as shown in FIG. 18A, an icon for resizing the left and right or the shape of the pointer may be changed and the size of the menu item may be changed according to the selection. have. If there is a size change, the digital device may further provide various data such as more detailed data, function icons, recommended content / applications, etc., on the menu item of the changed size as compared to the menu item of the original size. Or it is of course also possible to enlarge only the size of the menu item.

19 illustrates playback control of a menu item selected from the foregoing.

As shown in FIG. 19A, when the pointer 1420 is positioned on one menu item 1910 of the second part, the digital device does not execute an application of the menu item 1910 by switching / interchanging to the full screen. Plays in PIP form in the window as shown in 19b. In this case, referring to FIG. 19B, the digital device may provide a play bar 1930 in the menu item to control the position or play control of the currently executed part. Although not shown in FIG. 19B, the digital device may provide various function icons for playback control together with the play bar 1930.

In addition, as described above, in FIG. 19, an application is played on a window that provides a selected menu item. If a user wants to switch / interchange to a full screen to enlarge and play the application, a switching / interchange icon ( 1940 may be further provided.

20 illustrates control of the selected menu item.

In FIG. 20A, when one menu item is selected and the selected menu item is moved up or down, the digital device may play the application of the menu item as shown in FIG. 20B. For example, if it moves upward in FIG. 20A, the application is played as shown in FIG. 20B. Conversely, if the menu item is moved downward in FIG. 20A, playback of the application being played may be terminated.

Alternatively, when an application (eg, an MP3 application) of a menu item is being played back in FIG. 20A, the menu item may be directly moved to the next or previous song as the menu item moves upward or downward.

20A and 20B, in order to control playback of the corresponding application, if the menu item is selected and then moved upwards or downwards by a predetermined height, playback control is performed. In addition, the menu item for the application may disappear from the list. Alternatively, it may be defined to remove only the playback file of the application.

As shown in FIG. 21A, when one menu item of the second part is moved toward the bottom corner of the screen, playback-related

icons

2122, 2124, and 2126 are moved around the menu item (eg, the corresponding menu item) as shown in FIG. 21B. The top of the item, etc.). The playback related

icons

2122, 2124, and 2126 may be recommended content / applications, function icons for controlling playback of the corresponding application, and preview images of the corresponding application.

Meanwhile, in FIG. 21C, unlike in FIG. 21A, the selected menu item may be moved to the left and right corners instead of the bottom corners. Moving left and right in this way, the digital device may place the corresponding menu item between other menu items.

Next, as described above, the menu launcher configuration, control, and the like of the past mode menu will be described below in detail with reference to FIGS. 22 to 27.

Regarding the past mode, the entry can be made in a variety of ways. The past mode entry may be performed by accessing, for example, an accent part or an interface or part (eg, the first part 1120 or the third part 1140 in FIG. 11) for checking details of the accent part. Can be. For example, when accessing the first part 1120, the UI data guiding the entry into the past mode may be provided. Alternatively, although not shown, as shown in FIG. 11 or 12, when the initial menu is provided, a menu item or a menu icon for entering the past mode, the present mode, and the future mode may be provided and the past mode may be entered according to the selection. It may be.

Hereinafter, the scenario after entering the past mode will be described in detail.

Referring to FIG. 22A, historical modes, that is, historical data, are arranged and provided according to a used time sequence. In FIG. 22A, for example, it can be seen that the time used toward the left based on the right edge of the menu launcher is an old application / content.

Meanwhile, in FIG. 22A, the sorted applications are provided, for example, grouped by a date unit (eg, yesterday 2210). However, this may be arranged according to other criteria according to the number of applications to be provided in the past mode. For example, in FIG. 22A, even within a date (yesterday 2210) criterion, the data may be divided into time units, minutes, seconds, and the like, and provided to be distinguishable. In addition, although not shown, the applications, in addition to the time criteria such as date, day of the week, application / content properties, application / content quality, application / content providing channel, characters such as the main actor in the application / content, whether the application / content series, application / The content may be arranged and provided based on at least one of whether content usage is completed, application / content real-time data / streaming, application / content capacity, and the like. Meanwhile, the above-described criteria may be used for arranging applications in combination with at least two or more of each other.

FIG. 22B illustrates that when the application in use among the sorted applications is a real-time application, a time-limited application, or the like as shown in FIG. 22A, the use is terminated. If yes, it may be removed 2220 from the application list of the past mode. In this case, the digital device may provide guide data together to identify that the menu item is removed from the application list of the past mode due to the termination of the viewing. In the above description, the guide data is provided upon removal from the list. Alternatively, the guide data may be provided in advance before the removal, that is, a predetermined time before the application / content of the corresponding menu item is used, to induce additional actions of the user. . However, the guide data may be provided only when the past mode is performed. As stated above, providing guide data means that instant recording / reserved recording for convenience of use, since it is not easy to access again after the application / content of the menu item is removed from the list. This is to guide you to use functions such as time-shift and time-shift. As described above, in the case of a real-time application, a VOD server or an Internet server may be connected to check whether or not recording of related data is performed, and the alarm data may be provided later.

Although not shown, in the present mode, the past mode, and the future mode, various functions such as calling EPG / ESG data, instant recording / scheduled recording, and scheduled viewing can be directly accessed through menu items.

In addition, the digital device, when using the past mode, if any data has been updated, such as history data, that is, modifications or changes to the menu items, the device is provided on the menu item or a predetermined area of the screen. It can be provided so that it can be identified.

FIG. 23 is a diagram illustrating an embodiment of controlling a menu item in a past mode.

Referring to FIG. 23, as described above, the menu items are provided in line with the viewing time.

Here, the position of the menu item 2310 with respect to the previously viewed KBS2 professional baseball broadcast service application among the aligned menu items may be moved within the past mode. For example, the menu item 2310 may be positioned 2320 between the Halcyon Days application menu item and the Rain on Me application menu item arranged as viewed earlier. Although not shown, as shown in FIG. 23, when the position of the predetermined menu item 2310 is moved, the menu items included in the past mode associated with the menu item are arranged in a vicinity, or a function icon associated with the menu item is provided. May be provided.

24 is a diagram illustrating an application processing method in a past mode.

Referring to FIG. 24A, as described above, the past mode is provided by arranging menu items for previously viewed applications. At this time, each menu item captures an image immediately before the end of viewing and provides it on the menu item.

When the menu item 2420 selection signal is received through the pointer 2410, the digital device slightly raises the menu item 2420 upwards to indicate that the selection has been made, and a play icon 2430 and a play bar ( 2440). Here, when the play icon 2430 selection signal is received, the digital device may play back the previous play portion on the main area or the corresponding menu item window.

24B and 24C illustrate embodiments of providing a multi-view screen through, for example, accessing a menu item for a past mode.

In FIG. 24A, when the menu item is accessed, the application of the menu item is directly played in the window or the application of the menu item is played in full screen instead of the application running in the main area. In 24c, the application of the menu item is executed in a PIP window or a divided screen.

In FIG. 24B, an application of the selected menu item is provided through the PIP window 2450. In FIG. 24C, the PIP windows 2450 and 2560 are provided at positions similar to those of FIG. 24B. Meanwhile, in FIG. 24C, not only the PIP window 2460 for executing the application of the selected menu item but also one or more PIP windows for executing an application related to an application executed through the PIP window 2460 ( 2470, 2480 is further provided.

24B and 24C described above, unlike the above-described FIG. 24A, other applications may be used while minimizing use disturbance of previously used applications.

FIG. 25 is a diagram for explaining an application processing method in a past mode.

25A is different from FIG. 24 described above, after selecting a menu item corresponding to a previously used application through a pointer, and then moving to a main area where an already used application is provided through a full screen, as shown in FIG. 25B. The application of the menu item is provided over the full screen. In this case, in FIG. 25B, the menu launcher of the past mode currently provided may be hidden or terminated.

FIG. 26 is a diagram illustrating a method of controlling a menu launcher in a past mode.

Referring to FIG. 26A, the menu launcher of the past mode is configured and provided, and the menu items are classified and provided by grouping the

use time criteria

2610 and 2620, and each group is provided with guide data (ex, yesterday, one week). Etc.).

However, FIG. 26A may be provided by configuring only a TV service application, for example, in configuring a menu in a go-go mode.

In this case, if the user requests past mode data for other applications such as games, web browsers, HDMI, etc. in addition to the TV service application, it may be provided as shown in FIG. 26B.

In FIG. 26A, if the pointer is positioned on the lower predetermined area of the providing menu launcher, the digital device provides the menu item switching interface 2630, as in FIG. 26B, similar to the control method in the current mode.

When the menu item switching interface 2630 of FIG. 26B is accessed, the digital device reconfigures or switches the menu item for the past mode by excluding or including the menu items of the TV service application previously provided in FIG. 26A. .

In FIG. 26B, for convenience, an interface 2630 for controlling only a menu item between yesterday and a week ago is provided according to the location of the pointer. However, as described above with respect to the entire data of the past mode regardless of the location of the pointer, FIG. It can also be implemented to enable switching.

On the other hand, although not shown in the configuration of the menu in the past mode, when there is a lot of history data of the past mode, it is possible to provide only a group icon, unlike the above. As such a group icon, for example, a week ago in FIG. 26, yesterday, and the like. Alternatively, only one menu item for one representative application may be provided in each group icon. In the former or the latter, if a predetermined group icon selection signal is received, the digital device may expand and provide menu items belonging to the selected group.

The menu configuration method described herein may be, for example, according to a user's setting or determined by the digital device. In the latter case, various criteria such as user data, usage pattern, time, date, day of the week, weather, etc. may be used or determined by appropriate combination of the above criteria.

27 is a diagram for explaining the menu item control of the past mode as well.

As described above, even in the past mode, the menu item may move up, down, left, and right after selection to perform a predetermined function.

Referring to FIG. 27A, when a predetermined menu item 2710 is selected through a pointer and the selected menu item is dragged to move toward the bottom corner of the screen, as shown in FIG. 27B, data related to the selected menu item (see FIG. 27B) may be used. 2722,2724,2726.

The

related data

2722, 2724, and 2726 may be a function icon for performing a predetermined function of an application of the menu item, an application related to the application, or a preview image (including a previous viewing portion) of the application. . For example, if the selected menu item is for baseball broadcast content, the

related data

2722, 2724, 2726 may be baseball broadcast content of another channel.

Lastly, FIGS. 28 to 35 will be described in detail with reference to the accompanying drawings for menu configuration and control in the future mode.

FIG. 28 is a diagram to describe a method for entering a future mode menu, for example.

As described above, the future mode menu may enter through, for example, accessing the interface 2820 for future mode entry provided separately from the menu launcher via the pointer 2810, as shown in FIG. 28A, and FIG. It may be entered through the interface 2840 for entry into the future mode provided with the menu launcher of the current mode (including the past mode) provided according to the menu call, such as 28b.

The future mode entry interface 2820 of FIG. 28A may be provided without a current mode menu when a menu is called, for example, as shown in FIG. 28B, but may be provided directly on the screen in response to a key button signal of a remote controller even when the menu is not called. It may be. Meanwhile, although FIG. 28A illustrates that the future mode entry interface 2820 is provided at the middle of the screen, the present invention is not limited thereto. Thus, the future mode entry interface 2820 may be provided at various locations on the screen. In addition, the shape of the future mode entry interface 2820 is not limited to the shape shown in FIG. 28A, and may be variously configured.

As shown in FIG. 28, when a future

mode entry interface

2820 and 2830 selection signal is received, the digital device configures and provides a future mode menu as shown in FIG. 29.

Future mode refers to a menu configuration for recommendation for applications / content that the user has not yet used or is likely to use.

In future mode, you can provide recommended applications / content sorted by category. These categories include the Basic category, Live category, TV shows category, Movies category, Music category, Recommended Applications, Apps category, My Content ( My contents) category. 29 to 35 illustrate menu configurations of the respective categories. Meanwhile, the present invention is not limited to the above-described category of the future mode menu, and may be variously implemented according to a user's setting. In addition, as will be described later, the menu structure of each category may be implemented differently in consideration of characteristics of the corresponding category.

Referring to FIG. 29, the future mode menu includes a search area 2910, a category area, and a recommended content area 2930 corresponding to a selected category. In particular, in FIG. 29, the basic category 2920 is selected in the category area, and applications / contents belonging to the basic category are listed in the recommended content area 2930.

The basic category of FIG. 29 may list recommended applications / content selected according to all recommended contents or various criteria. Thus, in the former case, it may overlap with other categories of recommended applications / content. The order of sorting in the basic category may be set according to the category order, the priority order, and the like. In addition, with respect to the latter, the various criteria may include, for example, season, weather, time, identified user, recently used application / content attribute, type, etc., whether paid or not. For example, FIG. 29 lists the applications / content associated with selected and recommended winter or / and snow based on the current season (ex, winter) or weather (ex, snowy weather) in which future mode has been implemented. will be. In other words, in the embodiment of FIG. 29, the most popular and available applications / content are listed in the snowy winter.

Meanwhile, although not shown, one or more applications / content selected from the recommended applications / contents listed in FIGS. 29 to 35 are switched to the full screen according to the selection, and the PBP (Picture by It may be provided in the form of a picture (eg, a split screen), a picture in picture (PIP), a picture of picture (POP), or the like.

30 lists live category 3010 and FIG. 31 lists recommended applications / content according to TV show category 3110. 30 and 31 are both categories related to TV service or TV service applications, for example. However, while FIG. 30 provides EPG / ESG data for broadcast contents to be broadcasted later based on the current time, FIG. 31 mainly provides VOD contents.

Although not shown, if one menu item is selected from the guide provided in FIG. 30, immediate recording / reserved recording may be performed immediately in a future mode. In addition, when a predetermined menu item is selected in FIG. 30, a full screen or a preview image may be provided in a window in which a current menu item is provided. The same applies to reservation viewing and the like. If in the process of using the future mode, if the live broadcast time of the provided menu items, or by reservation viewing, etc. may be removed from the future mode or may be executed in the window as it is. In this case, after setting the reserved viewing and the like, if another category or menu is used, it may be played on the corresponding window through the full screen or after automatically executing the live category in the future mode. In each of the above-described cases, guide data for convenience of use or identification can also be provided.

As described above, FIG. 31 may be a VOD recommendation list according to an embodiment. In this case, the free VOD content and the paid VOD content may be separately provided. In addition, the VOD content may be classified and listed by the server unit providing the VOD content, the type or the attribute of the VOD content.

32 provides a list of recommended applications / contents belonging to the movie category 3210. The recommended application / content list belonging to the movie category may be, for example, a movie VOD or a list of movie contents stored in a digital device but not yet used. Alternatively, the recommended application / content belonging to the movie category may be content to be provided by a movie dedicated channel among one or more channels belonging to the TV service. Although illustrated in FIG. 32, it is apparent that menu items corresponding to the recommended application / content provided in the future mode may move their positions in any of up, down, left, and right directions. In addition, when there is such a location movement, if the meaning of the movement is, for example, a priority increase in the digital device, the recommended application / content in the previously provided list may be rearranged or reconfigured according to the meaning. The same is true for categories. In other words, the user may change the position of each category menu item arranged in the category area, such as moving or changing the recommended application / content list structure.

33 provides a list of recommended applications / contents belonging to the music category 3310, FIG. 34 provides a list of recommended applications / contents belonging to the application category 3410, and FIG. 35 is a recommendation belonging to the my content category 3510. Provide an application / content list.

36 is a diagram illustrating an example of a digital service system according to one embodiment of the present invention.

Referring to FIG. 36, the digital device 3610 and the second digital device 3640 communicate data with an external server (such as a cloud server) 3650 through menu communication data for configuring a menu according to the present invention. Receive application / content data for applications / content, etc. of my menu items. In addition to the digital device 3610 and the second digital device 3640, the mobile device 3620 or the repeater 3630 may perform data communication with the external server 3650 described above. In addition, the mobile device 3620, the repeater 3630 and the second digital device 3640, the data received through the data communication with the external server 3650 for the menu configuration of the digital device 3610, etc. I can deliver it. In this case, the data to be delivered, if necessary, may be processed and delivered in a format change or the like.

Meanwhile, at least one of the mobile device 3620, the repeater 3630, and the second digital device 3640 may receive and output menu data provided by the digital device 3610. For example, if the digital device 3610 is currently outputting a current mode menu, at least one or more of the mobile device 3620 and the second digital device 3640 may receive and output the current mode menu. Receives and prints the menu of past mode or future mode. In addition, each of the mobile device 3620 and the second digital device 3640 may receive and output menu data of different modes from the digital device 3610.

FIG. 37 illustrates an example in which a menu requested by the digital device 3710 is transmitted to and output from the mobile device 3720 instead of the screen of the digital device 3710.

38 may be an embodiment of a new type of menu launcher, that is, a menu launcher configuration in the form of a mobile device. As another example, FIG. 38 may be a UI implemented to execute various menus, data, applications, etc. of the mobile device in the digital device according to execution of an application such as OSP of the mobile device.

39 is a flowchart illustrating a menu data processing method in a digital device according to the present invention.

The digital device receives a broadcast signal including content and signaling data (S3902), decodes the received content and signaling data, and outputs it on a screen (S3904).

When the first user input signal for a menu call is received (S3906), the digital device composes a menu screen in a predetermined area of the screen on which the content is being output and outputs it in an overlay form (S3908).

As described above, after the second user input signal is received after outputting the menu together with the content (S3910), the digital device generates a GUI for switching the menu screen configuration / interchange according to the second user input signal. Output (S3912),

When the third user input signal is received (S3914), the digital device switches / interchanges the configuration of the menu screen according to the third user input signal (S3916).

The following describes the multi-view mode data processing in more detail with reference to the accompanying drawings.

FIG. 40 is a diagram for explaining a sound control method in screen division in relation to multi-view mode data processing.

In FIG. 40, the screen is divided into two screens for convenience. However, it will be appreciated that the control method described below can be applied in the same or similar manner in the multi-view mode to be described later as well as the two split screens shown.

As described above, the screen of FIG. 40 includes a first split screen 4010 and a second split screen 4020. In this case, the digital device turns on the sound of the first split screen 4010 and outputs it through a speaker, but turns off the sound of the second split screen 4020 and outputs it through a speaker. Can be controlled to prevent In this case, a subtitle (caption or subtitle) may be provided to the corresponding area for the second split screen 4020 in which the sound is not output.

Meanwhile, the above-described sound control method may be performed according to a user's setting or a basic setting of a digital device, for example, a sound priority output on a split screen on the left when splitting a screen. On the other hand, as described above, even if the user or the device is not set, it is possible to arbitrarily control the sound setting for any one or all divided screens based on the illuminance or noise of the surrounding environment. Such control may be temporary or may be performed in units of content. On the other hand, in addition to the above-described conditions, the user's propensity based on the personalized information or the attribute or type data of the content provided in each divided screen may be further referred to and used for the sound setting. As described above, this sound control method can be suitably applied even in the multi-view described later.

Hereinafter, the control method of the multi-view mode according to the present invention will be described in more detail. Here, the multi-view mode will be described assuming a current mode for convenience, but it is obvious that the multi-view mode can be controlled in the past mode and the future mode. In addition, although not shown, there is a separate multi-view mode in addition to the above modes, through which the multi-view mode control method according to the present invention may be described.

41 and 42 are diagrams for explaining a multi-view mode entry method according to the present invention.

Referring to FIG. 41, an application execution screen for one content is provided on a main screen, and a menu launcher of the current mode is overlaid on it.

In FIG. 41, the menu launcher of the current mode provides items for the plurality of applications. Here, when the pointer is positioned on any one item 4110 of the provided items, the digital device enlarges and provides the item 4110 corresponding to the pointer position compared to other items.

Meanwhile, as described above, when the pointer is positioned on the predetermined item 4110, the digital device may process as shown in FIG. 42.

Referring to FIG. 42A, when the pointer device stays on the corresponding item 4110 for a predetermined time or more, an icon for entering a multi-view on the upper part of the item 4110 (hereinafter, multi) is shown. View icon 4216 or playback bar 4214. For example, the provided play bar 4214 may be displayed and provided to be playable after the execution point immediately before the application of the item. In FIG. 42A, although the play bar 4214 is illustrated in FIG. 42A, a play icon is provided on the item or around the multi-view icon 4216 in place of the play bar 4214, and according to the play icon selection. A play bar 4214 may be provided. Here, as described above, playback control of the application of the item is possible through the play icon or the play bar. When the play command is received, the digital device may provide a corresponding application execution screen on the main screen. You can also play directly on Windows. In connection with the present invention, if the user enters a separate multi-view mode or a multi-view icon is provided, the user may enter the multi-view mode according to the playback command.

FIG. 42B illustrates that the multi-view icon 4202 provided in FIG. 42A can be selected through the pointer. When the multi-view icon 4202 is selected as described above, the digital device leaves only the corresponding item as shown in FIG. 42C. The remaining one or more items may be removed such that the menu launcher does not terminate or the menu launcher itself does not terminate and may immediately enter the multi-view mode to provide a multi-view window.

In addition, although not shown, the multi-view mode may be simply executed by a multi-view key button, a gesture, a voice, or the like provided in an input means such as a remote controller. The gesture or voice may be recognized or input through the corresponding digital device or through another digital device without necessarily using the input means. Although the multi-view entry has been described above, it is possible to enter in various ways in addition to the entry method described herein, which is also included in the present invention.

43 illustrates a multi-view mode providing screen and a multi-view control method after entering the multi-view mode.

43 is a diagram illustrating a multi-view entry screen according to an embodiment of the present invention.

Referring to FIG. 43A, the multi-view window 4310 is provided in a PIP form on the lower left side of the main screen. However, the location of the multi-view window 4310 may be changed depending on whether there is a menu launcher, whether there is subtitle data for an application being provided on the main screen, whether data broadcasting is provided, and the like. The multi-view window 4310 may be fixed at a corresponding position but may be moved under the control of a user. Meanwhile, the multi-view window 4310 may be arbitrarily adjusted in size as shown in the drawing. In addition, the shape of the multi-view window 4310 does not necessarily have to be a rectangle, and the user may arbitrarily change the setting. When necessary for changing the shape setting of the multi-view window 4310, the digital device may provide a template for convenience of selection. Although not shown in FIG. 43A, the multi-view window may have a specific color or be highlighted in order to distinguish it from the application screen being provided on the main screen and also to identify the execution of the multi-view mode. In addition, guide data may be separately provided to identify that the multi-view is executed in the form of text or icons on the main screen or around the window.

FIG. 43B illustrates an embodiment in which functional icon (s) are provided around the multi-view window, unlike FIG. 43A. FIG. 43B may be a multi-view configuration when the first multi-view window is provided or may be a multi-view configuration provided according to a predetermined action of the user in FIG. 43A.

When the multi-view window is called according to a user's request, the digital device may display an indicator 4320 provided on one side of the multi-view window when there are additional applications related to the application to be provided on the multi-view window. By providing a user with discernible presence of the relevant application. Meanwhile, a delete icon for removing the multi-view window may be provided on the corresponding multi-view window. If a plurality of multi-view windows are being provided, a recycle bin icon may be provided, and the delete icon may be provided on each multi-view window according to the selection of the recycle bin icon, and a multi-view window is defined as the recycle bin icon. You can control the multi-view window by dragging and dropping. On the other hand, the indicator 4320 may be provided in a variety of height, width, color, shape, etc. for the user's identification convenience. For example, in the case of two and four sub-view providing applications, the width of the indicator 4320 may be varied so that the number of additionally providing applications can be measured. In addition, although not shown, with respect to the indicator 4320, the number of sub-view-providing applications may be provided in the form of a number on or around the multi-view window or the indicator so as to be identified.

FIG. 44 is a diagram illustrating a multi-view window control method according to another embodiment of the present invention.

In particular, FIG. 44 is an embodiment of control of an application capable of providing a sub multi-view as in FIG. 43B described above.

When the pointer is positioned on the indicator 4410 provided on one side of the main multi-view window or selects the indicator 4410, as shown in FIG. 44A, the digital device corresponds to FIG. 44B in response to the access of the indicator 4410. One or more sub

multi-view windows

4420, 4430, and 4440 are unfolded and provided.

Here, the

sub multi-view windows

4420, 4430, and 4440 may be the same application as the application provided by the main multi-view window or may be another application. Meanwhile, each of the

sub multi-view windows

4420, 4430, and 4440 may be the same application or another application.

In addition, the

sub multi-view windows

4420, 4430, and 4440 may be provided with a size equal to or smaller than that of the main multi-view window. The resolution of the

sub multi-view windows

4420, 4430, and 4440 may be lower than that of the main multi-view window.

In addition, for convenience, the sub-multi-

view windows

4420, 4430, and 4440 are described as providing applications related to the initial multi-view window, but are not necessarily limited thereto and may not be related to each other.

45 is a diagram illustrating a detailed control method of a multi-view window according to one embodiment of the present invention.

44 and 45, for example, when an application of the main multi-view window is providing content of a sports channel to a TV application, the sub multi-view windows provide sports being provided by another channel.

45A and 45B, except for the main multi-view window, each of the sub-multi-view windows may be assigned to another sub-multi-view window according to the position or access of the

pointer

4510 and 4520. In comparison, the access is provided to be identified. In other words, the selected multi-view window is different in size compared to other multi-view windows or highlights, or as shown slightly up, to provide a discernible selection.

Or, since the resources of the digital device are limited, even if a plurality of sub-multi-view windows are provided through the indicator, the corresponding application may not be played in all the multi-view windows according to the resource state. In this case, the digital device provides a representative thumbnail image or captured image of the application of the corresponding multi-view window, and processes only the application corresponding to the selected multi-view window as shown in FIGS. 45A and 45B. can do. In this case, when resources are needed, playback of the main multi-view window may be paused in order to use resources for the main multi-view window.

46 and 47 are diagrams for describing a detailed playback control method of a multi-view window according to one embodiment of the present invention.

Referring to FIG. 46, when the main multi-view window is selected while the main multi-view window is provided, the digital device may provide a delete icon 4630 as described above.

When the delete icon 4630 is accessed via a pointer, the main multi-view window may be removed and the multi-view mode may end. The digital device may, in this case, include it in the list of rewards for the application provided through the removed multi-view window, and store the reproduction end time.

In addition, when the main multi-view window is dragged and dropped onto the main area 4610 via the pointer, the digital device terminates playback of the application being provided in the main area 4610 and is being provided on the main multi-view window. An application may be provided through the main area 4610. In this case, when there is a multi-view window other than the dropped multi-view window, the application provided in the main area 4610 and the application provided through the multi-view window are switched / inter as shown in FIG. 47. You can change it. However, if there is no multi-view window, the multi-view mode can be automatically terminated. Alternatively, as described above, the multi-view mode may be automatically terminated according to the drag & drop of the user regardless of whether the multi-view mode exists.

Although not shown, the above description may be applied to the sub-multi-view windows other than the main multi-view window in the same or similar manner.

48 is a diagram illustrating a method of exiting a multi-view mode according to the present invention.

48A, as described above, selects the delete icon 4820 provided on the multi-view via the pointer 4810, and the digital device removes the corresponding multi-view window in response to the selection. At this time, if all the multi-view windows are removed from the screen, the multi-view mode is automatically terminated.

FIG. 48B shows a state in which the corresponding multi-view window is removed, for example, when the delete icon of FIG. 48A is selected. Referring to FIG. 48B, the multi-view window is removed and the multi-view mode is removed by the penguin 4830 coming out and disappearing with the multi-view window out of the screen, such as animation, to remove the window according to the deletion icon selection. Ends.

On the other hand, although not shown, in response to entering the multi-view mode, termination can also be made simply through input means such as a remote control or multi-view key buttons, gestures or voices provided on the front panel.

49 to 58 are diagrams for explaining a sub multi-view operation control method according to the present invention.

In particular, FIGS. 49 to 52 disclose various embodiments of a method for controlling a sub multi-view operation through the aforementioned indicator.

FIG. 49 illustrates an embodiment in which an indicator 4920 is provided on the upper surface instead of the right side of the multi-view window 4910 unlike FIGS. 43B and 44.

When accessing the indicator 4920 provided on the upper surface of the multi-view window 4910 in FIG. 49, the digital device is one or more in the direction of the upper surface of the multi-view window 4910, as shown in FIG. 50. More sub-multi-view windows 5010 may be provided.

Meanwhile, referring to FIG. 50, the sub multi-view windows 5020 may also be provided on the right side of the main multi-view window. The sub-multi-view windows 5020 may be, for example, preview data for an application provided on the main multi-view window even if it is not accessed by a separate indicator 4920. The preview data may be a still image or a moving image. For example, if the application provided on the main multi-view window is a total of 60 minutes of video data and currently plays 40 minutes, the sub multi-view windows 5020 may include 10 minutes, 20 minutes, It may be preview data for 30 minutes, 50 minutes and 60 minutes. In addition, in the above, the application on the sub multi-view windows 5020 is described as data related to the application being provided on the main multi-view window, but is not necessarily limited thereto, and may be irrelevant application data. For example, in FIG. 49, only the sub-multi-view windows 5010 provided on the upper part as the user accesses the indicator 4920 is insufficient due to screen size or the like. May be allocated to the right side of the non-overlapping main multi-view window and provided.

In addition, in FIG. 50, the first sub multi-view window 5010 is associated with or preview data of an application on the main multi-view window, and the second sub multi-view window 5020 is associated with an application on the main multi-view window. It may be for an unrelated application. For example, suppose you are offering a seven-game first baseball game on a TV application on the main multi-view window. In this case, the first sub multi-view window 5010 may be data for one to six times of the first baseball game or may be a highlight image. On the other hand, the second sub multi-view window 5020 may provide data not related to the second baseball game or other broadcast program of the corresponding channel between the teams different from the first baseball game.

Referring to FIG. 51, a main multi-view window 5110 is overlaid on a main screen provided by a first application and provided in a predetermined region. Here, indicators for the sub multi-view window may be provided on each side of the main multi-view window 5110. In FIG. 51, the first indicator 5120 provided on the right side and the second indicator 5130 provided on the upper side are illustrated for convenience. However, the present invention is not limited thereto.

In the above-described embodiment, upon accessing the indicator provided on the main multi-view window 5110, the digital device provides sub-multi-view windows around the main multi-view window 5110 according to the accessed indicator. It was. In this case, however, in some cases, the main multi-view window 5110 may be obstructed as shown in FIG. 51 because it may impair the concentration of the application being provided on the main screen or the application on the main multi-view window or hide important data. It is also possible to provide sub-multi-view windows 5140 at the edge of the screen rather than around. The location or area in which the sub multi-view windows 5140 are provided is not necessarily limited to the right side of the screen illustrated in FIG. 51, but is automatically determined in consideration of various data such as the property or type of the application and the characteristics of the current playback scene. Or manually determined and provided. Meanwhile, in FIG. 51, the sub-multi-view windows 5140 are provided at the edges of the screen, not the periphery of the main multi-view window 5110 as described above. In this case, when three sub-multi-view windows are provided as shown in FIG. 51, each may be provided separately from each other at a different position.

52 illustrates that, unlike the above-described embodiments, the sub-multi-

view windows

5210, 5220, 5230 are automatically optimized for the main multi-view window size by the indicator or when the main multi-view window is provided. It may be provided at the top. In this case, the

sub multi-view windows

5210, 5220, and 5230 are not fixed and may be accessed left and right according to a user's action, and may be replaced with windows for other time zones or other applications. You can also switch between windows and main screens.

53 illustrates that the main multi-view window 5310 may be provided at the bottom of the center of the screen from the initial presentation. On the contrary, as shown in the previous embodiment, the main multi-view window 5310 may be provided at the bottom left of the screen. It may be the case that it is moved and provided.

For example, in the former case, the main multi-view window 5310 may be provided at the center of the bottom of the screen, unlike the above-described embodiments, if one or more sub-multi-view windows exist in the provision thereof. have. In this case, the sub multi-view window (s) 5320, 5330, 5340, and 5350 are shown on the left side and / or the right side of the main multi-view window 5310 provided at the bottom center of the screen. As may be provided. In this case, the

sub multi-view windows

5320 and 5330 may represent a time point provided through the main multi-view window 5310, that is, data previously provided before the present. In addition, the

sub multi-view windows

5340 and 5350 may indicate a time point provided through the main multi-view window 5310, that is, data to be provided after the present. In the former case, a still image or a moving image may be provided through the

sub multi-view windows

5320 and 5330. In the latter case, the still image data may be provided in the sub multi-view window (such as a thumbnail image because it is future data. 5340, 5350 may be provided. However, if the data to be provided through the

sub multi-view windows

5340 and 5350 is application data already stored or downloaded to the digital device, it may be a video. On the other hand, unlike the above, the

sub multi-view windows

5320, 5330, 5340, and 5350 are not the same application as the application provided on the main multi-view window 5310, but are related to or not related to the application. It may be.

For example, in the latter case, when the main multi-view window 5310 is moved from the lower left side of the screen to the center as in the above-described embodiment, the second sub-multi-view window is automatically provided accordingly. to be. For example, if the second

sub-view windows

5320, 5330, which were not provided when the main multi-view window is provided, move the main multi-view window 5310 within the screen, one or more of them may be changed accordingly. The second

sub multi-view windows

5320 and 5330 are automatically provided. The second

sub multi-view windows

5320 and 5330 may be provided differently from the first

sub multi-view windows

5340 and 5350, and may be moved or moved in the main multi-view window. The number or position of the sub multi-view windows provided according to the distance moving area may vary.

Referring to FIG. 54, assume that a user is watching a baseball application through a main screen in a digital device. In this case, in the case where 7 times 5410 is currently in progress, the digital device may provide a multi-view mode at the user's request or automatically. As described above, in providing the multi-view mode, an indicator indicating that the multi-view mode can be provided or an indicator for entering the multi-view mode is provided in one region on the main screen, and the indicator is shown in FIG. 54 through the provided indicator. As can be seen, it is possible to provide sub-multi-view windows including application data for six times 5520, five 5430, four 5440, three 5450, etc., seven times prior to the current process. . In FIG. 54, the main multi-view window 5410 is provided with the sub multi-view window 5520 for six times, which is currently just seven times, and the sub-views for five, four, and three times are sequentially around. The

multi-view windows

5430, 5440, and 5450 may be aligned and provided. However, this may be the opposite. In addition, although not shown, if the user selects any one of the sub multi-view windows, the application on the selected sub multi-view window may be played at the corresponding location or switched / interchanged with the application data on the main screen. . In particular, in the latter case, in the case of providing the application data on the selected sub multi-view window, when at least one resource is insufficient, the latter may switch / interchange with the application data of the main screen. In addition, if switching / interchanging is performed between the application data currently being provided and the application data being provided in the multi-view windows even if not for the above reason, the current application data is continuously continued on the switched / interpainted window. You can then play or stop, but perform the time-shift function. When the time-shifted application data is switched / interchanged back to the main screen, the user is provided with a choice whether to start playback from the time-shift start point or the current playback portion. Can provide.

Referring to FIG. 55, application data is output to the main screen 5510, and multi-view windows are also provided. Here, each of the multi-view windows may perform a group function, for example. For example, when the application data being provided on the main screen 5510 is eight episodes of season 4, the remaining episodes of the application of the season are provided to the first multi-view window 5520, and the remaining multi-view windows 5530 are provided. 5540 and 5550 provide episodes of different seasons. At this time, an indicator for accessing the episode of the season is provided on each multi-view window, and when the provided indicator is accessed, an episode list of the season is provided. Meanwhile, each of the multi-view windows may output data for identifying the corresponding season, image data, and data about the number of episodes belonging to the corresponding season. On the other hand, if a predetermined multi-view window itself is selected, the application data of the season can be automatically reproduced on the window or the main screen in sequence in episode order. On the other hand, if a predetermined time elapses after placing the pointer on the multi-view window, the episode list is automatically provided, and the episode may be automatically played on the window or the main screen according to the selection.

Referring to FIG. 56, similar to FIG. 55, each multi-view window is configured in group units, and when a predetermined multi-view window 5610 is selected,

episodes

5620 and 5630 belonging to the season are selected. It can be provided as shown in the form of a sub-window of.

Referring to FIG. 57, when the multi-view window provides an application being provided on the main screen, that is, episode 8 of season 4, each multi-view window provides previous episodes of the season and episodes of the season that can be provided later. In this case, the first multi-view window is provided with an episode immediately before the episode of the application currently being provided on the main screen. In this case, when there are a plurality of season episodes of the application being provided on the main screen, the first multi-view window may provide an indicator for access thereof. The remaining multi-view window provides episodes that can be provided after the season episode of the application being provided on the main screen. In this case, when the episodes have not been viewed or recorded, the function icons for the reserved recording / viewing reservations 5712/5714 are accessed according to the access of the corresponding multi-view window 5710. 5710) may be provided in the vicinity to immediately perform a function or schedule a performance.

FIG. 58 is similar to the above-described embodiments, but other contents of the previous embodiments include an area on the main screen that is largely covered by providing a multi-view window when additional seasons or additional episodes exist. It may be inconvenient to watch. Thus, in FIG. 58, accessible seasons or episodes are provided in the form of icons in the periphery of the main multi-view window, thereby minimizing screen obscurity and making it easier to control function execution.

Although not shown in FIG. 56, the sub multi-view window may also provide an indicator as an interface for providing additional sub multi-view windows, similar to the main multi-view window. For example, one or more second

sub multi-view windows

5620 and 5630 provided in FIG. 56 may be provided according to an access of an indicator provided on the first sub multi-view window 5610. .

A method of processing multi-view data in a digital device may be as follows.

The digital device receives content and signaling data for the content (S5902), decodes the content and signaling data (S5904), and outputs the decoded content to the screen (S5906).

The digital device receives a user input for requesting a menu launcher output (S5908), and outputs one or more menu items on the screen according to the menu launcher output request user input (S5910).

The digital device receives a user input for requesting output of the multi-view content (S5912), and overlays and outputs a window for outputting the multi-view content on a predetermined area on the screen where the content is output according to the first user input. (S5914).

The output window may include an interface for outputting one or more subwindows, and output the one or more subwindows together with the window to a screen area according to the selection of the interface.

The content output through the one or more sub-windows may be content associated with at least one of the content and the multi-view content output through the window.

The window for outputting the multi-view content may be output in an area not overlapping with the output menu launcher, and the one or more sub-windows may be output in a different size than the window.

The window may interact with content being provided on the screen according to drag and drop, and the one or more sub-windows may be output to a screen area adjacent to the window or an edge of the screen. .

In addition, when the output window is changed from an initial position in the screen area to another position, one or more sub windows may be output in addition to the sub windows output according to the interface selection based on the changed position. .

Meanwhile, at least one of the output window and the sub-windows is configured in a group unit including content of the same category, and when the pointer is located in the corresponding window or sub-window, the content list in the group unit is displayed in a window form or icon. In the present specification, in the main multi-view and sub-multi-view windows, various data that can be processed by the digital device may be provided. Such data may include only image, video, and audio. It can also support external input devices such as DVI / HDMI or gaming devices, or web browser formats. On the other hand, if a web browser is provided on the main multi-view window, the sub multi-view window may or may not be recommended or provided in the form of a web browser.

In addition, when the application provided on the main multi-view window is switched / interchanged with the application provided on the main screen or the sub multi-view window, the remaining multi-view windows except for the corresponding multi-view window remain unchanged. Conversely, the sub multi-view window may be changed in full depending on the application on the switched / interchained main multi-view window.

In the present specification, the contents described in the drawings are limited to the description of the drawings for convenience, but the technical idea of the present invention is not limited thereto. Accordingly, although not shown or described, any contents not shown in the drawings or referred to in the description of the drawings that do not conflict with each other may be applied to the related drawings and the description of the related drawings as they are or may be appropriately combined. Applicable, it is also included in the technical idea of the present invention.

According to various embodiments of the present disclosure, a digital device may make accessing and using data, such as a function and content desired by a user, easier and faster than in the related art, and prevents viewing of content being displayed on a main screen, that is, content previously viewed. Multi-view screens, based on the webOS platform, with the main content or / and menu launcher being displayed, allowing for quick and easy access and use of the desired data with minimal depth or screen changes on the called menus In addition to making it easy to control and control, as well as by providing a menu screen that is more intuitive and easier to use than the conventional, it is possible for anyone to use the digital device easily and conveniently.

The digital device and the method for controlling the digital device according to the present invention are not limited to the configuration and method of the embodiments described as described above, but the embodiments are all or part of the embodiments so that various modifications can be made. May be optionally combined.

On the other hand, the operating method of the digital device of the present invention can be embodied as a processor-readable code on a processor-readable recording medium provided in the digital device. The processor-readable recording medium includes all kinds of recording devices that store data that can be read by the processor. Examples of the processor-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like, and are implemented in the form of a carrier wave such as transmission over the Internet. It includes being. The processor-readable recording medium can also be distributed over network coupled computer systems so that the processor-readable code is stored and executed in a distributed fashion.

In addition, although the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the above-described specific embodiment, the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to digital devices, and is applicable to various types of digital devices, and thus there is industrial applicability.

Claims

In a digital device,

A receiver which receives content and signaling data for the content;

A user input receiver configured to receive a first user input for requesting output of the multi-view content;

A decoder for decoding the content and signaling data;

A controller configured to control the output of the decoded content on a screen and to overlay and output a window for outputting the multi-view content in a predetermined area on a screen on which the content is output according to the first user input; And

An output unit for outputting the content and the window,

The control unit,

If a pointer is located in the window, the interface for outputting one or more subwindows is controlled.

And controlling the one or more sub-windows to be output to the screen area together with the window according to the interface selection.
The method of claim 1,

Content output through the one or more sub-windows,

And at least one of the content and the multi-view content output through the window.
The method of claim 1,

The user input receiver,

Receive user input requesting menu launcher output,

The control unit,

And output one or more menu items on the screen according to the menu launcher output request user input.
The method of claim 3,

The first user input is

And a multi-view function icon selection signal provided around the selected menu item in response to selecting one of the one or more menu items belonging to the output menu launcher.
The method of claim 3,

The control unit,

And output the window for outputting the multi-view content to an area not overlapping with the output menu launcher.
The method of claim 1,

The control unit,

And controlling the one or more sub-windows to output a different size than the window.
The method of claim 1,

The control unit,

And control to interchange with the content being provided on the screen as the window is dragged and dropped.
The method of claim 1,

The control unit,

And control the one or more sub-windows to be output to a screen area adjacent to the window or an edge of the screen.
The method of claim 1,

The control unit,

When the output window is changed from the initial position in the screen area to another position, the control unit outputs one or more sub windows in addition to the sub windows output according to the interface selection based on the changed position. Digital device.
The method of claim 1,

The control unit,

At least one of the output window and the sub-window is configured in a group unit including content of the same category, and when the pointer is located in the corresponding window or sub-window, the content list in the group unit corresponds to a window or icon form. Digital device for controlling the output on the window.
In the method for processing multi-view data in a digital device,

Receiving content and signaling data for the content;

Decoding the content and signaling data;

Outputting the decoded content to a screen;

Receiving a first user input requesting multi-view content output; And

And overlaying and outputting a window for outputting the multi-view content in a predetermined area on a screen where the content is being output according to the first user input.

The output window includes an interface for outputting one or more sub-windows, and the multi-view data processing in the digital device outputting the one or more sub-windows together with the window to the screen area according to selection of the interface. Way.
The method of claim 11,

Content output through the one or more sub-windows,

And content associated with at least one of the content and the multi-view content output through the window.
The method of claim 11,

Receiving a user input requesting a menu launcher output; And

And outputting one or more menu items on the screen in accordance with the menu launcher output request user input.
The method of claim 13,

The first user input is

And a multi-view function icon selection signal provided around the selected menu item in response to selecting any one of one or more menu items belonging to the output menu launcher.
The method of claim 13,

The window for outputting the multi-view content,

And outputting the data to an area not overlapping with the output menu launcher.
The method of claim 11,

The one or more subwindows,

And outputting in a different size as compared to the window.
The method of claim 11,

The window is,

And interacting with content being provided on the screen according to drag and drop.
The method of claim 11,

The one or more subwindows,

And outputting to a screen area adjacent to the window or to an edge of the screen.
The method of claim 11,

When the output window is changed from the initial position in the screen area to another position, on the basis of the changed position, in addition to the sub-window output according to the interface selection, one or more sub-windows further output How to handle multi-view data on a device.
The method of claim 11,

At least one of the output window and the sub-window,

In a digital device comprising a group unit including content of the same category, and if the pointer is located in the window or sub-window, the content list of the group unit is output on the window in the form of a window or icon How to handle multi-view data.