KR20150033185A - Method and apparatus for supporting multi-screen in a communication system - Google Patents

Abstract

The present invention provides a method for supporting multi-screens in a communication system. The method includes receiving, by a first device, a list including devices that can execute an application program installed on the first device by interworking with the first device through a web server; selecting a second device to execute the application program installed on the first device from the devices included in the list; and transmitting information on the application program installed on the first device to the second device.

Description

[0001] METHOD AND APPARATUS FOR SUPPORTING MULTI-SCREEN IN A COMMUNICATION SYSTEM [0002]

The present invention relates to a method and apparatus for supporting multi-screen in a communication system.

Demand for multimedia is steadily increasing, and the need to drive the same application on multiple screens, rather than on a single screen, is also increasing in order to provide better service to users.

Accordingly, the present invention provides a method and apparatus for supporting multi-screens by interfacing a first device and a second device in a communication system.

The present invention also provides a method and apparatus for transmitting and receiving application program information between a first device and a second device in a communication system.

A method according to an embodiment of the present invention comprises: A method of supporting multi-screen in a communication system, the method comprising: receiving a list including devices capable of operating an application program of the first device in cooperation with the first device through a web server; Selecting a second device for driving an application program of the first device from the devices included in the list, and transmitting information about the application program of the first device to the second device.

An apparatus according to an embodiment of the present invention includes: A device for supporting multi-screen in a communication system, the device comprising: a receiver for receiving a list including devices capable of operating an application program of the first device in cooperation with the first device via a web server; A controller for selecting a second device for driving an application program of the first device from the devices included in the list, and a transmitter for transmitting information on the application program of the first device to the second device.

In the home network environment in which a plurality of devices are connected, the present invention finds devices usable between the first device and the second device based on web technology, and transmits application program information from the first device to the second device Alternatively, application program information may be transmitted from the second device to the first device, thereby displaying only the sub information in the second device while displaying the main screen such as broadcast information in the first device.

In addition, the present invention can view the main screen in the first device while viewing a different screen in the multi-view mode in the second device, and can display the application program of the first device from the application program of the second device, for example, EPG (Electronic Program Guide) For example, there is an effect that a broadcast or the like can be driven.

In addition, the present invention can perform control such as volume change and channel change of the first device from an application program of the second device, for example, remote control.

FIG. 1 illustrates a multi-screen device and a multi-screen device according to an embodiment of the present invention.
2 is a diagram for explaining an example in which a first device discovers and pairs a second device according to an embodiment of the present invention;
3 is a diagram for explaining an example of driving an application program of a second device in a first device according to an embodiment of the present invention,
4 is a diagram for explaining an example of driving an application program of a first device in a second device according to an embodiment of the present invention,
FIG. 5 illustrates a multi-screen device and a multi-screen device according to another embodiment of the present invention. FIG.
6 is a view for explaining an example of discovering a first device and a second device according to another embodiment of the present invention,
7 is a diagram for explaining an example of driving an application program of a second device in a first device according to another embodiment of the present invention,
8 is a diagram illustrating an example in which a web server supports multi-screens according to another embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. The following terms are defined in consideration of the functions of the present invention, and these may be changed according to the intention of the user, the operator, or the like. Therefore, the definition should be based on the contents throughout this specification.

In the embodiment of the present invention described below, a method for interworking between a plurality of devices in the same network is provided, and a specific protocol necessary for interworking between a plurality of devices is defined. In addition, it provides a common framework for transmitting and receiving application program information between a plurality of devices, and provides a remote user interface (RUI) service in the future or provides application program information among a plurality of devices And the like. It is assumed that the application includes both a web application written in HTML5 and a native application dependent on the operating system of the device.

In addition, the embodiment of the present invention described below provides the following concrete method.

- System configuration for interworking between the first and second devices

- How to find the second device in the first device

A method for driving a specific application program of the second device in the first device

- How to find the first device in the second device

- a method of driving a specific application program of the first device in the second device

FIG. 1 is a diagram illustrating a multi-screen device and a multi-screen device according to an exemplary embodiment of the present invention. Referring to FIG.

Referring to FIG. 1, a web server 130 supporting multi-screens and a first device 100 and a second device 110, which are multi-screen devices, are shown. In the first device 100, A Web browser 102 and a Web browser 102 that are commonly included in a general smart device. And a multi-screen management unit (MS Manager) 106 for interworking with the second device 110. The second device 110 includes a general application program (Web application) A Web browser 112 commonly included in the smart device and an application program such as a web application program 114 and a native application program 116 and a multi screen launcher 118 .

The web server 130 provides web application programs 104 and 114 that are executed in the first device 100 and the second device 110. The web server 130 may exist in the network, and the web application programs 104 and 114 are generally composed of HTML (Hypertext Markup Language) CSS (Cascading Style Sheet) and JavaScript.

The first device 100 refers to a main device on which an application program is run, and generally includes a DTV (Digital TV), a STB (Set Top Box), a public terminal, and the like.

The second device 110 refers to an ancillary device in which an application program is operated in cooperation with the first device, and generally includes a personal terminal such as a mobile terminal or a tablet.

The web browsers 102 and 112 execute web applications 104 and 114, and are commonly included in common smart devices and are indispensable components for web application execution.

The web applications 104 and 114 are application programs executed in the web browsers 102 and 112, and generally comprise HTML, CSS, JavaScript, images, and moving images.

The native application program 116 refers to an application program that is executed in the OS of the second device 110. It is generally compiled into binaries and is dependent on each OS. For example, an application running on an OS such as Android, iOS, or TiZen.

The MS Manager 118 is included in the first device 100 and performs necessary operations for interworking with the second device 110. [ A function of transmitting the application program information of the second device 110 at the request of the application program, a function of requesting the second device 110 to drive the application program, And a function of performing communication between the application programs of the first device 100 and the second device 110.

The MS Launcher 118 is included in the second device 110 and performs an operation required for interworking with the first device 100. [ A function of requesting the first device 100 to operate the application program at the request of the application program, a function of requesting the first device 100 to operate the application program, And a function of performing communication.

2 is a diagram illustrating an example in which a first device discovers and pairs a second device according to an embodiment of the present invention. Although the first device 200 is a DTV and the second device 206 is a personal terminal, the first device 200 and the second device 206 may be any device supporting the corresponding technology .

Referring to FIG. 2, the first device 200 transmits a URL (Uniform Resource Locator) request to the web server 212 through a first application program, for example, the web browser 202 The web browser 202 requests the transmission of the web page and receives the web page transmitted from the web server 212, for example, the main application program in step 214. In step 216, And displays it.

The web browser 202 of the first device 200 calls the getAvailableCSDevice () function to access the MS Manager 201 of the first device 200 (Step 218). Information to be transmitted in the call of the getAvailableCSDevice () function will be described in more detail with reference to Table 1, which will be described later.

The MS Manager 204 of the first device 200 inquires the list of available second devices 206 in the network through the MS Launcher 210 of the second device 206. In step 220, Which protocol to use for retrieval of the device 206 is not defined at this stage. The MS Manager 204 of the first device 200 receives a response from the MS Launcher 210 of the second device 206 and processes the received response to create a list of available devices.

The MS Manager 204 of the first device 200 transmits the list to the main application program 202 of the first device 200. In step 224, ) Displays the list of the received second devices 206 to the user and selects one of them (step 226). Here, the user can select whether to display the list to the user, 200 will not automatically define the second device 206 in the present invention. However, since the main application program is defined as a web application in the present invention, the list received in step 224 may be processed as JavaScript and displayed on a web page and selected from the user.

The main application program 202 of the first device 200 stores the information of the corresponding device when the second device 206 is selected, for example, URL information capable of driving the application program, and the like. In step 228, Is used to drive an application program of the second device 206 from the first device 200 in the future.

3 is a diagram for explaining an example of driving an application program of a second device in a first device according to an embodiment of the present invention. Although the first device 300 is a DTV and the second device 306 is a personal terminal, the first device 300 and the second device 306 may be any device supporting the technology .

Referring to FIG. 3, it is assumed that discovery and pairing of the second device 306 are completed through steps 214 through 228 of FIG. 2 described above (step 314)

The first application program of the first device 302, for example, the web browser 302 calls the sendCSURI () function to inform the MS Manager 304 of the application program information to be executed in the second device 306 (Step 316). Information to be transmitted when calling the sendCSURI () function will be described in more detail with reference to Table 1 described later.

The MS Manager 304 transmits the information of the application program to be driven to the MS Launcher 310 of the second device 306 in step 318. The MS Launcher 310 receives the CS application program, (Step 320), and the web browser 308 determines the received URI. At this time, if the web browser 308 is a native application program, the native application program is driven. (Step 322)

If the MS Launcher 310 is a Web application program, the URI is transmitted to the Web browser 308. The Web browser 308 transmits a URL request including the received URI to the Web server 312, (Step 324), receives a web page, for example, a companion application program, transmitted from the web server 312, and displays the received web page (step 326)

Since the application program of the second device 306 thus driven is driven from the application program of the first device 300, the application program of the first device 300 and the application program of the second device operate in association with each other .

Table 1 describes the APIs that can be called in the application program of the first device in more detail.

[Table 1]

4 is a diagram for explaining an example of driving an application program of a first device in a second device according to an embodiment of the present invention. Although the first device 400 is a DTV and the second device 406 is a personal terminal, the first device 400 and the second device 406 may be any devices supporting the technology .

4, the user executes a second application program (Secondary App.) In the second device 406 (step 414). At this time, an application program executable in the second device 406 is a native application program There is a web application. The web browser 408 transmits a URL request to the web server 412 to request transmission of the web page to the web server 412 and transmits the request to the web server 412 from the web server 312 Receives the received web page, for example, the second application program (step 418)

The web browser 408 of the second device 406 calls the getAvailablePrimaryDevice () function for the application program, for example, the web application program and the native application program (step 420)

The MS Launcher 410 of the second device 406 searches for a first device 400 available in the network in step 422. At this time, it is determined which protocol to use for the search of the first device 400 Do not define. The MS Launcher 410 receives a response from the first device 400 of the network and processes it to create a list of available devices (step 423)

The MS Launcher 410 forwards the list to the CS app 408 of the second device 406. The secondary application 408 of the second device 406 then sends The list of the received first devices is displayed to the user and one of them is selected. Herein, it is not defined in the present invention how the list is displayed to the user, whether the user selects the device, or whether the second device automatically selects the first device. However, in the present invention, since the list of the available devices is transmitted to the auxiliary application, the auxiliary application can process the list to be displayed and selected by the user.

A launchPrimaryAppURI () is called in the assistant application program 408 of the second device 406 to inform the MS Launcher 410 of the information of the first device 400 and the information of the main application program , For example, the URI is transmitted (step 426)

The MS Launcher 410 transmits the information of the application program (Primary App.) To be executed in the first device to the MS Manager 404 of the first device 400 with this information, and in step 428, The URI is transferred to the first application program 402 (step 430)

The web browser 402 judges the received URI, and if it is a native application program, drives a native application program. If the web application 402 is a web application program, the web browser 402 delivers the URI to the web browser 402 of the first device 400. The web browser 402 transmits a URL request including the received URI to request the web server 412 to transmit the web page to the web server 412 in step 434 and transmits the web page received from the web server 412, And receives and displays the primary application program (step 436).

Since the application program of the first device 400 driven in this manner is driven from the application program of the second device 406, the application program of the second device 406 and the application program of the first device 400 are associated with each other So that it can be operated.

Table 2 describes the APIs that can be called in the application of the second device in more detail.

[Table 2]

FIG. 5 is a diagram illustrating a multi-screen device and a multi-screen device according to another embodiment of the present invention.

Referring to FIG. 5, a Web server 530 that supports multi-screens and a first device 500 and a second device 520, which are multi-screen devices, are shown. In the first device 500, A Web browser 504 and a Web browser 504 which are commonly included in a general smart device. (MS Manager) 508 for interworking with an application program, for example, a web application (Web App.) 506, a native application program (Native App.) 502 and a second device 520 And a second device 520. The second device 520 includes a web browser 522 commonly included in a general smart device, an application program, for example, a web application program 522 and a native application program 526, And a multi-screen launcher (MS Launcher) 528.

5, the configuration and roles of the web applications 506 and 524, the native applications 502 and 526, the MS manager 508, and the MS Launcher 528 including the web browsers 504 and 522 are the same as those of FIG. 5, the UPnP (Universal Plug and Play) 512, the HTTP server 514, the WebSocket server 510, and the UPnP (Universal Plug and Play) server are configured with the detailed module configuration of the MS Manager of the first device 500 Only the UPnP 530 and the HTTP server 532 will be described in detail with the module configuration of the MS Launcher 528 of the second device 520. [

The UPnPs 512 and 530 are commonly included in the first device 500 and the second device 520 and perform operations necessary for device-to-device discovery. Specifically, SSDP (Simple Service Discovery Protocol) is used to discover another device in the network and to inform the network of its existence.

The HTTP servers 514 and 532 are commonly included in the first device 500 and the second device 520, and receive and receive requests through the HTTP protocol. Specifically, the present invention plays a role of receiving an HTTP GET and an HTTP POST request from another device to process an application protocol. In the present invention, the HTTP server performs only a simple role as described above, and does not play the role of a general web server for distributing web pages.

The WebSocket server 510 is included only in the first device 500 and plays a role of processing the WebSocket protocol. In the present invention, the second device 520 receives a WebSocket request and processes the received WebSocket request to enable communication between the application programs of the first device 500 and the second device 520.

Hereinafter, detailed operations performed by the MS manager 508 and the new module added to the MS Launcher 528 will be described with reference to FIGS. 6 through 7. FIG.

6 is a diagram for explaining an example of discovering a first device and a second device according to another embodiment of the present invention. Although the first device 600 is a DTV and the second device 612 is a personal terminal, the first device 600 and the second device 612 may be any device supporting the technology . 6 corresponds to the concrete operation of the detailed protocol of getAvailableCSDevice () in Table 1 described above.

Referring to FIG. 6, an example in which the first device 600 discovers the second device 612 will be described first. The UPnP module 606 of the first device 600 transmits M-SEARCH to find available second device 612 of the network and then executes the command as shown in Table 3 using the SSDP protocol in step 622 do.

[Table 3]

The UPnP module 618 of the second device 612 receiving the M-SEARCH sends a Response with LOCATION Header as shown in Table 4 to inform that it is the corresponding device (Step 624)

[Table 4]

The UPnP module 606 of the first device 600 receiving the response requests the device description of the device using the LOCATION URL information of the LOCATION header in the response header (HTTP GET request) )

The HTTP Server 620 of the second device 612 includes a specific header (x-Application-URL) capable of executing the application program of the second device 612 in the following HTTP GET request, 5 (HTTP Response) (Step 628)

[Table 5]

Next, an example in which the second device 612 discovers the first device 600 will be described.

The UPnP module 618 of the second device 612 transmits M-SEARCH to find the available first device 600 of the network (step 630), and executes the command as shown in Table 6 using the SSDP protocol do.

[Table 6]

The UPnP module 606 of the first device 600 receiving the M-SEARCH sends a Response with LOCATION Header as shown in Table 7 below to inform that it is the corresponding device (step 632)

[Table 7]

The HTTP Server 620 of the second device 612 receiving the response requests the device description of the device using the LOCATION URL information of the LOCATION header in the response header (HTTP GET request) (Step 634 )

In step 636, the first device 600 responds to the HTTP GET request by including a specific header (x-Application-URL) capable of executing an application program of the first device 600 as follows. The first device 600 has a web socket server 610 capable of performing application program communication in an application program of the second device 612. The x-WebSocket header includes a URL (Step 628)

The HTTP responses transmitted in steps 636 and 638 are shown in Table 8 below.

[Table 8]

7 is a diagram for explaining an example of driving an application program of a second device in a first device according to another embodiment of the present invention. Although the first device 700 is a DTV and the second device 712 is a personal terminal, the first device 700 and the second device 712 may be any devices supporting the technology . 7 corresponds to a concrete operation of the detailed protocol of sendCSURI () in Table 1 described above.

Referring to FIG. 7, the process of driving an application program of the second device 712 in the first device 700 will be described as follows.

The MS Manager 704 of the first device 700 receives two arguments from the browser 702 of the first device 700 via the sendCSURI () function (step 722). The first argument, uint target_device, Information of the second device 712 stored in the MS Manager 704 is matched with the ID value of the second device 712 to which the application program is to be driven and the x-Application-URL information of the second device 712 .

Table 9 shows an example of the information table of the second device 712 managed by the MS Manager 704.

[Table 9]

The MS Manager 704 of the first device 700 transmits an HTTP Post request as shown in Table 10 to the HTTP server 720 of the second device 712 with the URL obtained in step 722. In step 724,

[Table 10]

At this time, the application program to be executed by the second device 712 is described by a URI in CSAppPath and a char * URI value which is the second argument of the sendCSURI () API. When the application program of the second device 712 is required to be transmitted, an argument is sent to the HTTP body in the format shown in Table 10.

The HTTP server 720 of the second device 712 parses the requested POST and transmits the CSAppPath information together with the argument of the Body to the MS Launcher 716. (Step 726) The MS Launcher 716 transmits the CSAppPath information The application program of the second device 712 is driven by the URI (steps 728 and 730), and the result of the application program is transmitted to the HTTP server 720 of the second device 712 (step 732)

The HTTP server 720 of the second device 712 transmits a response code as shown in the following Table 11 to the MS Manager 704 of the first device 700 according to the result (201 CREATED) (Step 734)

[Table 11]

Next, a process of driving an application program of the first device 700 by the second device 712 will be described.

The MS Launcher 716 of the second device 712 receives two arguments from the browser 714 of the second device 712 via the sendPrimaryAppURI () function (step 736). The first argument, uint target_device, Application-URL information of the first device 700 by matching the first device 700 information table stored in the MS Launcher 716 with the ID value of the first device to be driven by the application program .

Table 12 shows an example of the first device 700 information table stored in the MS Launcher 716.

[Table 12]

The MS Launcher 716 of the second device 712 transmits an HTTP Post request as shown in Table 13 to the first device 700 using the URL obtained in step 736. In step 738,

[Table 13]

At this time, the application program to be executed by the first device 700 is described as a URI in the PrimaryPath and a char * URI value as the second argument of the ssendPrimaryAppURI () API. In addition, when it is desired to transmit a factor required for driving the application program of the first device 700, an argument is sent to the HTTP body in the format shown in Table 13.

The HTTP server 708 of the first device 700 parses the requested POST and transmits the PrimaryAppPath information to the MS Manager 704 together with the argument of the Body in step 740. In step 740, The application program of the first device 700 is driven by the URI (steps 742 and 744), and the result of the application program is transmitted to the HTTP server 708 of the first device 700 (step 746)

The HTTP server 708 of the first device 700 transmits a response code as shown in the following Table 14 to the second device 712 according to the result (201 CREATED) (step 748)

[Table 14]

8 is a diagram illustrating an example in which a web server supports multi-screens according to another embodiment of the present invention.

Referring to FIG. 8, it is assumed that a user receives broadcast information from a broadcast channel using a first device, for example, the DTV 802, and watches a movie. The user wants to receive additional information about the movie through the second device, for example, the terminal 804 while watching the movie. Here, the additional information for the movie may be, for example, casting information, synopsis of a movie, online shopping, and the like.

The first device 802 drives the application program of the second device 804 capable of displaying the additional information and provides the application to the user.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments. Therefore, the scope of the present invention should not be limited by the illustrated embodiments, but should be determined by the scope of the appended claims and equivalents thereof.

It will also be appreciated that the method and apparatus for supporting multi-screens in accordance with embodiments of the present invention may be implemented in hardware, software, or a combination of hardware and software. Such arbitrary software may be stored in a memory such as, for example, a volatile or non-volatile storage device such as a storage device such as ROM or the like, or a memory such as a RAM, a memory chip, a device or an integrated circuit, , Or a storage medium readable by a machine (e.g., a computer), such as a CD, a DVD, a magnetic disk, or a magnetic tape, as well as being optically or magnetically recordable. The method of supporting the multi-screen of the present invention can be implemented by a computer or a mobile terminal including a controller and a memory, and the memory is suitable for storing a program or programs including instructions embodying the embodiments of the present invention It is an example of a machine-readable storage medium.

Accordingly, the invention includes a program comprising code for implementing the apparatus or method as claimed in any of the claims herein, and a storage medium readable by a machine (such as a computer) for storing such a program. In addition, such a program may be electronically transported through any medium such as a communication signal transmitted via a wired or wireless connection, and the present invention appropriately includes the same.

Also, a method and apparatus for supporting multi-screens according to an embodiment of the present invention may receive and store the program from a program providing apparatus connected by wire or wireless. The program providing apparatus may include a memory for storing a program including instructions for causing the graphic processing apparatus to perform a predetermined content protection method, information necessary for the content protection method, and the like, and performing wired or wireless communication with the graphic processing apparatus And a control unit for requesting the graphics processing apparatus or automatically transmitting the program to the transmission / reception apparatus.

Claims (2)

A method of supporting multi-screen in a communication system,
Receiving a list including devices capable of operating an application program of the first device in cooperation with the first device through a web server,
Selecting a second device for driving an application program of the first device from the devices included in the list;
And transmitting information about an application program of the first device to the second device.
An apparatus for supporting multi-screen in a communication system,
A receiver for receiving a list including devices capable of operating an application program of the first device in cooperation with the first device through a web server,
A controller for selecting a second device for driving an application program of the first device from the devices included in the list,
And transmitting the information about the application program of the first device to the second device.

Images