KR101277831B1 - Network-Based Collaborative Displaying Method and Apparatus - Google Patents

Abstract

The present invention enables a collaborative display in which one device divides and outputs a single content by simply disposing a plurality of mobile devices without physical connection.
According to an embodiment of the present invention, there is provided a collaborative display method comprising: generating, at one of a plurality of devices, one of user input information and device arrangement information; Transmitting, by a device, the generated information to a display control apparatus that performs control of a collaborative display; The device receiving display control information from the display control device; And rendering all or a part of the content to be displayed in the group on the screen of the device based on the display control information.

Description

Network-Based Collaborative Displaying Method and Apparatus}

The present invention relates to a network-based collaborative display method and apparatus, and more particularly, to a network-based collaborative display method and apparatus for displaying one content through the screen of a plurality of mobile devices.

Recently, the use of portable mobile devices such as smartphones and tablet PCs is increasing rapidly. A user may communicate with a web server and receive various contents through the Internet in a portable mobile device. However, in general, since portable mobile devices have a limited screen size, the form of content consumption in a mobile computing environment is more limited in a desktop PC.

As a conventional prior art displaying a single content through multiple mobile device screens, the article "A Collaborative Web Browsing System for Multiple Mobile Users" (Proc. Of IEEE Int'l Conf. On Pervasive Computing and Communications (PerCom 2006), pp 22-35, Mar. 2006).

This paper proposes a method of parsing-segmentation of hierarchical HTML documents to render each piece of content in multiple pieces. However, this technique is a method of rendering different nodes for each device by parsing HTML web content that is specialized in the root-node hierarchy, and content that cannot be represented in a hierarchy (for example, large image and map information). Etc.) is not available.

Another prior art related to collaborative displays is EP 1271841 (name of invention: Server apparatus, mobile terminal, contents distribution method, content reception method, and program product), in which case the server-client structure In this case, the server separately transmits content to be rendered by each client in a self-contained content storage medium and has a module that can respond to user input of the client. However, in the present invention, since the server includes a content storage medium, the types of contents used are extremely limited.

Moreover, both of the above prior arts do not consider mobile device relocation and session management accordingly, and thus there is a problem in that they cannot cope with mobility and freedom of deployment unique to mobile devices, and thus display arbitrary content through multiple mobile devices. A new collaborative display method is needed.

According to an embodiment of the present invention to propose a network-based collaborative display method and apparatus that can consume and control a single content in cooperation with multiple devices having different resolutions and different screen sizes without physical connection between devices For work purposes.

According to an embodiment of the present invention, a method of generating mutual placement information with a neighboring second device in a first device on a network, wherein each device includes at least one of an acceleration sensor, a touch sensor, and an illumination sensor. Generating sensing information by using; Generating device placement information from the sensing information in each device; And generating mutual placement information of the first and second devices based on the device placement information of the first and second devices.

According to another embodiment of the present invention, a device on a network, the device comprising: a layout recognition unit for generating layout information of the device; A transmitter / receiver for transmitting the device arrangement information to a display control device and receiving display control information from the device; And a rendering unit configured to render all or a portion of content to be displayed on the screen of the device on the screen of the device, based on the display control information, wherein the arrangement information of the device includes an acceleration sensor and a touch sensor of each device. , And based on sensing information from one of the illuminance sensors.

According to another embodiment of the present invention, a display control apparatus capable of providing display control information to the group for a cooperative display of a group including one or more devices in a network, comprising: device device information and device arrangement information from the device A transmission / reception unit configured to receive at least one of the two and to transmit the display control information to the device; A session manager which receives the device information of the device from the transceiver and generates and stores session state information for a cooperative display for all devices belonging to the same group; A layout manager configured to calculate content division and distribution information indicating a ratio of dividing and distributing content for each device, based on device information and device arrangement information of devices for all devices belonging to the group; And a message processor configured to generate the display control information based on the content division and distribution information, wherein the arrangement information of the device is based on sensing information from one of an acceleration sensor, a touch sensor, and an illumination sensor of each device. Provided is a display control device characterized in that it is generated.

According to another embodiment of the present invention, there is provided a collaborative display method in a group including one or more devices in a network, the method comprising: generating, at a first device of the one or more devices, one of user input information and device arrangement information; step; Transmitting, by the first device, the generated information to an apparatus for performing control of a collaborative display; The first device receiving display control information from the apparatus; And rendering all or a part of contents to be displayed in the group on a screen of the first device based on the display control information, wherein the device arrangement information includes an acceleration sensor, a touch sensor, And generated based on sensing information from one of the illuminance sensors.

According to another embodiment of the present invention, a method for providing display control information to a collaborative display group including one or more devices in a network, the method being provided on a screen of the first device from a first device of the one or more devices. Receiving user input information about content; Generating display control information to instruct each device of the one or more devices based on the user input information; And transmitting the display control information to each of all devices in the group.

According to another embodiment of the present invention, there is provided a method for providing display control information to a collaborative display group including one or more devices in a network, the method comprising: receiving device placement information from a first one of the one or more devices; Calculating division and distribution information of content for all devices in the group based on the received placement information; Generating display control information for instructing each device of the at least one device based on the partitioning and distribution information of the calculated content; And transmitting the display control information to each of all devices in the group.

According to an embodiment of the present invention, a plurality of mobile devices may be divided into a single content and output a single content and control the corresponding content by simply arranging a plurality of mobile devices without a physical connection. This allows the limited user experience in mobile computing environments with small screen sizes to be extended to the user experience level on desktop computers.

1 is an overall network configuration according to an embodiment of the present invention.
2 is a block diagram showing an internal configuration of a client device 10 according to an embodiment of the present invention.
3 is a block diagram showing an internal configuration of a display control server 50 according to an embodiment of the present invention.
4 is a flowchart illustrating a process of forming a collaboration group according to an embodiment of the present invention.
5 is a flowchart illustrating a process of processing user input information of a client device 10 according to an embodiment of the present invention.
6 is a flowchart illustrating a process of processing device arrangement information of a client device 10 according to an embodiment of the present invention.
7A and 7B show an example of a cooperative display according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features, and advantages of the present invention will become more readily apparent from the following description of preferred embodiments with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described herein but may be embodied in other forms. Rather, the embodiments disclosed herein are provided so that the disclosure can be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. In this specification, when an element is referred to as being on another element, it may be directly formed on another element, or a third element may be interposed therebetween.

The terminology used herein is for the purpose of illustrating embodiments and is not intended to be limiting of the present invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. As used in the specification, "comprises" or "comprising" does not exclude the presence or addition of one or more other components in addition to the components mentioned.

Hereinafter, the present invention will be described in detail with reference to the drawings. In describing the following specific embodiments, various specific details are set forth in order to explain and understand the invention in more detail. However, those skilled in the art can understand that the present invention can be used without these various specific details. In some instances, it should be noted that portions of the invention that are well known in the description of the invention and are not significantly related to the invention do not describe confusion in describing the invention.

1 shows an overall network configuration according to an embodiment of the present invention. In the illustrated embodiment, the entire network includes a number of client devices 10, content cloud 30, and display control server 50.

The client device 10 is a conventional portable mobile device, and may be, for example, a mobile device such as a smartphone, PDA, tablet PC, netbook, or the like. Hereinafter, in the present specification, these devices will be referred to as 'client devices' or simply 'devices'.

Each of the client devices 10 according to an embodiment of the present invention for collaborative display receives general content from the content cloud 30 and transforms (moves, enlarges, reduces) the content from the display control server 50. And display control information indicating rotation, and the like, and display only the content divided and distributed among the contents received from the content cloud 30 based on the display control information.

The client device 10 according to the invention also comprises an apparatus which makes it possible to detect a placement relationship with neighboring devices. According to an embodiment of the present invention, when arranging a plurality of devices 10, the arrangement order information of the devices may be obtained by sequentially touching the touch sensor or the acceleration sensor displacement information generated by hitting the devices 10 with each other. The device 10 may include such an acceleration sensor or a touch sensor. The information sensed by these various sensors is transmitted to the display control server 50, which parses the information to secure physical placement information of the entire device.

In addition, the client device 10 according to the present invention is a user's input information, that is, click, drag and drop, flicking (flicking the screen with a finger), pinch (pinch: touch two fingers on the screen to spread or Narrowing), and the like, to the display control server 50. The detailed configuration of such a client device 10 will be described later with reference to FIG.

In FIG. 1, the content cloud 30 includes any server or network device capable of providing content to the client device 10. The content cloud 30 may mean a web server, but the present invention is not limited thereto and may include any server capable of transmitting arbitrary content to the client device 10. In the present specification, 'content' refers to a file of any format that the content cloud 30 provides to the device 10. For example, it includes various types of files such as document files, image files, as well as web content composed of HTML, CSS, JavaScript, and the like.

In FIG. 1, the display control server 50 may correspond to the physical screen layout of the plurality of client devices 10, and may display the layout and rendering range of the content and the movement and enlargement level information of the content in consideration of different resolutions for each device. By sending it to, multiple devices can collaborate to display a piece of content. Although the display control server 50 is shown as a separate component from the client device 10 in the embodiment shown in FIG. 1, in alternative embodiments, at least one of the plurality of client devices 10 may be provided. It may replace the function of the display control server 50. That is, in this case, a separate server 50 does not exist, but a plurality of client devices 10 including one or more client devices having the function of the server 50 may collaborate with each other to display content. The display control server 50 will be described later with reference to FIG. 3.

Meanwhile, in FIG. 1, communication between the client device 10 and the content cloud 30, between the client device 10 and the display control server 50, and each server in the content cloud 30 is performed by a network such as a mobile communication network and the Internet. Through the network. The network may be a combination of any one or two or more of various types of mobile communication networks, such as 3G, 4G, local area networks, telecommunications networks, Wi-Fi, Bluetooth, and the like.

Now, the client device 10 and the display control server 50 will be described in detail with reference to FIGS. 2 and 3, respectively.

2 is a block diagram showing an internal configuration of a client device 10 according to an embodiment of the present invention. In the illustrated embodiment, the device 10 includes a layout recognizer 11, a user input processor 13, a message processor 15, a message transceiver 17, a renderer 19, and an exception processor 21. Include.

The layout recognizing unit 11 communicates with an acceleration sensor, a gravity sensor, a touch sensor, or an illuminance sensor in the device 10 to generate 'device layout information' of the device 10 from these sensors. Here, the 'device arrangement information' is, for example, information about which direction the device 10 itself is placed in the horizontal or vertical direction, and in what order the device 10 and the neighboring device 10 are adjacent to each other. The information may indicate any one of the information regarding the arrangement or both information, and in the present specification, these information will be collectively referred to as 'device arrangement information'.

Information regarding the arrangement of the device itself among the arrangement information of the device is obtained from a gravity sensor or an acceleration sensor included in the device 10. Information regarding the order of placement with other neighboring devices may be obtained by various methods. In an embodiment of the present invention, for example, the acceleration sensor displacement information generated by colliding the device with a neighboring device with each other may be parsed or aligned. After arranging, the information on the arrangement order can be secured by touching the screens of the devices in turn.

Looking at the first method above, by comparing the values of how the two devices are arranged with each other by the acceleration value of each axis of x / y / z caused by hitting the two devices 10 with each other between the two devices 10 Relationship, i.e., where each device is located in its left-right or up-down array, where it is located, and thus the placement information with the neighboring devices obtained from each device 10, 50, and the server 50 may obtain the mutual arrangement information of the entire devices by analyzing the device arrangement information received from each device 10.

For example, suppose you have two neighboring devices left and right within a collaborative display group. If you bump two neighboring devices against each other, the acceleration sensor on the left device is shocked when the device accelerates to the right and suddenly stops. Detect. That is, the acceleration sensor generates displacement information according to the acceleration change, and the layout recognizing unit 11 generates device arrangement information from the displacement information and transmits it to the server 50. In addition, the acceleration sensor of the right device generates displacement information according to the impact when the device accelerates to the left and suddenly stops, and the layout recognition unit 11 of the device generates device arrangement information from the displacement information to generate a server ( 50). The server 50 may analyze device placement information received from each device to obtain mutual placement information on how each device is disposed as a whole.

In the second method, that is, by sequentially touching the screens of the devices to obtain arrangement order information of the devices, when all devices to be used for the collaborative display are arranged in a line and the screens are touched in sequence, the touch information is transferred to the display control server 50. The server 50 is a method of determining the arrangement order of all devices by comparing the time difference of the touches based on the touch information received from each device. In this case, the devices may be arranged horizontally or vertically, and by touching the screen sequentially, it is not possible to specify which of the horizontal array and the vertical array. The user may additionally receive information specifying the form from the user and specify one of the horizontal array and the vertical array.

For example, if the device 10 includes an illuminance sensor, the arrangement of the devices may be arranged in a row, and then the illuminance sensor of each device may be covered by hand to determine the arrangement order of the devices. Various methods can be used.

The user input processing unit 13 receives a user input 20 for manipulating the contents of the screen and generates user input information. Examples of user input 20 for manipulating content on the screen include clicking, dragging and dropping, flicking, flipping, or pinching two fingers on the screen to expand or narrow the content. Rotation (eg, rotating only image files in the content), and the like.

The message processing unit 15 generates various types of information generated in the device 10 in a message form that the server 50 can read, and parses the message received from the server 50 to other functional units in the device 10. It serves to convey.

Specifically, in the illustrated embodiment, the message processing unit 15 receives the device arrangement information from the layout recognition unit 11 and generates it in the form of a message that can be read by the server 50, and from the user input processing unit 13. Received various user input information can also be generated in the form of a message that can be read by the server 50. The message transmitted from the display control server 50 is received through the message transmission / reception unit 17, parsed, and transmitted to another function unit (for example, the rendering unit 19) in the device.

The message transmitting and receiving unit 17 transmits a message generated by the message processing unit 15 to the server 50 and receives a message transmitted from the server 50. In this case, the communication module may be extended to application protocols such as UDP, multicasting, and broadcasting in addition to the basic TCP / IP protocol, and the present invention is not limited to any one.

The rendering unit 19 receives display control information indicating how to display the content from the display control server 50, and outputs the content received from the content cloud 30 on the screen based on the control information.

In addition, the exception processing unit 21 serves to handle when an error occurs in one of the functional units, the exception processing function is a general well-known function, detailed description thereof will be omitted.

Referring now to Figure 3 will be described with respect to the display control server 50 according to an embodiment of the present invention. In the illustrated embodiment, the display control server 50 may include a message transceiver 51, a session register 53, a session manager 55, a layout manager 57, a message processor 59, and an exception processor 61. Include.

The message transceiving unit 51 receives a message generated and transmitted by the message processing unit 15 of each client device 10 and sends a message generated by the message processing unit 59 of the server 50 to each client device 10. send. In this case, the communication module may be extended to application protocols such as UDP, multicasting, and broadcasting in addition to the basic TCP / IP protocol, and the present invention is not limited to any one.

The session register 53 and the session manager 55 have a function of handling the first registration of any device 10 in the device group for the cooperative display. The session register 53 receives the device information of the device 10 requesting a new participation and transfers the device information to the session manager 55. In the present specification, the 'device information' refers to device information including information about a device type, a screen size, and a screen resolution (eg, DPI). The session manager 55 receives the device information from the session registerer 53 and manages session state information of all client devices 10 existing in the same group.

The layout manager 57 divides and distributes the content based on the device information, that is, device arrangement information of all the client devices 10 subscribed to the group, information on a device type, a screen size, and a screen resolution. Compute the information. Herein, 'content division and distribution information' refers to information indicating a ratio of dividing and distributing content for each device based on the arrangement of devices constituting a group for collaborative display. For example, as shown in FIG. When three client devices 10 of the same model are arranged side by side, "The content screen divided by 1/3 is divided into the leftmost device by distributing the leftmost screen among the content screens divided into 1/3 by the leftmost device The device may include a middle screen among content screens divided by 1/3, and the rightmost screen of content screens divided by 1/3 may be allocated to the rightmost device.

The message processor 59 receives the message transmitted by the client device 10 from the message transceiver 51, parses it, and delivers the message to the layout manager 57 as necessary. In addition, the message processing unit 59 generates display control information to be transmitted to each client in the form of a message that can be read by the client device 10 and transmits the same to the message transmitting and receiving unit 51. Here, the 'display control information' may include, for example, information for instructing deformation, such as movement, enlargement, reduction, and rotation of content, such as movement, enlargement, reduction, and rotation of the content. The information indicative of the transformation will be specifically called 'content transformation information'. In addition, in one embodiment, the display control information is generated with reference to the content division and distribution information and the screen size and resolution of each device.

The exception processing unit 61 plays a role when an error occurs in one of the functional units. For example, if one device suddenly falls out of the group while a group of devices are performing a collaborative display, the exception processing unit 61 recognizes this and notifies the session register 53, and the session register 53 is the session manager 55. In this case, the session state information of the devices in the group can be changed.

In alternative embodiments, the client device 10 described with reference to FIG. 2 may include the functionality of the display control server 50 described with reference to FIG. 3. That is, the client device 10 illustrated in FIG. 2 includes all the functional units illustrated in FIG. 3, or the message transceiver 51 and the exception processor 61 may include the message transceiver 17 and the exception processor (FIG. 2). 21 and the corresponding functional units 53, 55, 57, and 59 may be omitted. In this case, even if a separate server 50 does not exist, a plurality of client devices including the client device 10 having the function of the server 50 may display one content by cooperating with each other. Therefore, according to the embodiment of the present invention, the 'display control server 50' referred to in this specification may be interpreted by replacing the client device 10.

4, a process of forming a collaboration group according to an embodiment of the present invention will be described. 4 shows a process in which any device 10 joins a group for a collaborative display, which does not show all the steps of the participation process and shows only the steps necessary to describe one embodiment of the invention. Thus, of course, additional steps in addition to those shown may be added to any position in the flowchart.

Referring to Fig. 4, first, in step S101, the user activates the cooperative display function of the client device 10. Here, activation of the cooperative display function may mean, for example, running a dedicated application for the cooperative display.

When the collaboration display function is activated, the next step (S103), the device selects whether to create a new collaboration display group or join an existing group. If it is to create a new group directly proceeds to the step (S107) of transmitting the device information of the device to the display control server 50, otherwise, if you are to join the existing group to select which group to join (S105) Is added.

In one embodiment, step S105 of selecting a group is performed while communicating with the display control server 50. For example, since the display control server 50 stores and manages both the list of the cooperative display groups currently existing in the session manager 55 and the session state information of devices belonging to each group, the display control server 50 is stored in the session manager 55. The group list can be sent to the client device 50 for selection from there. Also, as an alternative embodiment, the location information of the client may be acquired using GPS or the like, from which the cooperative display groups close to the location of the corresponding device may be displayed and the user may select one of them.

If the device 10 creates a new group, the device information of the device to form the new group is transmitted from the device 10 to the display control server 50 in step S107. In this case, the device information means device information including information on a device type, a screen size, and a screen resolution. The display control server 50 having received the device information transmits this information to the session manager 55 through the session register 53, and the session manager 55 generates and stores device session state information for each group ( S109).

If the device 10 joins an existing group, the device information transmission step S111 is performed after the group selection step S105, and the device information of the device 10 is transmitted to the display control server 50. . In this case, the device information means device information including information on a device type, a screen size, and a screen resolution. The display control server 50 having received the device information transmits this information to the session manager 55 through the session register 53, and the session manager 55 updates and stores device session state information of the existing group. (S113).

Thereafter, in step S115, device arrangement information of the corresponding device 10 to join the existing group is transmitted to the server 50. As described above with reference to FIG. 2, the arrangement information of the device may include any one of information regarding the arrangement (horizontal or vertical arrangement) of the device itself or information indicating the arrangement order with neighboring devices in the group.

When the device 10 to join the group transmits its placement information to the server 50 in step S115, the server 50 receives the information and transmits the information to the message processing unit 59, and in step S117. The message processing unit 59 parses this information and delivers it to the layout management unit 57. Thereafter, in step S119, the layout manager 57 calculates content division and distribution information based on device arrangement information, device type, screen size, and screen resolution of all the client devices subscribed to the group.

On the other hand, if the device 10 is a process of forming a new group, the flowchart of FIG. 4 may end in step S109. That is, since only one device is registered in the group, device arrangement information, content division, and distribution information for the cooperative display are not necessary at this stage.

However, in alternative embodiments of the present invention, steps S115, S117, and S119 may be further added after step S109. That is, even if there is device information and device arrangement information for only one client device 10, the server 50 generates a device state information by generating a group consisting of this one device (step S109) and divides the content division and distribution information. It can calculate (step S119). At this time, the content division and distribution information may be content to display all of the content on the screen of the device 10 or may be content to display a part of the content. In this case, although the plurality of client devices are not collaboratively displayed, even if there is only one client device, when the corresponding device 10 transmits the device arrangement information to the server 50, the predetermined display control information is received from the server 50. Since the display can be displayed on the screen of the device 10, as a result, the server 50 can control the display contents of the device 10 regardless of the actual arrangement of the client device 10. Done.

Through the above flow, the process of registering a client device to form a new collaborative display group or to join an existing collaborative display group may be performed. The flowchart illustrated and described in FIG. 4 is merely for device registration. It is just an example and some steps may be reordered or new steps may be added.

A process of processing user input information of the client device 10 according to an embodiment of the present invention will now be described with reference to FIG. 5. Examples of user input here include click, drag and drop, flicking, pinch, rotation of content, and the like, such as up, down, left, right, up, down, and rotation of the currently displayed content, and new URLs. Content modification such as display of the content of the user occurs.

Referring to FIG. 5, first, when the user input is executed in the client device 10 in step S201, the user input information is displayed on the display control server 50 by the message processing unit 15 in the device 10. Generates a message in a form that can be read and transmits it to the server 50. Then, in step S203, the message transmitting and receiving unit 51 of the server 50 receives it and delivers it to the message processing unit 59, and in step S205, the message processing unit 59 parses the message to interpret the user input information. do.

However, in this process, since two or more client devices 10 exist in one group, user input may be simultaneously received by two or more devices 10. In an embodiment of the present invention, in order to avoid such a situation, the device 10 may include event lock information in the user input information transmitted to the server 50 to transmit the event lock information. The event lock information has a content to be ignored even when receiving user input information from another device while receiving user input information from one device. In this case, the server 50 may receive user input information from a plurality of devices. The processing of the corresponding information may be performed only on the user input information received first among the user input information including the event lock information.

Therefore, for each of the at least one user input unit, the device that has transmitted the user input information which is processed is operated as a master device for the user input, and the rest of the devices in the same group receive only the result of the content modification according to the user input. It can be seen that it operates as a slave device that receives the data from 50) and renders it on its screen. In one embodiment, the master-slave relationship may be changed for each user input unit. That is, when all the rendering processing (step S215) according to one user input is completed, if a user performs user input in any device that was a slave device until then, this device operates as a master device for the user input from then on. And the remaining devices in the same group act as slave devices.

Returning to the drawing again, in step S207 the message processing unit 59 generates display control information to instruct each device according to the user input information parsed and interpreted in step S205. In more detail, the display control information herein refers to content deformation information for instructing deformation, such as movement, enlargement, reduction, and rotation, of content, for each device, and includes content division and distribution information calculated by the layout manager 57. The screen size and resolution of each device are used to indicate how much to move, enlarge, reduce, or rotate the content for each device.

When content modification information to be indicated for each device is generated in step S207, content modification information corresponding to each device is transmitted to all devices in the group in step S209. The content modification information is received by the message transmission / reception unit 17 of each device (step S211), the message processing unit 15 receives and parses this message (step S213), and the content is rendered by the rendering unit 19 according to the parsed result. The image is rendered on the screen of each device by moving, enlarging, reducing, or rotating it (step S215).

Meanwhile, in the user input, for example, the master device may click a hyperlinked string on an existing content screen or input a new URL address in the URL address box. Since there is no communication, the slave device cannot know the URL address entered by the master device. Therefore, in one embodiment of the present invention, in this case, the URL address input from the master device is transmitted to the server 50 as user input information, and the server 50 treats this URL address as display control information and transmits it to each device. do. Then, each device 10 may request content from a server of a corresponding URL, receive content from the server, and render the content.

As described above, the server 50 generates and transmits display control information for each device in the group to each device according to the user input information input from the master device, and accordingly, the plurality of device screens. Through this, a collaborative display that displays one piece of content is possible.

Referring now to FIG. 6, a description will be given of a process for relocating the client device 10 according to an embodiment of the present invention. Here, the relocation of the device means changing the position of the devices, such as placing the device itself horizontally or vertically or vice versa, or changing the order among the devices placed in a line.

The device arrangement information generated by relocating the device 10 is basically processed in a flow similar to that of the user input information shown in FIG. However, since device arrangement information is based on the calculation of content division and distribution information performed by the layout management unit 57, as can be seen from the comparison between FIGS. 5 and 6, the content management is performed by the layout management unit 57 as the device is rearranged. And recalculating distribution information (S307).

To briefly describe the flow of Fig. 6, in step S301, relocation occurs at any client device 10 in the group. The device relocation is recognized by the layout recognizing unit 11 in the corresponding device 10 as described with reference to FIG. 2 so that the device arrangement information is transmitted to the message processing unit 15, and the message processing unit 15 controls the display. The server 50 generates a message that can be read and transmits the message to the server 50.

As described with reference to FIG. 5, in this case, a situation may occur in which two or more client devices 10 are simultaneously relocated within a group. In one embodiment of the present invention, as in the case of FIG. It is possible to include the event lock information in the device arrangement information to be transmitted to the server 50 so that the server 50 transmits only the device placement information including the event lock information received first. Processing can proceed.

The message transmitting and receiving unit 51 of the display control server 50 receives the device arrangement information and transmits it to the message processing unit 59 in step S303. In step S305, the message processing unit 59 parses and arranges the message. Interpret the information.

If the message processing unit 59 determines that the client device 10 has been rearranged as a result of the interpretation of the message, the layout information is transmitted to the layout management unit 57, and in step S307, the layout management unit 57 includes the new layout information. Content division and distribution information are calculated based on the arrangement information of all the client devices 10 in the group, the screen size and screen resolution of each device, and the like.

Next, in step S309, the message processing unit 59 generates content modification information to instruct each device by referring to the content division and distribution information calculated in step S307 and the screen size and resolution. Thereafter, the content modification information is transmitted to all the devices 10 in the group (step S311), and all the devices 10 receive the content modification information from the message transmitting and receiving unit 17 (step S313). The message processing unit 15 in each device 10 receives and parses this content modification information (step S315), and the rendering unit 19 renders the content on the screen according to the parsed result (step S317).

Through the steps shown in FIG. 6, when relocation occurs in any device 10, the server 50 recalculates content division and distribution information according to the new device arrangement information, and calculates content modification information for each device based on the calculation result. It generates and transmits to each device, thereby enabling a collaborative display that displays a single content through a plurality of device screens.

An example of a cooperative display according to the present invention will now be described with reference to FIGS. 7A and 7B. 7A and 7B show a cooperative display of web content transmitted from a general web server. As shown in FIG. 7A, three different types of client devices are arranged side by side to form a group for a collaborative display, and the main screen of an arbitrary portal site (www.naver.com) is currently a collaborative display. Is indicated by. In this state, when the user drags upwards on the screen of the rightmost device, the user recognizes the operation as a user input and generates a message for the user input information in the message processing unit 15 in the device and transmits the message to the display control server 50. do.

The server 50 receives this message, parses the message in the message processing unit 59 according to the flowchart shown in FIG. 5 (step S205), and generates content variation information for each device according to a user input (drag operation) ( Step S207) is transmitted to each device (step S209). Each device receives the content modification information, parses it in its own message processing unit 15, and renders the deformed content (that is, when the content moves upwards) on the screen (steps S211 to S215).

In this way, the user input generated in one client device is transmitted to all the client devices through the control in the display control server 50 so that the content is modified and displayed at the same time, thereby enabling the collaborative display of the content.

As described above, the present invention proposes a collaborative display model that is independent of the content cloud 30 and has no limitation on the type of content. In addition, the sensor module (gravity sensor, acceleration sensor, touch sensor, illuminance sensor, etc.) of the client device 10 is used to transmit the physical placement information of the device to the display control server 50, which is based on this information. By calculating content division and distribution information and transmitting display control information to each device, client session management can be performed and continuous collaborative display can be performed regardless of device relocation.

Although the present invention has been described by way of limited embodiments and drawings, the present invention is not limited to the above embodiments. For example, as described with reference to FIG. 4, the invention may be applied to only one client device 10 in an alternative embodiment of the invention. In this case, only one client device is registered in the group for the collaborative display, but the server 50 inputs various information (device device information and device arrangement information) from the device as in the case where a plurality of client devices are registered. Receive and provide display control information based on this, and as a result, the present invention is a technology in which the content to be displayed by the device 10 is controlled by the server 50 regardless of the actual arrangement of the client device 10. Can be implemented.

As such, various modifications and variations are possible to those skilled in the art to which the present invention pertains. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the equivalents of the claims, as well as the claims.

10: client device 11: layout recognition unit
13: user input processor 15: message processor
17: message transceiver 19: renderer
30: content cloud 50: display control server
51: message transceiver 53: session register
55: session manager 57: layout manager
59: message processing unit

Claims (34)

A method of generating mutual placement information with a neighboring second device in a first device on a network, the method comprising:
Generating sensing information by using at least one of an acceleration sensor, a touch sensor, and an illuminance sensor in each device;
Generating device placement information from the sensing information in each device; And
Generating mutual placement information of the first and second devices based on the device placement information of the first and second devices;
The sensing information is information generated by each device when acceleration displacement occurs simultaneously in the first and second devices, or when any one of touch or illuminance change of the screen occurs sequentially. The method of claim 1, wherein the at least one sensor is an acceleration sensor,
The sensing information is characterized in that the acceleration displacement information due to the collision of the first and second devices. The method of claim 1, wherein the at least one sensor is a touch sensor,
The sensing information is touch information by sequential touch on the first and second devices. The method of claim 1,
And the device arrangement information includes information indicating which arrangement of the horizontal arrangement and the vertical arrangement of the first and second devices. In the device on the network,
A layout recognizing unit 11 generating layout information of the device;
A transmission / reception unit (17) which transmits arrangement information of the device to a display control device and receives display control information from the display control device;
A rendering unit (19) for rendering all or part of content to be displayed on the screen of the device on the screen of the device based on the display control information; And
And a message processing unit 15 generating at least one of arrangement information and user input information of the device into a message that can be read by the display control apparatus and parsing a message received from the display control apparatus.
And the arrangement information of the device is generated based on sensing information from one of an acceleration sensor, a touch sensor, and an illuminance sensor of each device. 6. The device of claim 5, wherein the display control information is provided from the display control device to the device for a collaborative display of a group comprising one or more devices. 7. The device of claim 6, further comprising a user input processor (13) for receiving user input for the content and generating user input information. 8. The device of claim 7, wherein the user input comprises at least one of a click, drag and drop, flicking, pinch, and rotation of the content on a screen of the device. delete 6. The device of claim 5, wherein the placement information of the device is obtained from acceleration sensor displacement information generated by colliding the device with a physically neighboring device. The device of claim 5, wherein the arrangement information of the device is obtained from touch information generated when all devices in the group are arranged in a line and each device is touched in turn. The device of claim 6, wherein the display control device is one of devices belonging to the group. A display control apparatus capable of providing display control information to the group for a collaborative display of a group comprising one or more devices in a network, the display control apparatus comprising:
A transceiver 51 for receiving at least one of device device information and device arrangement information from the device and transmitting the display control information to the device;
A session manager (55) for receiving device information of the device from the transceiver and generating and storing session state information for a cooperative display for all devices belonging to the same group;
A layout manager (57) for calculating content division and distribution information indicating a ratio of dividing and distributing content for each device based on device information and device arrangement information of devices for all devices belonging to the group; And
And a message processor 59 for generating the display control information based on the content division and distribution information.
And arrangement information of the device is generated based on sensing information from one of an acceleration sensor, a touch sensor, and an illumination sensor of each device. The device of claim 13, wherein the device information includes information about a device type, a screen size, and a screen resolution, and the device arrangement information includes information about the arrangement of the device and the device and other neighboring devices. Display control device characterized in that it comprises information about the mutual arrangement order. The apparatus of claim 14, wherein the display control information is content deformation information indicating content deformation according to any one of moving, expanding, reducing, and rotating up, down, left, and right of the content displayed on the device. . The apparatus of claim 15, wherein the display control information is generated by referring to the content division and distribution information and the screen size and resolution of each device. The display control apparatus according to claim 13, wherein the display control apparatus is at least one device belonging to the group. A collaborative display method in a group including two or more devices in a network, the method comprising:
Generating, at a first of the two or more devices, one of user input information and device placement information;
Transmitting, by the first device, the generated information to an apparatus (hereinafter, referred to as a "control apparatus") that controls the cooperative display;
Receiving, by the first device, display control information from the control device; And
Based on the display control information, rendering all or part of the content to be displayed in the group on the screen of the first device;
The device placement information is generated based on sensing information from one of an acceleration sensor, a touch sensor, and an illumination sensor of each device,
The first device is a master device and the remaining devices in the group are slave devices,
And the control device does not process the user input information or the device arrangement information received from the slave device while receiving and processing the user input information or the device arrangement information received from the master device. 19. The apparatus of claim 18, wherein each of the user input information and device placement information includes event lock information,
And the event lock information instructs the control apparatus to process the corresponding user input information or device arrangement information only with respect to user input information or device arrangement information including the event lock information received first. Display method. The method of claim 18, wherein the display control information includes content deformation information indicating at least one deformation of moving, expanding, reducing, and rotating up, down, left, and right of the content. 19. The method of claim 18, wherein the apparatus for performing control of the display is one of devices belonging to the group. A method of providing display control information to a collaborative display group comprising one or more devices in a network, the method comprising:
Receiving user input information on content displayed on a screen of the first device from a first device of the one or more devices (S203);
Generating display control information to instruct each device of the at least one device based on the user input information (S207); And
Transmitting the display control information to each of all devices in the group (S209). The method of claim 22, wherein the display control information includes content deformation information indicating at least one deformation of moving, expanding, contracting, and rotating the content. The method of claim 22, wherein prior to generating the display control information,
Receiving device device information from all devices in the group (S111); And
Receiving (S115) device placement information from all devices in the group. The method of claim 24, wherein the device device information comprises information on a device type, a screen size, and a screen resolution. 25. The method of claim 24, wherein the device placement information is obtained from one of an acceleration sensor or a gravity sensor in the device. 25. The method of claim 24, wherein the device placement information is obtained from acceleration sensor displacement information generated by bumping the device against other devices in the group. The method of claim 25,
Calculating segmentation and distribution information of content for all devices in the group based on device device information and device placement information of all devices in the group (S119). 29. The method of claim 28, wherein the display control information is generated with reference to the division and distribution information of the content and the device device information. A method for providing display control information to a group by a display control device (hereinafter referred to as a "control device") for a cooperative display of a group including one or more devices in a network, the method comprising:
Receiving, by the control apparatus, device arrangement information from each of all devices belonging to the group (S303);
Calculating, by the control apparatus, the division and distribution information of the content indicating a ratio of dividing and distributing the contents for each device of all devices in the group based on the received arrangement information (S307);
Generating, by the control apparatus, display control information to instruct each of the devices based on the calculated division and distribution information of the calculated content (S309); And
Transmitting, by the control device, the display control information to each of all devices in the group (S311). The method of claim 30, wherein the display control information includes content deformation information indicating at least one deformation of moving, expanding, contracting, and rotating the content. 31. The method of claim 30, further comprising receiving (S111) device device information from all devices in the group before calculating the division and distribution information of the content. 33. The method of claim 32, wherein the device device information includes information on a device type, a screen size, and a screen resolution. 34. The method of claim 33, wherein the display control information is generated with reference to the division and distribution information of the content and the device device information.

Images