KR100866244B1 - Multi-Party Interactive Collaboration System And Method For Organizing Multi-Party Interactive Collaboration System - Google Patents

Abstract

Disclosed are a multi-party collaboration system and a method of configuring a multi-party collaboration system for a collaboration environment in which a collaboration environment having different system and network environments can be easily constructed and utilized. First, at least one component service divided based on a function of a collaboration service performed in each of a plurality of collaboration nodes is provided, and a connection between at least one component service selected based on a collaboration purpose is established. Then, the component service is configured in the collaboration nodes to which the component services are connected or in the own collaboration node based on the resource situation of each of the plurality of collaboration nodes. Accordingly, the collaboration service can be flexibly selected by selecting the collaboration service according to the capabilities of a given collaboration node. Nodes can be built, and component services can be configured for the purpose of collaboration, and the services can be synthesized to provide complex and new collaboration services.

Description

Multi-Party Interactive Collaboration System And Method For Organizing Multi-Party Interactive Collaboration System

1 is a conceptual diagram illustrating a multilateral collaboration system according to an embodiment of the present invention.

2 is a block diagram illustrating a configuration of a multilateral collaboration system according to an embodiment of the present invention.

3 is a flowchart illustrating a message exchange process between ACE arbiters and a Venue server in a multi-party collaboration system according to an embodiment of the present invention.

4 is a block diagram illustrating a software structure of a multilateral collaboration system according to an embodiment of the present invention.

5A to 5E illustrate service capabilities for component services included in each service category of a component service group according to an embodiment of the present invention.

6 is a flowchart illustrating a service initialization setting process in a multi-party collaboration system according to an embodiment of the present invention.

7 is a flowchart illustrating a service synthesis process in a multi-party collaboration system according to an embodiment of the present invention.

8 is a conceptual diagram illustrating an example of service setting and service synthesis in a multi-party collaboration system according to an embodiment of the present invention.

Explanation of symbols on the main parts of the drawings

400: Component service group 410: Media service category

420: Networking Service Category 430: Display Service Category

440: Multimodal Interaction Service Category

450: Data / task sharing and move service category

500: ACE Arbitration Module 600: Decision Making Module

The present invention relates to a collaboration system, and more particularly, to a method for configuring a multi-party collaboration system and a multi-party collaboration system for an intelligent collaboration environment supporting interaction.

Thanks to high-performance networks, protocols and advanced multimedia technologies, traditional video conferencing concepts are increasingly being extended to an advanced collaboration environment (abbreviated as ACE).

ACE is a remote collaboration environment where you can share common data with the right people at the right time, perform common tasks, solve problems, and discuss common interests. Initial prototypes that are ACE-oriented are now being used for multi-party meetings, multi-party seminars, multi-course lectures, and remote collaboration and teletraining, especially in research institutes and large corporations.

Effective collaboration in the ACE environment requires users to interact and share a wide variety of data and applications, such as streaming audio / video, 2D / 3D graphics data, applications, and documents.

In addition, the data and applications should be able to be displayed on a personal display device as well as a large display device commonly used. And networking should be considered to distribute the real time media streams of remote users. In addition, the user interaction method should be configured to enable intuitive and easy use of such a collaborative environment.

However, the conventional multi-party meeting system has a disadvantage in that it can only share a predetermined data type and application among the collaboration nodes participating in the multi-party meeting system. In addition, the conventional multi-party meeting system has a drawback that each of the collaboration nodes participating in the multi-party meeting can participate in the multi-party meeting only if they use the same system and application.

Accordingly, a first object of the present invention is to provide a method for configuring a multilateral collaboration system that can easily build and utilize a collaboration environment between collaboration nodes having different systems and network environments.

In addition, a second object of the present invention is to provide a multi-party collaboration system that enables easy collaboration between collaboration nodes having different systems and network environments.

A method of configuring a multilateral collaboration system according to an aspect of the present invention for achieving the first object of the present invention described above comprises at least one component service classified based on a function of a collaboration service performed at each of a plurality of collaboration nodes. Providing and establishing a connection between at least one component service selected based on a collaboration purpose, wherein the connection between the selected component services is made between the plurality of collaboration nodes or within its own collaboration node; and Setting the component service to collaboration nodes to which the component services are connected or to the own collaboration node based on a resource situation of each of the plurality of collaboration nodes. Providing the at least one component service classified based on the function of the collaboration service may include the collaboration service as a media service category, a networking service category, a display service category, a multimodal interaction service category, and data and task sharing and mobile service categories. Each category may include at least one component service corresponding to a function of the category. The establishing of the connection between the at least one component service may include transmitting and receiving a service capability of each of the collaboration nodes by participating in the collaboration session. The plurality of collaboration nodes participating in the collaboration session and transmitting and receiving the service capability of each collaboration node may include a step in which the collaboration node wanting to participate in the collaboration request to participate in the collaboration session, and the first collaboration node that requested the participation. Designating as a central coordinator, providing information about the moderator of each collaboration node participating in the collaboration to the collaboration node requesting to join the collaboration, and providing the collaboration node requesting to participate in the collaboration to the central coordinator. Providing the capability, wherein the service capability is for describing attributes of the component service, including role, type, and interface attributes of the service; and other collaborations that the central coordinator participates in collaboration with the collaboration node that provided the service capability. Provides a list of service capabilities that represent the service capabilities of the collaboration node. It may include a step. The establishing of the connection between the component services may include generating context information based on connection information between the component services, generating a service configuration policy based on the generated context information, and generating the service configuration policy. Establishing a connection between the component services based on this. The generating of the service setting policy may include searching for collaboration knowledge in a collaboration database based on the context information, inferring a current collaboration environment based on the searched collaboration knowledge, and corresponding to the inferred collaboration environment. Searching for the service setting policy in a policy database to generate the service setting policy. The setting of the component service may limit service capability within a range that can be negotiated between the collaboration nodes by setting parameters of the selected component services. The method of configuring the multilateral collaboration system may further include synthesizing a service function to perform a complex collaboration task. The synthesizing of the service function may include providing a service synthesis target and a list of services to be synthesized to the service synthesizer, and providing the inference engine with context information including the service synthesis target and service connection information. A reasoning engine for providing a service synthesis policy corresponding to the context information to the service synthesizer and a service interaction method in which the service synthesizer is used for service synthesis and service synthesis of the synthesized service. Synthesizing a service based on including a policy for the service.

A method of configuring a multilateral collaboration system according to another aspect of the present invention for achieving the first object of the present invention described above is a component having at least one component service each executed in the at least one collaboration node and classified according to a service function. Providing a service group, providing a collaboration mediation module that performs service connection and service setting between the component services, and a decision module that automatically provides a policy for the service connection, service setting, and service synthesis And providing the at least one component service based on the policy provided from the decision module by the collaboration mediation module according to the collaboration purpose and the performance of the collaboration node, wherein the at least one component service is between the selected component services. Connection of the plurality of collaboration Making-connecting between nodes or within its own collaboration node.

The collaborative arbitration module includes a service connector for connecting a service based on a collaboration use mode and a list of component services required for the collaboration use mode, and setting parameters of the component services based on a resource situation of a collaboration node. A service synthesizer for performing an optimization task and a service synthesizer for synthesizing the component service to provide an extended service function. The decision making module generalizes collaboration by inferring from the past an inference engine that provides a policy for the service connection, service setting, and service synthesis based on the context information provided from the collaboration mediation module, and contextual contexts for collaboration purposes and collaboration contexts. It may include a knowledge database in which information for the environment is built and a policy database in which policies corresponding to the situation information are accumulated.

The multilateral collaboration system according to an aspect of the present invention for achieving the above-described second object of the present invention sets up at least one component service and is a component corresponding to a collaboration purpose based on the resource situation of each of the plurality of collaboration nodes. The collaboration is based on a plurality of collaboration nodes having a collaboration moderator that searches for a service and configures a service for collaboration, and the collaboration information that is connected through the network with the plurality of collaboration nodes and distributed by the plurality of collaboration nodes. Includes a Venue Server that performs initialization tasks between nodes. The Venue Server may provide information of the Collaboration Moderators participating in the Collaboration Session and designate a Collaboration Moderator initially connected to the Venue Server as a Central Coordinator so that the at least one Collaboration Moderators can communicate through the Central Coordinator. . The plurality of collaboration nodes may include a smart display system in which a plurality of network display devices and a display control device are connected through a local area network (LAN). The plurality of collaboration nodes may include at least one of a smart pointer, a smart hand tracker, and a location tracker for supporting an interactive display.

As the present invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description.

However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes a combination of a plurality of related items or any item of a plurality of related items.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Hereinafter, the same reference numerals are used for the same components in the drawings, and duplicate descriptions of the same components are omitted.

Hereinafter, the multi-party collaboration system represents a system including multi-party meetings, multi-party seminars, multi-party lectures, multi-party remote collaboration, multi-party remote training, and the like.

1 is a conceptual diagram illustrating a multilateral collaboration system according to an embodiment of the present invention.

Referring to FIG. 1, a multi-party collaboration system according to an embodiment of the present invention shares media, data, and applications through networking between Advanced Collaboration Environment (ACE) nodes 100, which are participants participating in a collaboration session.

Each user of the ACE node 100 can share audio, video, 3D graphics, and charts on a networked display device and share documents and work to achieve a common goal in order to achieve a common goal. Can effectively collaborate with other users.

Media services such as audio and video of each ACE node 100 obtain real-time audio and / or video and provide them to other ACE nodes 100 through a network in the form of a media stream. Accordingly, by providing high-quality video to users of all ACE nodes 100 through multicast, the user's appearance and overall background of a user close-up at a specific ACE node 100 can be displayed. ) Can be shown to users.

In addition, the display service of each ACE node 100 receives various image formats such as H.261, digital video (DV), high-definition DV (HDV), and the like, and receives the received image from a projector, an LCD monitor, and / or a grid. Display on display devices such as displays. In particular, the network display capability for grid displays supports high resolution displays by supporting multiple high resolution images and images received from remote ACE nodes 100.

In addition, the pointing service of each ACE node 100 allows users to point to a specific location or move images / images on an interactive grid display. The location tracking service tracks the user's location, and the hand motion tracking service is used to manipulate 3D models (e.g., grab virtual objects and display them on a grid display and share them among ACE nodes). It provides an interactive way to support interactive displays.

2 is a block diagram illustrating a configuration of a multilateral collaboration system according to an embodiment of the present invention.

2, in a multi-party collaboration system according to an embodiment of the present invention, at least one ACE node 100 and a personal ACE node 200 are connected to each other through a P2P overlay network 300. .

Each ACE node 100 includes at least one audio device 101, a video receiving device 103 and a video supply device 105 to obtain and transmit high quality voice and video. It also includes a smart display system 120 to display the acquired video data. The smart display system 120 includes at least one network display device 121, a display control device 123, and a gateway device 125 connected by a high performance local area network (LAN).

In addition, the ACE node 100 includes a smart pointer 131, a smart hand tracker 133, and a location tracker 135 to conveniently and intuitively manipulate the display. can do.

In addition, each ACE node includes a connector 141 for constructing a P2P overlay network 300 to transmit and receive collaborative data with another ACE node 100. Here, the collaborative data is a concept including multimedia data such as audio, video, and text used for collaboration.

In addition, each of the ACE nodes 100 includes an ACE Mediator 150, and each of the ACE Moderators 150 connects with a Venue Server 350 via a P2P overlay network 300. do.

ACE moderator 150 configures the service for a given collaboration session. That is, the ACE arbiter 150 sets up a component service, searches for a component service suitable for a collaboration purpose, and combines the component service into an extended service having a complex function.

Venue server 350 is a centralized virtual space for synchronizing collaboration information distributed by ACE node. Venue server 350 performs an initialization operation to enable message exchange between each ACE arbiter 150.

That is, the Venue Server 350 informs the ACE Moderators 150 currently participating in the collaboration session and designates the ACE Moderator 150 that is first connected as the central coordinator as a central server for communicating with other ACE Moderators 150. do. In addition, the Venue Server 350 designates any ACE Moderator 150 currently participating in the Collaboration Session as the Central Coordinator when the ACE Moderator 150 acting as the Central Coordinator is separated from the collaboration session. .

Personal ACE nodes 200 are abbreviations of ACE node 100 for personal use, and include audio device 101, video receiving device 103, video supply device 105, laptop (or personal computer) 137. ), A connector 141 and an ACE arbiter 150.

Each component included in the private ACE node 200 performs the same function as a component having the same reference numeral included in the ACE node 100 and thus description thereof will be omitted to avoid duplication.

Each device included in the ACE nodes shown in FIG. 2 may be selectively used according to a user's needs and a collaboration environment such as a collaboration purpose, a system, network performance, and the like. In addition, devices included in each ACE node are substantially connected to a 1 to 10 Gbps high-performance local area network (LAN), and the ACE nodes may be connected to a 10 Gbps wide area network (WAN).

3 is a flowchart illustrating a message exchange process between ACE arbiters and a Venue server in a multi-party collaboration system according to an embodiment of the present invention. In FIG. 3, three ACE arbiters exchange messages with a Venue server. For example, the number of ACE arbiters may be three or more.

First, the ACE arbiter 1 100.1 initializes and transmits a join request signal for joining a collaboration session to the venue server 350 (step S101). Venue server 350 receives the join request signal from ACE Moderator 1 100.1, and designates the ACE Moderator 1 100.1, which sent the join request signal first, as the central coordinator (step S103).

Thereafter, the ACE arbiter 2 (100.2) is initialized to transmit a join request signal for joining the collaboration session to the venue server 350 (step S105). Venue server 350 receives a join request signal from ACE moderator 2 (100.2) and delivers information of all ACE moderators (eg, ACE moderator 1) participating in the collaboration session to ACE moderator 2 (100.2) (step). S107).

ACE arbiter 2 (100.2) is based on the information of the ACE arbiter 1 (100.1) from the Venue server 350, the service capabilities (e.g. roles, types, and interfaces) of the component services it manages in the ACE arbiter 1 (100.1) (Step S109).

The ACE Moderator 1 (100.1) serving as the central coordinator provides the ACE Moderator 2 (100.2) with a list of the service capabilities of each ACE Moderator 1 (100.1) (step S111).

Thereafter, the ACE arbiter 3 (100.3) is initialized to transmit a join request signal for joining the collaboration session to the venue server 350 (step S113). Venue server 350 receives a join request signal from ACE moderator 3 (100.3) and transfers information of all ACE moderators (eg, ACE moderator 1, ACE moderator 2) participating in the collaboration session (step S115).

The ACE arbiter 3 (100.3) provides the service capability of the component services managed by the ACE arbiter 1 100.1 based on the information of the ACE arbiter 1 100.1 from the venue server 350 (step S117).

Acting as the central coordinator, ACE Moderator 1 (100.1) provides ACE Moderator 3 (100.3) with a list of service capabilities received from each of the ACE Moderators (eg, ACE Moderator 1 and ACE Moderator 2). S119).

Thereafter, each ACE arbiter performs operations such as service connection, configuration, and synthesis through ACE arbiter 10.1, which serves as a central coordinator.

4 is a block diagram illustrating a software structure of a multi-party collaboration system according to an embodiment of the present invention, and FIGS. 5A to 5E illustrate component services included in each service category of a component service group according to an embodiment of the present invention. Indicates service capability.

4 and 5A to 5E, the software structure of the multilateral collaboration system according to an embodiment of the present invention will be described. The software structure of the multilateral collaboration system includes a component service group 400, an ACE arbitration module 500, and a pseudo doctor. Each ACE node is configured by including a decision module 600 and componentizing each service that performs a unique collaborative element function to select a set of component services according to the capabilities and usage modes of each ACE node. Allow for flexible deployment.

The component service group 400 includes a media service category 410, a networking service category 420, and a display service category 430 classified according to the function of each component service. , A multimodal interaction service category 440 and a data / task sharing and migration service category 450.

Each component service included in each service category 410 through 450 of the component service group 400 has a basic and unique function to assist in the configuration of the ACE node. Component services can access resources such as various devices and software programs (eg, audio or video tools) as shown in FIG. 2 to perform specific functions to add value to advanced collaboration. .

Each component service is developed based on distributed component technology. Therefore, each ACE node is basically allowed to access and control all instances of each component service on the network.

Each component service of the component service group 400 includes information for describing an attribute of each component service called a service capability.

Service capabilities consist of role, type, and interface attributes. Role attributes are classified into producers, consumers, and coordinators according to the provision, acceptance and coordination of service functions for a given component service.

The type attribute represents a functional classification (eg, video, display, networking, interaction, sharing, mobility) for a given component service. The interface property represents a calling method for accessing a component service.

Media service category 410 includes component services of Media Producer Service 411, Media Consumer Service 413, and Media Arbitration Service 415.

The media supply service 411 acquires the foreground and background images of the ACE node and delivers it to other ACE nodes. To this end, the media supply service configures and / or sets various video devices and applications to reflect selection factors of image quality such as image codec, image size and transmission rate intended by the user through management and operation of a media tool. To obtain the image.

In addition, the media supply service 411 delivers the foreground and / or background image of the collaboration space obtained in real time to the media receivers of the ACE nodes participating in the collaboration session in the form of a stream through a predetermined transmission protocol and a multicast address. Here, the transport protocol and the multicast address may be set through the media coordination service 415 or the user's manual configuration.

The media receiving service 413 receives a collaborative video from other ACE nodes and displays the video.

To this end, the media receptive service 413 receives the image provided by the media provider through the mutually agreed transport protocol and multicast address through the media coordination service 415 or the user's manual setup.

The media receiving service 413 decodes the received image using an image codec and / or an application and converts the received image into RGB (Red, Green, Blue) data. The decoded video is played on a local device that is not shared or provided to a network supported display device.

The media coordination service 415 plays a role of trouble shooting for improving the quality of experience (QoE) of the real-time media service.

To this end, media coordination service 415 allocates a multicast address for video delivery to the media provider of its own ACE node and then adjusts media provider's properties (e.g., video codec and application) to media of other ACE nodes. Provide service. In addition, the user of the media receiver selects a desired media provider and connects with the media receiver to induce the media receiver to receive a selective image.

In addition, the media coordination service 415 identifies the service capability difference between the media provider and the media receiver in case of abnormal video delivery due to the service capability difference between the media provider and the media receiver, and then provides appropriate measures to overcome the problem. Allow for continuous media delivery.

For example, the media coordination service 415 overcomes the service capability difference by providing an additional service such as a transcoder when there is a service capability difference between the media provider and the media audience.

Media coordination service 415 then analyzes end-to-end media quality dependent on network and system performance to cope with variable network and system performance, and adapts appropriately such as application level image quality control parameters based on the analyzed results. Providing media delivery policies to media providers and media audiences enables them to deliver adaptive media suitable for network and system performance.

The networking service category 420 is a component service of the Multicast Network Service (421), the NAT / Firewall Traversal Service (423), and the Network Monitoring Service (425). It includes.

The multicast connection service 421 provides multicast connectivity between ACE nodes that do not have multicast connectivity.

That is, the multicast connection service 421 is a multicast tunneling protocol and a multicast in the application layer in the process of communication between ACE nodes based on multicast networking existing on a partially multicast network. It provides transparent and efficient multicast connectivity without modification of media transmission program or complicated network setting through loop resolution technique and / or server information supporting partial multicast.

Network Address Translation (NAT) / Firewall Crossing Service 423 provides multicast connectivity between ACE nodes connected to a NAT and / or firewall-based private network.

That is, NAT / firewall traversal service 423 provides transparent multicast connectivity to ACE nodes connected to a private network without changing NAT and / or firewall devices or changing the settings of NAT and / or firewall devices by an administrator.

To this end, the NAT / Firewall Crossing Service 423 precedes the connection request from the internal network to the external network when establishing a network connection by taking advantage of the fact that the connection request from the internal network to the external network must be preceded when passing through the NAT and the firewall device. At this time, the connection information is requested from the external network to the internal network using the set information to provide interactive multicast connectivity.

The network monitor service 425 monitors multicast connectivity and networking performance between ACE nodes communicating on a multicast basis.

That is, the network monitor service 425 periodically transmits a probe packet indicating its presence to a multicast address shared by each ACE node through whether network monitoring is executed and network monitoring server information setting. Based on the information of the ACE node, packet serial number and / or time information provided by the probe packet, the multicast performance between each ACE node is identified.

Multicast performance between each ACE node is collected on a central server and provides a comprehensive view of multicast connectivity and networking performance information for each ACE node in which the central server is participating in a collaboration session.The information provided is a key indicator for multicast performance analysis. Is utilized.

The network monitor service 425 analyzes the multicast performance by using the multicast connectivity and networking capability information for each ACE node provided from the central server.

The display service category 430 includes component services of an interactive display control service 431, a visualization service 433, and a networked display service 435.

The interactive display control service 431 receives a display control command from a user or interactive services and provides a control service for integrally operating networked displays. Here, the network display refers to a display device capable of receiving and displaying display and display control data through a display device or a network interface device on which a network interface device is installed.

To this end, the interactive display control service 431 provides a messaging method for a user to generate a network display control command through a Graphic User Interface (GUI) or an interactive service. Here, the network display control command includes a control command for starting, ending, moving, and resizing an image, for example.

In addition, the interactive display control service 431 decodes and / or renders the visualization data through a media accommodation service and / or a visualization service to display visualization data selected by a user, that is, an image, an image, graphics, and the like, on a network display. It plays a role of relay that connects with network display service.

The visualization service 433 performs a function of converting image / graphics files of various formats into an RGB form.

To this end, the visualization service 433 performs a rendering function of converting an image compressed in various formats into RGB data using an image library. In addition, the visualization service 433 sets the transmission protocol and the IP (Internet Protocol) address of each network display device, and divides the image / graphics data according to the corresponding area in consideration of the resolution and location of each network display device. By using the Transmission Control Protocol (TCP) or the User Datagram Protocol (UDP), the divided data and the corresponding data size, output location, and ratio information are integrated and transmitted to the network display device.

The network display service 435 receives the image / graphics data in the form of RGB and expresses the image and graphics data in the positions and sizes designated by the respective network display devices.

To this end, the network display service 435 provides a synchronization scheme for network display devices having different resolutions, shapes, and positions to be organically operated like one display device, manages status information of each network display device, Each network display device is integrated and controlled to express RGB data according to the intention.

In addition, the network display service 435 expresses the RGB data to the network display device by using the RGB data received through the network and various display information (for example, display size and position) on the OpenGL-based display.

The multimodal interaction service category 440 includes component services of a pointing service 441, a hand motion tracking service 443, and a location tracking service 445. .

The pointing service 441 performs a function of using a laser pointing device to point or gesture at a specific location within an ACE node. To this end, the pointing service 441 acquires a pointing image using a laser device for pointing and a camera filter for distinguishing a laser point, and a camera for tracking a pointing position, and converts the obtained pointing image into a computer vision technology (eg, For example, the pointing position is tracked by calculating the correct pointing coordinates using the calibration between the cameras and the camera lens undistortion.

In addition, the pointing service 441 analyzes the pointing location information tracked in real time and recognizes a gesture intended by the user.

The hand motion tracking service 443 supports the control and sharing of 3D computer aided design (CAD) models using tracking of hand gestures.

That is, the hand motion tracking service 443 collects the joint data of the hand in real time using a device capable of tracking the hand motion to configure the kinematic appearance of the hand. In addition, the position of the 3D CAD model is controlled by observing the 3D CAD model on a display in real time and interpreting hand gestures to track the current position and the state of the hand to interact with the 3D CAD model.

Location tracking service 445 provides the ability to track the location of a particular target within an ACE node. That is, the location tracking service 445 estimates the 3D location coordinates inside the ACE node for the target that needs location tracking by using the location sensor based user location calculation technology for dynamic user context information recognition and analysis. In addition, in order to improve the accuracy of the position estimation, the conventional RF (Radio Frequency) based position recognition method uses an additional sensing technique such as ultrasonic (ultrasonic) to provide the accuracy of the estimation error within the position 0.5m.

The data / task sharing and movement service category 450 includes data sharing services 451, data adaptation services 453, and remote user interface services 455. Contains component services.

The data sharing service 451 supports distributed shared data storage for sharing data held by individual ACE nodes globally among all ACE nodes participating in a collaboration session.

That is, the data sharing service 451 provides a distributed shared data store that can store any data file that other users who have participated in the collaboration session can access, and upload, download, and share the shared data. Set retrieve information. Shared data repositories, along with shared data files, provide knowledge about file ownership, file access rights, and controls.

The data adaptation service 453 provides a processing technique for delivering adaptive shared data and / or application views in consideration of the performance of each ACE node. That is, the data adaptation service 453 identifies the unique characteristics of each ACE node such as network, system performance, and / or user preference for each ACE node, and then processes the view of the shared data to be accepted by the node (e.g., For example, transcoding is provided to the user.

The remote user interface service 455 supports a user interface for sharing and controlling data and / or applications of remote ACE nodes. That is, the remote user interface service 455 obtains a screen in which an application operates in real time, delivers it to a remote user, and transmits a control right of an input device such as a keyboard and a mouse together to share the application in real time and operate the remote. Provides interface functionality.

The ACE arbitration module 500 includes a service connector 510, a service setter 520, a service synthesizer 530, and a context database 540.

The ACE arbitration module 500 first connects to the Venue Server and requests to join the collaboration session in order to join the collaboration session. In addition, information of the currently participating ACE arbiters from the Venue server and the information of the central server designated as the moderator by the Venue server are returned.

In addition, the ACE arbitration module 500 provides the central coordinator designated as the central server with the service capability (eg, role, type, interface) of the component services managed by the ACE node to which it belongs.

After the service capability exchange is performed between the ACE arbitration modules 500, the ACE arbitration modules 500 of each ACE node perform tasks such as service connection, service configuration, and service synthesis through a central coordinator.

The service matchmaker 510 performs a connection work between component services using a usage mode suggested by a user and a list of component services required for this purpose. In this case, the usage mode may be content of collaboration performed in a multilateral collaboration system. For example, the mode of use may be teleconference, telepresentation, remote lecture, or the like.

In addition, the component service connection operation may be performed between a plurality of collaboration nodes that are functionally dependent on a given component service or between component services to be suitable for a usage mode by connecting different kinds of component services within their own collaboration nodes. It is to set the order of processing and to set logical transmission path for message transmission between them.

The service connector 510 utilizes the service capabilities (eg, roles, types, and interfaces) of each component service to create a series of component services in which dependencies are established.

The function of other component services that the current component service needs to connect to can be determined by type, and the role of providing or accepting service can be determined by role.

For example, a component service whose role is a provider can provide functional deliverables to a component service whose role is an acceptor. A component service whose role is a coordinator does not provide a functional output, but rather mediates between multiple component services for a given purpose. Play a role.

When a series of component services are generated by the service connector 510, messages can be exchanged through the connected component services.

The service configurator 520 optimizes the component service in consideration of the resource situation of each ACE node. Resource context here includes all the resources needed for collaboration, such as the type of device configured on each ACE node, the performance of the device, the performance of the software supporting each device, the network connecting each device, and the network connecting each ACE node. do.

In order to provide an efficient collaboration environment, it is necessary to select a component service for configuring each ACE node according to the performance and usage mode of each ACE node and set parameters of the selected component services according to the situation.

To this end, the service setter 520 performs a function to assist in flexibly performing a service function by limiting service capability within a negotiable range. As an example of a service parameter, the media supply services can be upper bound of the video frame rate, the media receptive service can be the number of available computers for decoding, and the pointing service is pointing. Can be accuracy.

The service setter sets the component service to the collaboration nodes to which the component services are connected or to the own collaboration node based on the information manually set by the user or the service policy setting information provided by the decision module 610.

The service compositor 530 provides a method of extending a service function for performing a complex task. In other words, the service synthesizer satisfies the various requirements of the user by synthesizing the service to provide new and expanded functions.

For example, the service synthesizer 530 transmits the situation information composed of the target of the service synthesis and the service connection information to the decision module 600, and performs the service synthesis based on the information provided from the decision module 600. do.

The context database 540 is context information collected from each component service in a collaboration session, and is used as data for identifying the intention of the current user.

The decision module 600 includes an inference engine 610, a knowledge database 620, and a policy database 630.

The inference engine 610 infers context information from the knowledge database 620 and obtains a corresponding policy from the policy database 630 and provides the ACE arbitration module 500 with instructions for configuring an ACE node.

For example, the inference engine 610 receives the situation information from the ACE arbitration module 500 and searches the knowledge database 620 for collaboration knowledge accumulated from the past corresponding to the received situation information. After inferring the current situation based on the retrieved knowledge, a policy corresponding to the inferred situation is searched in the policy database 630 to obtain an appropriate service setting policy. The inference engine 610 provides the obtained service setting policy to the ACE arbitration module 500.

The knowledge database 620 builds information for a generalized collaboration environment by accumulating collaboration objectives and contextual contexts from the past. The knowledge database 620 is used to predict key contextual information for each collaboration mode.

The policy database 630 accumulates a policy corresponding to the context information built in the knowledge database 620. The policy database 630 is utilized to provide the ACE arbitration module with the policies necessary to configure the ACE node suitable for a collaborative environment.

6 is a flowchart illustrating a service initialization setting process in a multi-party collaboration system according to an embodiment of the present invention.

Referring to FIG. 6, first, component services 410 to 450 belonging to a predetermined ACE node register their service capability with the service setter 520 of the ACE arbiter in charge of itself (step S201).

Then, the user transmits the usage mode and a list of services required for this to the service setter 520 of the ACE node to which the user belongs (step S203).

Thereafter, the service setter 520 requests the service connector 510 to connect with another ACE node (step S205), and the service connector 510 connects to the service connector 510 belonging to another ACE node. After performing the operation, connection information between services is generated and provided to the service setter 520 (step S207).

The service setter 520 provides the inference engine 610 with the situation information including the service connection information provided from the service connector 510, the resource state for each ACE node, and the usage mode (step S209).

The inference engine 610 searches the knowledge database 620 for generalized collaboration knowledge accumulated from the past corresponding to the situation based on the situation information provided from the service setter 520 (step S211).

The inference engine 610 infers the current collaboration environment based on the searched knowledge (step S213), and then obtains the service setting policy by searching the policy database 630 for a policy corresponding to the inferred collaboration environment (step S213). S215).

The inference engine 610 provides the obtained service setting policy to the service setter 520 (step S217), and the service setter 520 manages the component service 410 to which it manages based on the received service setting policy. 450 are set (step S219).

When the configuration of the component service is completed, the ACE arbiters forward the service start request to the activated component services. Each component service received the service start request performs a given task in consideration of the connection order and interface information between the component services.

7 is a flowchart illustrating a service synthesis process in a multi-party collaboration system according to an embodiment of the present invention.

Referring to FIG. 7, the user first provides the service synthesizer 530 with a target of service synthesis and a list of services to be synthesized (step S301). The service synthesizer 530 requests a service connection task from the service connector 510 (step S303), and the service connector 510 provides service connection information to the service synthesizer 530 (step S305).

Thereafter, the service synthesizer 530 provides the inference engine 610 with the context information composed of the service synthesis target and the service connection information (step S307), and the inference engine 610 is provided with the context information provided from the service synthesizer 530. After searching the knowledge database 620 based on (S309), and inferring the situation of the collaborative environment through the retrieved knowledge (S311), the policy database 630 applies a service synthesis policy corresponding to the inferred situation of the collaborative environment. Search in step S313.

The inference engine 610 then provides the searched service synthesis policy to the service synthesizer 530 (step S315). The service synthesis policy provided by the inference engine 610 here includes a policy on the capabilities of the synthesized service (eg, role, type, interface) and how to interact between the services used for service synthesis.

The service synthesizer 530 performs service synthesis based on the service synthesis policy provided from the inference engine 610 (step S317), and the synthesized service is registered in the service setter 520 (step S319).

8 is a conceptual diagram illustrating an example of service setting and service synthesis in a multi-party collaboration system according to an embodiment of the present invention.

Referring to FIG. 8, the hand motion tracking service 443 of the ACE node 2 transmits the 3D graphics information manipulated by the hand gesture to the network display service 435 belonging to the ACE node 1, the ACE node 2, and the ACE node 3, respectively. To provide. To this end, the network display services 435 of each ACE node are provided with information necessary for service setting and various parameter values necessary for service acceptance through the service setter 520 belonging to each ACE node. Therefore, collaboration can be performed between ACE nodes having different environments while maintaining a constant quality of service.

In addition, the user of the ACE node 1 generates a smart pointing service 447 by combining the location tracking service 441 and the pointing service 445, and the generated smart pointing service 447 is the ACE node 1 and ACE node 2 Network display service 435 is provided. When the user of the ACE node 1 manipulates the pointing device to which the location service 445 is added using the smart pointing service 447, the manipulated result is displayed on the grid display devices of the ACE node 1 and the ACE node 2. Can be.

According to the method of configuring a multi-party collaboration system and a multi-party collaboration system for a collaboration environment as described above, in order to flexibly configure the ACE node in consideration of the performance of the ACE node, the collaboration services are componentized to collaborate according to the capabilities of a given ACE node. Select a service.

In other words, the collaboration services are classified into five service categories: media, networking, display, multimodal interaction, and data / task sharing and movement according to their functions, and each service category includes corresponding component services such as computer / networking. ACE nodes can be flexibly constructed by selecting the appropriate component services and building ACE nodes according to the environment and the purpose of collaboration.

In addition, the ACE moderator is used to set component services for collaboration purposes and to synthesize the services to provide complex and new collaboration services.

Therefore, it is possible to easily build and utilize a collaboration environment between collaboration nodes having different system and network environments.

Although described with reference to the above embodiments, those skilled in the art will understand that various modifications and changes can be made without departing from the spirit and scope of the invention as set forth in the claims below. Could be.

Claims (17)

In the method of configuring a multi-party collaboration system consisting of a plurality of collaboration nodes, Providing at least one component service classified based on a function of a collaboration service performed at each of the plurality of collaboration nodes; Establishing a connection between component services based on at least one of the provided usage mode and the service list; Generating context information based on a connection establishment between the component services; Generating a service setting policy based on the generated situation information; And Setting the component services based on the created service setting policy. The method of claim 1, wherein providing at least one component service divided based on a function of the collaboration service comprises: The collaboration service is divided into a media service category, a networking service category, a display service category, a multimodal interaction service category, and a data and task sharing and mobile service category, each of at least one component corresponding to a function of the category. How to configure a multi-party collaboration system comprising a service. delete The method of claim 1, wherein establishing a connection between component services based on at least one of the provided usage mode and service list comprises: Making a collaboration node requesting to participate in a collaboration request to join a collaboration session; Designating, as a central coordinator, a collaboration node that has made a request to join first; Providing information about a moderator of each collaboration node participating in the collaboration to the collaboration node requesting to participate in the collaboration; Providing, by the collaboration node requesting to join the collaboration, to the central coordinator a service capability, the service capability being for describing an attribute of the component service, including a role, type, and interface attribute of a service; And And providing, by the central coordinator, to a collaboration node providing the service capability, a service capability list indicating service capability of another collaboration node participating in the collaboration. delete The method of claim 1, wherein the generating of the service setting policy is Retrieving collaboration knowledge from a collaboration database based on the contextual information; Inferring a current collaboration environment based on the retrieved collaboration knowledge; And And searching the service setting policy corresponding to the inferred collaboration environment in a policy database to generate the service setting policy. The method of claim 1, wherein the setting of the component services based on the generated service setting policy is performed. And setting service parameters of the selected component services to limit service capability within a negotiable range between the collaboration nodes. The method of claim 1, wherein the multi-party collaboration system is configured. Providing a service synthesizer with a service synthesis target and a list of services to be synthesized; Providing, by the service synthesizer, situation information including a service synthesis target and service connection information to an inference engine; Providing, by the inference engine, a service synthesis policy corresponding to the context information to the service synthesizer; And And the service synthesizer synthesizing a service based on the service synthesis policy, the service capability of the synthesized service and a policy on how to interact with services used for service synthesis. How to configure a collaboration system. delete In the method of configuring a multilateral collaboration system including at least one collaboration node, Providing a component service group having at least one component service each executed in the at least one collaboration node and classified according to a service function; The collaboration mediation module performs connection of services based on the collaboration usage mode and the list of component services required for the collaboration usage mode, and performs the optimization of services by setting parameters of the component services based on the resource situation of the collaboration node. Providing a; And Providing a decision module that automatically provides a policy for the service connection, service settings and service synthesis, The collaboration arbitration module selects the at least one component service based on the policy provided from the decision module, in accordance with the collaboration purpose and the capabilities of the collaboration node, wherein the connection between the selected component services is determined by the plurality of collaboration nodes. A method of constructing a multi-party collaboration system comprising the steps of—connecting between them or within their own collaboration node. The method of claim 10, wherein the component service group The collaboration service is divided into a media service category, a networking service category, a display service category, a multimodal interaction service category, and a data and task sharing and mobile service category, each of at least one component corresponding to a function of the category. How to configure a multi-party collaboration system comprising a service The method of claim 10, wherein the collaborative intervention module A service connector for connecting a service based on a collaboration usage mode and a list of component services required for the collaboration usage mode; A service setter configured to optimize a service by setting parameters of the component services based on a resource situation of a collaboration node; And And a service synthesizer for synthesizing the component services to provide an extended service function. The method of claim 10, wherein the decision module An inference engine that provides a policy for service connection, service setting, and service synthesis based on context information provided from the collaboration arbitration module; A knowledge database in which information for a generalized collaboration environment is constructed by accumulating collaboration objectives and contextual contexts from the past; And And a policy database in which a policy corresponding to the contextual information is accumulated. In a multi-party collaboration system composed of a plurality of collaboration nodes, A plurality of collaboration nodes having at least one component service and a collaboration arbiter configured to configure a service for collaboration by searching for a component service corresponding to a collaboration purpose based on a resource situation of each of the plurality of collaboration nodes; And The at least one collaboration is performed through the central coordinator by connecting the plurality of collaboration nodes through a network, providing information of the collaborator moderators participating in the collaboration session, and designating a coordinator moderator elected by a predetermined method as a central coordinator. And a Venue server for performing initialization operations between the collaboration nodes so that moderators can communicate. delete 15. The method of claim 14, wherein the plurality of collaboration nodes And a plurality of network display devices and a display control device comprising a smart display system connected via a local area network (LAN). delete

Images