KR101077660B1 - Multi-Party Interactive Collaboration System And Method for organizing thereof, and the recording media storing the program performing the said method - Google Patents

Abstract

The present invention proposes an agent-based service synthesis model based on a service-oriented architecture as a multi-party collaboration system, and provides a multi-party collaboration system and a method of configuring the same, and methods for accessing data and interfaces of existing collaboration software through a network. It is an object of the present invention to provide a recording medium on which a program to be implemented is recorded.

To this end, the method of configuring a multi-party collaboration system according to an example of the present invention targets a user through a mediator module controlled by the user to use basic collaboration devices in each of the plurality of collaboration nodes. A task step of providing task data for performing a task, and at least one manager module controlled by the task data provides the service command data according to a preset method for processing the task data. And a service step of providing at least one service module controlled by the service command data to provide service data for executing the collaboration devices; And a system setting step of the mediator module finding / setting / synthesizing the collaboration devices and the service module available in the collaboration node.

 Multiparty Collaboration System, Mediator, Manager, Service, Service Synthesis

Description

Multi-Party Interactive Collaboration System And Method For Organizing One, And The Recording Media Storing The Program Performing The Said Method}

The present invention relates to a multi-party collaboration system and a method of configuring the same. More specifically, the present invention provides a service module, which is a component-based software, to enable a network access to data and interfaces of an existing collaboration module. Register as a manageable component in the environment, and through service synthesis algorithms, users can find and discover the service modules available in the collaborative environment with easier operation, and the interaction of the service modules through the connection / synthesis of the service modules The present invention relates to a possible multilateral collaboration system and a method of configuring the same.

High performance computing and networking technologies have led to the proliferation of advanced collaborative systems around businesses and research institutions. In particular, it is widely used in various applications such as business meetings using commercial systems, employee training, scientific joint experiments using research-oriented collaboration systems, and remote lectures. This multi-party collaboration system uses multi-display devices and high resolution cameras to provide a realistic teleconferencing environment among knowledge workers in remote locations.

In addition, such a collaboration system forms a collaboration environment that can provide voice / video / data sharing services among multiple collaboration spaces connected by multicast, and provides additional services for users to build an extended collaboration environment for their purpose. It provides a framework for development.

This advanced collaboration system not only allows sharing of high-quality video, audio, and graphics, but also provides an interactive, high-resolution grid display using multiple pointing and location tracking devices to create a more realistic grouped collaborative data environment. Providing.

However, since this type of collaboration environment is composed of numerous hardware and software, it is not only difficult for a user unfamiliar with the operation of the collaboration system to directly manipulate all components, but also has an inefficient disadvantage.

In addition, abstraction technology, which facilitates the management of components in a collaborative environment, makes each component accessible to its own functionality through Active Space API (application programming interfaces), and then is widely used in object-oriented programming (MVC). An application framework that extends the model (view, controller) to implement a software architecture that allows the user to program interactions between components, or define interactions between service agents as workflows for a given conference scenario. Thus, service agents are organically cooperated to provide a service in a set manner. Alternatively, INDIVA provides an abstraction for hierarchically managing the distributed multimedia hardware and software that make up an Access Grid node like a file system, so that all resources can be discovered, linked, and manipulated through high-level instructions. There is a way to implement this through middleware.

However, in order to use these methods, the user must predetermine a procedure for selecting a service to be used from among a large number of components, connecting the services, and determining the order of interaction of the services. Due to these limitations, it is difficult to dynamically respond to variable collaboration contexts such as heterogeneous performance characteristics of participating nodes, available resource states that change over time, and different user intentions.

An object of the present invention is to propose an agent-based service synthesis model based on a service-oriented architecture as a multi-party collaboration system, and to provide a multi-party collaboration system and a method of constructing the data and an interface of an existing collaboration software. It is done.

In addition, the present invention proposes a service synthesis algorithm for providing a complex collaboration application service, thereby discovering / connecting available service modules that meet the user's requirements with a simple operation of the user, and the user in the An object of the present invention is to provide a multi-party collaboration system and a method of configuring the same that can more easily access / use services.

A method of configuring a multi-party collaboration system according to an example of the present invention is a task that a user targets through a mediator module controlled by a user to use basic collaboration devices at each of a plurality of collaboration nodes. A task command step of providing task data for performing task; and a service command step of providing service command data according to a preset method for processing at least one manager module controlled by the task data; and A service step of providing at least one service module controlled by the service command data to provide service data for executing the collaboration devices; And a system setting step of the mediator module finding / setting / synthesizing the collaboration devices and the service module available in the collaboration node.

 Here, the system setting step includes a service discovery step of searching for a service module that can interface with the cooperative devices and the manager module and storing the state data of the service module in a collaboration situation storage unit; and executing / ending the searched service module. Service setting step of setting the manager module and the service module to enable; And a service synthesis step of synthesizing the manager modules through the mutual interface binding between the manager modules to process the task data.

Here, the system setting step may further include a node status display step of displaying, on a node graphical user interface (GUI), a service module available in the collaboration devices and the collaboration devices found in the service discovery step and an administrator module controlling the service module. can do.

In addition, the service setting step includes setting the manager module and the service module to enable the mediator module to execute / shut down the service module when the collaboration devices displayed in the node status display step and the service module available in the collaboration device are selected by the user. It may include a step.

The service discovery step includes searching for a device agent module mounted in the collaboration devices to search for the collaboration devices, and displaying the collaboration device in the node GUI through the found device agent module. The setting step may include adding / deleting a service module or an administrator module available through the collaboration device in which the mediator module is found according to a user's selection.

The system setting step may further include displaying task data available through the manager module and the service module set in the service setting step on a space graphical user interface (GUI).

Also, the service synthesis step may include: automatically selecting / synthesizing at least one manager module for processing the task data when any one of the displayed task data is selected by the user; And searching / setting at least one service module to be synthesized for processing the task data by the selected / synthesized manager module.

In addition, the manager module is divided into a media service category, a networking service category, a display service category, a multimodal interaction service category, and a data and task sharing and moving service category. The service command step includes a service in which each manager module corresponds to each category. And transmitting the command data to the at least one service module.

The service step may include transmitting service data to a service agent module such that the collaboration application module directly controlling the collaboration devices by the at least one service module is controlled by the service data; and the service agent module is preset according to the service data. Sending an API to a collaborative application module; And directly controlling the execution / termination of the collaboration devices by the collaboration application module.

In addition, the method of configuring a multi-party collaboration system is a mediator module of a first node to newly participate in a plurality of collaboration nodes using a contract network-based technology, which is an interaction protocol, and a mediator of a second node already participating in a plurality of collaborations. The module may further include a network synthesis step of combining the manager modules of the first node and the second node with each other.

Here, in the network synthesis step, the mediator module of the first node searches for a manager module in the first node, and sends a notification message to the mediator module of the second node to find the manager module of the second node to be synthesized with the searched manager module. Work notification step; and if there is a manager module in the second node to be synthesized with the founder module of the second node, a suggestion message indicating that the second node may be the target of connection to the mediator module of the first node. Proposal step of transmitting; and the mediator of the first node selects a second node that can be synthesized from the proposal message, and transmits the contract message for the synthesis of the manager module and the agent ID of the manager module of the first node to the mediator module of the second node. Contract step to do; And a result reporting step of, by the mediator module of the second node, adding / updating the manager module of the first node using the agent ID of the manager module of the first node, and transmitting an update fact to the existing manager module in the second node. It may include.

In addition, the network synthesis step may further include, after the contracting step, transmitting, by the mediator module of the second node, the agent ID of the manager module that is already participating in the collaboration to the mediator module of the first node.

In addition, the multi-party collaboration system according to an exemplary embodiment of the present invention provides a mediator for providing a task data for performing a target task by using a basic collaboration devices in each of a plurality of collaboration nodes. And at least one manager module controlled by the task data and providing the service command data in a preset manner for processing the task data; And at least one search service module controlled by the service command data and providing service data for executing the collaboration devices, wherein the mediator module includes at least one service module and the collaboration devices available in the collaboration node. Discover / Set / Synthesize.

Here, the mediator module searches for the collaboration devices available in the collaboration node and the service modules that can interface with the manager module in each device, and stores the data in the collaboration situation storage that stores the state data of the service module. A service setting / executing unit configured to set an administrator module and a service module to execute and terminate the module and the service module, and to transmit task data to the manager module; And a service synthesis unit for synthesizing the manager module through mutual interface binding between the manager modules to process the task data.

In addition, the mediator module may include a node GUI (Graphical User Interface) unit for displaying the collaboration devices retrieved by the service discovery unit and the service module available in the collaboration devices and an administrator module controlling the service module.

In addition, the service setting / executing unit sets the manager module and the service module to enable / disable the service module when the collaboration devices displayed in the node GUI unit and the service module available in the collaboration device are selected by the user. Task data can be sent to the manager module for execution / exit.

In addition, the service discovery unit searches for a device agent module (Device agent) mounted in the collaboration devices to search for collaboration devices, and displays the collaboration device in the node GUI through the discovered device agent module, and the service configuration / execution unit Optionally, you can add / delete an available service module or administrator module for the retrieved collaboration device.

In addition, the mediator module may further include a space GUI (Graphical User Interface) unit for displaying task data available through the manager module and the service module set in the service setting unit.

In addition, the service synthesis unit may automatically select / synthesize at least one manager module for processing the task data when any one of the task data displayed in the space GUI is selected by the user.

Here, the manager module automatically selected / synthesized retrieves / sets at least one service module to be synthesized to process the task data, and in a preset manner according to the task data transmitted from the mediator module (service setting / executing unit). The service command data can be transmitted to the service module that is searched / set.

In addition, the manager module is divided into a media service category, a networking service category, a display service category, a multi-modal interaction service category, and a data and task sharing and mobile service category. Each of the manager modules includes at least service command data corresponding to each category. It can transmit to one service module.

Here, each of the at least one service module may include a service agent module receiving the service data so that the collaboration application module directly controlling the collaboration devices is controlled by the service data.

In addition, the multi-party collaboration system includes a mediator module of a first node that intends to newly participate in a plurality of collaboration nodes and a second mediator module that participates in a plurality of collaborations, wherein the mediator module of the first and second nodes includes an interaction protocol. Can be synthesized with each other using a technology based on a contract network.

In addition, the task data includes component data for setting the manager module in the service setting / executing unit and composite data for synthesizing the manager module in the service synthesizing unit. The component data includes the name, type, and service command data of the manager module. And the synthesis data may include service synthesis data necessary for synthesis of the manager module.

In addition, the service data may include specification data of the name, role, type, maximum instance number or setting of the service agent module for the service module to control the service agent module.

In addition, the computer-readable recording medium according to an example of the present invention includes a recording medium on which a program for implementing the above-described method for configuring a multilateral collaboration system is recorded.

The present invention proposes an agent-based service synthesis model based on a service-oriented architecture, and provides a multi-party collaboration system and a method of configuring the same, which can access data and interfaces of existing collaboration software through a network, thereby making it easier for users. Service modules can be interconnected to share the information of the entire collaboration space in a formal format, so that each service module can share the state information of the service modules of interest in the collaboration environment, and the interaction between the service modules according to the collaboration context. There is an effect that makes this possible.

In addition, the multi-party collaboration system and its configuration method according to the present invention discovers / connects available service modules that meet the user's requirements with a simple operation of the user, and makes it easier for the user to services of other nodes in the multi-party collaboration environment. There is an effect that can be accessed / used.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

First, in adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible, even if shown on different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. In addition, the following will describe a preferred embodiment of the present invention, but the technical idea of the present invention is not limited thereto and may be variously modified and modified by those skilled in the art.

1 is a table for explaining a conceptual model of a multi-party collaboration system and a configuration method according to an embodiment of the present invention.

As shown in FIG. 1, the conceptual model of the multilateral collaboration system according to an example of the present invention is composed of a task data layer, a service manager layer, and a service layer at an abstraction level, and an interaction model, an environment model, It is divided into expression model, application model and data model.

In functional division, the interaction model determines how to change the state of the data model in conjunction with the application model. The environment model describes information about the various factors that affect the performance of the function in the collaborative environment, and the presentation model. Represents the elements for the user to show the state of the data model, the application model defines the functions necessary for the operation of the data model, and the data model refers to the kind of data that the user can create or interact with. According to this functional aspect, the categories of manager modules to be described later are divided, and the functions of each service module corresponding to the categories of each manager module are distinguished.

In terms of the level of abstraction, the task layer describes the characteristics of the complex collaborative application services desired by the mediator module, and provides task data to the relevant service manager modules to perform the tasks that the user wants to target. By transmitting, the responsibility for the configuration and operation of the service module is assigned. In the service manager layer, based on the delivered task data content, the manager module sends the service command data to the service module based on the prepared service synthesis logic, finds the required service modules, connects them in the order of execution, and then integrates them. Run In the service layer, the service module provides service data for executing the collaboration devices in order to perform a target task.

The task layer is an abstraction for sharing application services (eg, video conferencing, collaborative task data, visualized content sharing, etc.) required for collaboration in a multi-party collaboration environment, and ensuring that the content to be shared is handled in a user-friendly way. To provide. Here, the content to be shared is defined as a shared object. This task layer provides task data for the user to perform a task targeted by a mediator module controlled by the user in order to use the basic collaboration devices in each of the plurality of collaboration nodes. Implemented with the task steps provided.

The interaction model defines interaction interfaces for manipulations specific to shared objects. For example, consider a visual sharing GUI that places the presenter's material in a desired location. The environment model describes the information of the manager modules for processing the shared objects associated with the task data, and the presentation model indicates the progress of the given task data and the state of the shared objects associated with the task data. In other words, it represents the state of all contents associated with a given task data.

The application model also provides a method for manipulating (eg, changing the view of the presenter) the shared objects associated with the task data, the data model describing the data for the shared objects associated with the task data. For example, assuming that the content related to the presenter is a shared object, the shared object includes the point of view of the presenter, the background information included in the presenter's appearance, the presenter's voice, the presentation material, and the interaction with virtual objects using hand gestures. It can be expressed as a data model of.

The service manager layer can then automatically select relevant service modules and order appropriate actions based on the understanding of the shared object's processing requirements and the context of the collaborative environment in order to process the shared objects required by a given task data. That requires a high level of management. In this way, the high-level manager for processing a given shared object is defined as an administrator module. Such a service manager layer is implemented as a service command step in which at least one manager module controlled by the task data provides the service command data in a preset manner for processing the task data.

In the service manager hierarchy, the interaction model describes the interfaces for high-level functional control and management of a given shared object, and the environment model describes the state data of the available service modules for processing a given shared object. Shows the connection relationships (in functional dependency graph format) between related service modules for handling shared objects. In addition, the application model provides the synthesis logic of service modules for processing a given shared object, and instructs relevant service modules according to each operation. It includes a workflow that describes the selection of related service modules and their combination (by the actual interface connection), and in the data model, data is collected from service modules running to process a given shared object. Managed in form data. For example, when performing follow-me display task data that moves visual content between display devices according to the user's location, the user's location information must be collected and represented from a location tracking service that detects the user's location. It stores the structured information by associating the identifier for the image.

Next, the service layer provides basic service functions for users to use basic collaborative devices. To this end, service data, which is additional software that can directly access collaborative application modules (such as voice / video transmission programs), is provided. do. Since service modules are developed based on distributed component technology, they basically provide an interface to allow access to other service modules on a network and to use service functions. However, actual access and control may be limited by given privileges. The approach to service functions over the network allows service modules to interact and make them flexible as new collaborative application services. Such a service layer is implemented as a service step in which at least one service module controlled by the service command data provides service data for executing the collaboration devices.

The interaction model describes the interface for running, configuring, and stopping the service module, the environment model provides the capability and configuration data of the existing collaborative application modules used by the service module, and the presentation model describes the operational state of the service module. We present a method to show the user the status data and the result obtained when executing the service module. In addition, the application model describes the collaborative functions provided by the service module, and the data model describes the state of the service module (execution / standby state) and the context obtained as the service data is processed (e.g., packet loss rate for network monitoring services). Data for monitoring results, such as).

FIG. 2A is a diagram for describing a concept of a hierarchical structure of a multilateral collaboration system and a configuration method according to an example of the present invention, FIG. 2B is a diagram for explaining an example of connection between manager modules, and FIG. 2C is a diagram for task data. 2D is an example for explaining the interaction between the manager module and the service module, and FIG. 2E is an example of an audio service description for explaining the service data.

 Hereinafter, a description will be given with reference to FIGS. 2B to 2E based on FIG. 2A to describe a hierarchical structure of a multilateral collaboration system and a method of configuring the same.

As shown in FIG. 2A, the hierarchical structure of the multi-party collaboration system and its configuration method includes a respective meeting node in which the multi-party collaboration environment participates in the collaboration, and each meeting node includes a mediator module and at least one manager module. And at least one service module controlled by each manager module.

The mediator module functions to provide task data for performing a target task by the user using basic collaboration devices in each of the plurality of collaboration nodes. The mediator module also functions to discover / configure / synthesize the collaboration devices and at least one service module available within the collaboration node.

Such a mediator module represents a meeting node, and each of the mediator modules can be synthesized using a contract network-based technology, and the network synthesis is connected to other nodes. This allows each meeting node to share the application services needed for collaboration.

The manager module is controlled by task data and functions to provide service command data in a preset manner for processing task data.

The service module is controlled by the service command data and functions to provide service data for executing the collaboration devices.

In more detail, the mediator module uses task graphs to represent the collaboration between manager modules (on local or remote collaboration nodes) to accomplish the purpose of collaboration in a collaboration environment. Such a task graph is implemented by a node graphical user interface (GUI) or a space GUI. To this end, the task graph describes the list of manager modules, how they operate, the interrelationships between manager modules, and the user context. Here, the interconnection relationship between the manager modules indicates whether direct interaction between the manager modules is possible. The user context includes the privileges, roles, service symbols, etc. that the user has in the collaboration node, and based on this context, the administrator module can provide differentiated collaboration service functionality.

The manager module is controlled by the task data provided by the mediator module, and each manager module can be connected / synthesized through the mediator module within each node, and the network module of the mediator module can be combined with the manager module of other nodes. Synthesis and linking are possible.

As shown in FIG. 2B, when the user of each node uses the interaction manager module and the display manager module, the interaction manager module and the display manager module of each node are combined and connected through the mediator module of each node. Each node's mediator module is connected to each node's interaction manager module to create a collaborative environment. This allows the user of each node to communicate the interaction information shared with the display to the remote participants, thereby enabling the interactive sharing display operation to be realized.

An example of task data for realizing such an interactive sharing display operation is shown in FIG. 2C. As shown in FIG. 2C, the task data includes component data for setting the manager module and composition data for synthesizing the manager module, and component data includes the name of the manager module. name), type, type, and service command data, and the composition data may include service composition data necessary for composition of the manager module.

As the task data, the manager module is selected according to the component data, and service command data (mode) to be transmitted to the service module to process the task data according to the type of the manager module is specified. In addition, according to the composition data (composition), the manager module required for processing the task data is designated, and the manager modules are synthesized with each other.

The manager module is a device or software that manages related service modules classified into common functional groups such as media, networking, display, multimodal interactions, data / task sharing, and movement. Therefore, it is responsible for setting / controlling the actions of service modules. This automatic service setting / control can hide complicated service operations from the user.

In the process of participating in the processing of a given task data, manager modules can collaborate with other manager modules. In addition, manager modules can share important collaboration contexts and negotiate with other manager modules to coordinate details such as scope and method of cooperation.

For example, as illustrated in FIG. 2D, in the task layer, a user moves an image represented on a network display using a pointing device through the mediator module 100. When selected (1), the mediator module 100 requires an interaction manager, which requires task data (operation mode, user context), as shown in FIG. 2C, to perform an interaction networked display task. The module 120 is transmitted to and commanded by the Interaction Manager 120 and the Display Manager 123 (2).

In the Service Manager Layer, the interaction manager module 120 and the display manager module 123 receive and control task data and process the task data according to a preset method.

The interaction manager module 120 transmits service command data to each service module according to the task data to execute a pointer service module 130 connected with the device (3), and adapts the display. A Service Module (Display adaptation Service) 133 is synthesized (4).

In the service layer, the pointer service module 130 transmits service data related to interaction service execution, so that the 3D mouse 140 or the space mouse 140 connected to the pointer service module 130 may be connected. The infrared rays pointer 141 is operated, and the display adaptation service module 133 transmits service data corresponding to the display adaptation service module 133 and is connected to the display adaptation service module 133. The position coordinate information according to the operation of the 3D mouse (Space Mouse) 140 or the infrared ray pointer (Infrared rays Pointer) 141 is obtained through the display adaptation module 142.

The display manager module 123 is synthesized and connected with the interaction manager module 120 according to the task data, and interacts with the interactive display control service module 135 and the network display service to process the task data. A module (Network Display Service) 137 transmits and operates the service command data corresponding to each.

Each service module 135 and 137 transmits service data related to display execution and transmits it to a network display device 143 connected to each service module 135 and 137, and displays a network display. The device (Network Display) 143 receives the position information according to the operation of the three-dimensional mouse 140 or the infrared pointer 141 from the display adaptation module 142, and displays the position change accordingly.

In the task data processing, if the interaction manager module 120 requests information about an image (5), the display manager module 123 provides the image information (5). If the interaction manager module 120 makes a request to move the image, the display manager module 123 determines whether to accept the request and then processes it.

At this time, since each collaboration node has limitations on service availability due to different resource conditions and task data processing, competition for preempting service usage rights between manager modules is generated, which makes it difficult to provide stable collaboration application services. May occur. To address this, the administrator module can determine priorities for service usage based on the mode of operation and the user context.

In more detail, the number of instances of the service data and the numerical range of the QoS parameters available to the service module are differentiated according to the operation method and the user context to adjust the problems caused when using the service.

For example, the media provider service module may have an upper limit of the image frame rate, the media consumer service module may set the number of devices available for decoding the image, and the pointing service module may set an upper limit on the pointing accuracy. have. If the collaborative node cannot provide the functions required by the task data from the perspective of the manager module, the service module is provided by adjusting the QoS parameter of the service module down.

To process a given task data, the manager module performs the synthesis of service modules based on the workflow. Workflows describe a formal way of coordinating the task data flow between service modules with interdependent relationships. Intermediate to connect and execute the interface between service modules according to the workflow selected by the specific operation method, and monitor whether the required function is performed normally.

For example, as illustrated in FIG. 2D, if a particular shared object is to be represented on a grid network display, the display manager module may be configured to display a plurality of display services having various resolutions and sizes present in the collaboration node. In combination, present content with the appropriate location and size on the network display.

Regarding the change of the collaboration node, the selection of the services corresponding to the workflow and the determination of the parameter value are set in an autonomous manner. This means that after searching for the most appropriate service module among the available service modules existing at the present time, the parameter value corresponding to the current situation should be input.

To respond to context variability over time, workflows are described through the mediator module's APIs that support service discovery, configuration, synthesis, and monitoring. In this way, if the workflow describes the appropriate conditions of the service module, the mediator module finds the service module that satisfies the given condition and maps it to the workflow.

Service modules must easily accommodate a diverse set of collaborative application modules with different capabilities and dedicated interfaces. To address these requirements, service modules need to understand the attributes of the collaborative application modules and the interfaces for running them in a predetermined way.

To solve this problem, the service data (<service>) provided by the service module as shown in FIG. 2E is the name, role, type, maximum of the collaboration application module for controlling the collaboration application module. It may contain specification data for the instance number (maxInstanceNumber) or configuration (cfg) and is characterized by such specification data elements. Roles are factors that describe whether service data supplies content, receives and utilizes it, or handles both. Type is an element that specifies the format of the content. The maximum instance number is the maximum allowable number of service instances. The configuration includes additional information, such as elements required to manipulate the service agent module, such as transport protocol, command line interface, QoS factor.

3 is a view for explaining the multi-party collaboration system and its configuration method according to an example of the present invention.

As shown, the multilateral collaboration system may include a mediator module 300, manager modules 400, and service modules controlled by each manager module.

The mediator module 300 provides task data for the user to perform a targeted task using basic collaboration devices within the collaboration node, and provides at least collaboration devices available within the collaboration node and at least It finds, configures, and synthesizes one service module. The mediator module 300 may include a service discovery unit 310, a service setting / executing unit 320, and a service synthesis unit 330.

The manager modules 400 are controlled by task data, and serve to provide service command data in a preset manner for processing task data.

The manager modules 400 may include media service categories 410, networking service categories 420, display service categories 430, and multimodal interactions classified according to the function of a service. Multimodal Interaction) service categories (440) and Data / Task Sharing and Migration (category) service categories (450), each of the administrator module is a service command data corresponding to each category at least one service Can be sent to the module.

The manager modules 400 each service module included in each service category 410 to 450 has basic and unique functions for assisting in the configuration of the collaboration node, and the user targets tasks within the collaboration node. To do so, you can access resources such as multiple devices in the collaboration node and collaboration application modules (eg, audio or video tools) that directly control each device.

Each service module is developed based on distributed component technology. Therefore, each collaboration node is basically allowed to access and control all instances of each service module on the network.

Each service module of the manager modules 400 includes information for describing an attribute of each service module called a service capability.

Service capabilities consist of role, type and interface attributes. Role attributes are classified into producers, consumers and coordinators according to the provision, acceptance and coordination of service functions for a given service module.

The type attribute represents a functional classification (eg, video, display, networking, interaction, sharing, mobility) for the service module. The interface property represents a calling method for accessing a service module.

The service module is controlled by the service command data and serves to provide service data for controlling the cooperative devices.

The media service category 410 is a Media Producer Service module 411, a Media Consumer Service module 413, and a Media Arbitration Service module 415, each of which is a service module. By transmitting the service command data for performing the corresponding function, the execution / end of each service module is controlled.

In more detail, the media supply service module 411 obtains the foreground and background images of the collaboration node, and delivers the foreground and background images to other collaboration nodes. To this end, the media supply service module configures and / or executes various video devices and applications to reflect selection factors of video quality such as video codec, video size and transmission rate intended by the user through management and operation of the media agent module. Acquire an image.

In addition, the media supply service module 411 delivers the foreground and / or background image of the collaboration space obtained in real time to the media receivers of the collaboration nodes participating in the collaboration session in the form of a stream through a transmission protocol and a multicast address. Here, the transport protocol and the multicast address may be set through manual configuration of the media coordination service module 415 or the user.

The media receiving service module 413 receives a collaboration image from other collaboration nodes and displays the received image.

To this end, the media receiving service module 413 receives an image provided by the media provider through a mutually agreed transport protocol and a multicast address through the media adjustment service module 415 or a user's manual setting.

The media receiving service module 413 decodes the received image using an image codec and / or an application and converts the received image into RGB (Red, Green, Blue) type data. The decoded video is played on a local device that is not shared or provided to a network supported display device.

The media coordination service module 415 plays a role of trouble shooting for improving the quality of experience (QoE) of the real-time media service.

To this end, the media coordination service module 415 assigns a multicast address for video delivery to a media provider of a collaborative node to which the media coordination node belongs, and then assigns the media provider's properties (eg, video codec and application) to the media of other collaboration nodes. To the coordination service module 415. In addition, the user of the media receiver selects a desired media provider and connects with the media receiver to induce the media receiver to receive a selective image.

In addition, the media coordination service module 415 identifies appropriate service capability differences between the media provider and the media receiver in case of abnormal video delivery due to a difference in service capability between the media provider and the media receiver, and then provides appropriate measures to overcome them. This enables continuous media delivery.

For example, the media coordination service module 415 overcomes the service capability difference by providing an additional service such as a transcoder when there is a service capability difference between the media provider and the media receiver.

In addition, the media coordination service module 415 analyzes end-to-end media quality dependent on network and system performance to cope with variable network and system performance, and based on the analyzed result, appropriate media such as application level control parameters. By presenting adaptive delivery policies to media providers and media audiences, adaptive media suitable for network and system performance can be delivered.

The networking service category 420 includes a Multicast Connection Service Module 421, a NAT / Firewall Traversal Service 423, and a Network Monitoring Service Module 425. To control.

The multicast connection service module 421 provides multicast connectivity between collaboration nodes having no multicast connectivity.

That is, the multicast connection service module 421 is a multicast tunneling protocol, multicast in the application layer in the process of communication between the multicast networking-based collaboration nodes existing on the multicast partially supported network (hybrid multicast) network Through the use of cast loop resolution techniques and / or setting up information on servers that support partial multicast, transparent and efficient multicast connectivity can be provided without modifying separate media transmission programs or setting up complex networks.

The network address translation (NAT) / firewall crossing service module 423 provides multicast connectivity between collaboration nodes connected to a NAT and / or firewall-based private network.

That is, the NAT / Firewall Cross Service Module 423 provides transparent multicast connectivity to collaborative nodes connected to a private network without changing NAT and / or firewall devices or changing the settings of NAT and / or firewall devices by an administrator. .

To do this, a connection request from the internal network to the external network is established at the time of network connection establishment by using the NAT / firewall traversing service module 423 that the connection request from the internal network to the external network must be preceded when passing through the NAT and the firewall device. Prior to this, the request for connection from the external network back to the internal network using the set information provides interactive multicast connectivity.

The network monitor service module 425 monitors multicast connectivity and networking performance between the collaborative nodes communicating on the basis of multicast.

That is, the network monitor service module 425 periodically transmits a probe packet informing its existence to a multicast address shared by each collaboration node through setting whether to execute network monitoring and setting network monitoring server information. Therefore, based on the information of the collaboration node provided by the probe packet, the packet serial number and / or time information, the multicast performance between each collaboration node is identified.

Multicast performance between each collaboration node is collected on a central server and provides a comprehensive view of multicast connectivity and networking performance information for each collaboration node in which the central server is participating in a collaboration session, and the information provided is a key indicator for multicast performance analysis. Is utilized. The network monitor service module 425 analyzes the multicast performance by using the multicast connectivity and networking capability information for each of the collaboration nodes provided from the central server.

The display service category 430 controls the interactive display control service module 431, the visualization service module 433, and the networked display service module 435.

The interactive display control service module 431 receives a display control command from a user or interactive services modules and provides a control service for integrally operating networked displays. Here, the network display refers to a display device capable of receiving and displaying display and display control data through a display device or a network interface device on which a network interface device is installed.

To this end, the interactive display control service module 431 provides a messaging method for a user to generate a network display control command through a Graphic User Interface (GUI) or an interactive service. Here, the network display control command includes a control command for starting, ending, moving, and resizing an image, for example.

In addition, the interactive display control service module 431 decodes and / or renders the visualization data through a media accommodation service and / or a visualization service to display visualization data selected by a user, that is, an image, an image, graphics, and the like, on a network display. It then plays a role of relay that connects with network display service.

The visualization service module 433 performs a function of converting image / graphics files of various formats into RGB format.

To this end, the visualization service module 433 performs a rendering function of converting an image compressed in various formats into RGB data using an image library. In addition, the visualization service module 433 sets the transmission protocol and the IP (Internet Protocol) address of each network display device, and divides the image / graphics data according to the corresponding area in consideration of the resolution and location of each network display device. By using the Transmission Control Protocol (TCP) or the User Datagram Protocol (UDP), the divided data and the corresponding data size, output location, and ratio information are integrated and transmitted to the network display device.

The network display service module 435 receives the image / graphics data in the RGB format and expresses the image and graphics data in the designated position and size in each network display device.

To this end, the network display service module 435 provides a synchronization scheme for network display devices having different resolutions, shapes, and positions to be organically operated like a single display device, manages status information of each network display device, and provides a user with In order to express RGB data, each network display device is integrated and controlled.

In addition, the network display service module 435 expresses the RGB data to the network display device by using the RGB data received through the network and various display information (for example, display size and position) on the OpenGL-based display. .

The multi-modal interaction service category 440 controls the pointing service module 441, the hand motion tracking service module 443, and the location tracking service module 445. .

The pointing service module 441 performs a function of enabling a user to point or gesture to a specific location within a collaboration node by using a laser pointing device. To this end, the pointing service module 441 acquires a pointing image using a laser device for pointing, a camera filter for distinguishing a laser point, and a camera for tracking a pointing position, and uses the computer vision technology (eg For example, the pointing position is tracked by calculating the exact pointing coordinates using the calibration between the cameras and the camera lens undistortion.

In addition, the pointing service module 441 analyzes the pointing location information tracked in real time and recognizes a gesture intended by the user.

The hand motion tracking service module 443 supports the control and sharing of a 3D CAD model using hand gesture tracking.

That is, the hand motion tracking service module 443 collects the joint data of the hand in real time by using a device capable of tracking the hand motion to configure the kinematic appearance of the hand. In addition, the position of the 3D CAD model is controlled by observing the 3D CAD model on a display in real time and interpreting hand gestures to track the current position and the state of the hand to interact with the 3D CAD model.

The location tracking service module 445 provides a function for tracking the location of a specific target in the collaboration node. That is, the location tracking service module 445 estimates the 3D location coordinates inside the collaboration node with respect to the target for which location tracking is required by using location sensor-based user location calculation technology for dynamic user context information recognition and analysis. In addition, in order to improve the position estimation accuracy, an additional sensing technique such as an ultrasonic (ultrasonic) is used in the existing RF (Radio Frequency) based position recognition method to provide an accuracy having an estimation error within a position of 0.5m.

The data / task sharing and movement service category 450 includes a data sharing service module 451, a data adaptation service module 453, and a remote user interface service module. Control 455.

The data sharing service module 451 supports distributed shared data storage for sharing data held by individual collaboration nodes in all collaboration nodes participating in a collaboration session.

That is, the data sharing service module 451 provides a distributed shared data storage that can store any data file to which other users participating in the collaboration session are allowed to access, and upload and download shared data. And retrieve information. Shared data repositories, along with shared data files, provide knowledge about file ownership, file access rights, and controls.

The data adaptation service module 453 provides a processing technique for delivering adaptive shared data and / or application views in consideration of the performance of each collaboration node. That is, the data adaptation service module 453 identifies the unique characteristics of each collaboration node such as network, system performance, and / or user preference for each collaboration node, and then processes the view of the shared data to be accommodated in the corresponding node (eg, For example, transcoding is provided to the user.

The remote user interface service module 455 supports a user interface for sharing and controlling data and / or applications of the remote collaboration node. That is, the remote user interface service module 455 obtains a screen in which an application operates in real time, delivers it to a remote user, and transfers the control right of an input device such as a keyboard and a mouse together so that the corresponding application can be operated in real time. Provide remote interface function.

As described above, FIG. 3 illustrates a horizontal structure of each manager module and each service module according to its function, but FIG. 4 looks at a vertical structure according to the functions of the mediator module, the manager module, and the service module.

4 is a diagram illustrating an agent-based management toolkit structure in a multilateral collaboration system and a method of configuring the same according to an embodiment of the present invention.

As shown, the agent-based management toolkit structure includes a mediator module 300, an administrator module 400, and a service module.

The collaboration devices in each node of the multiparty collaboration system include a respective device agent module, and each device agent module includes a service agent module that operates on each device agent.

In addition, the mediator module 300 includes a service discovery unit 310 (Service Discoverer), a service setting / executing unit 320 (Service Configurator), a service synthesis unit 330 (Service Composer), and a node GUI (Node Graphical User Interface). A unit 340, a space GUI (Space Graphical User Interface) unit 350, and a collaboration situation storage unit (Space Repository) 360 may be included.

The mediator module 300 discovers / configures / synthesizes the collaboration devices and the service module available in the collaboration node to set up / execute the system. In addition, it represents a collaboration node and functions to synthesize services and other collaboration nodes participating in the collaboration environment.

The mediator module 300 has configuration information of all service modules available in the collaboration node, checks the status of service modules available to the user, and targets tasks through the collaboration device in the collaboration node. When the service modules require a specific function to perform, each service module may process a specific function with the help of the mediator module 300.

Looking at the components of such a mediator module 300 in more detail as follows.

The service discovery unit 310 searches for the collaboration modules available in the collaboration node and the service modules that can interface with the manager module 400 in each device, and discovers the available service modules, and the status data of the service module. Function to store in the collaboration situation storage unit 360 for storing the.

In addition, the service discovery unit 310 searches for a device agent module mounted in the collaboration devices to search for the collaboration devices, and displays the collaboration device in the node GUI 340 through the searched device agent module. Do it.

The service setting / executing unit 320 sets the manager module 400 and the service module to execute / end the searched service module and transmits task data to the manager module 400.

In addition, the service setting / executing unit 320 may add / delete a service module or an administrator module 400 available for the searched collaboration device displayed on the node GUI 340 according to a user's selection, or the node GUI unit 340. When the collaboration devices indicated by the user and the service module available within the collaboration device are selected by the user, the manager module 400 and the service module are automatically set to enable the execution / end of the service module, and the execution / end of the service module is performed. For transmitting the task data to the manager module 400.

In order to perform a given task, the service synthesis unit 330 queries the mediator module 300 of other collaboration nodes whether there is an administrator module 400 to process task data in the space GUI 350 and according to the result. In order to form a connection relationship between the manager modules 400, the manager module 400 plays a role of synthesizing the manager module 400 through the mutual interface binding between the manager modules 400.

More specifically, the service synthesizing unit 330 automatically selects / synthesizes at least one manager module 400 for processing task days when any one of task data displayed in the space GUI 350 is selected by the user. Function

Here, the automatically selected / synthesized manager module 400 searches / sets at least one service module to be synthesized to process task data, and services in a preset manner according to the task data transmitted from the mediator module 300. It sends the command data to the searched / configured service module.

The node graphical user interface (GUI) unit displays a list of the cooperative devices retrieved by the service discovery unit 310 and the service module available in the cooperative devices and the manager module 400 controlling the service module, and operating the service module. It provides basic functions related to (setting, starting, ending).

The space GUI (Graphical User Interface) unit displays the task data available through the manager module 400 and the service module set in the service setting / executing unit 320 to provide an interface function for a user to perform a desired task, and the actual work. When executed, it provides the ability for the user to manipulate the shared objects in detail.

The collaboration situation storage unit 360 stores a spatial state of a meeting node, and more specifically, collects state data (service capability, service status, contextual information such as user preferences, etc.) from service modules in a collaboration environment. Function to save.

The manager module 400 is controlled by the task data supplied from the mediator module 300, provides the service command data in a preset manner to process the task data, the service module is controlled by the service command data, Provides service data for running collaborative devices.

Here, the service module includes a service agent module. The service agent module receives the service data from the service module and controls the collaboration device to be executed / ended by allowing the collaboration application module that directly controls the collaboration devices to be controlled by the service data. do.

In more detail, when at least one service module transmits service data to a service agent module operating on the device agent module of the collaboration device, the service agent module transmits a predetermined API according to the service data to the collaboration application module. Thereafter, the collaboration application module directly controls the execution / end of the collaboration devices. In this way, the user can perform a target task in a system in which a multi-party collaboration environment is established.

Here, the device agent module and the service agent module will be described in more detail. The device agent module is mounted in a PC, a mobile terminal, and an embedded device operating in a collaboration environment, and registers the collaboration device with the mediator module 300. The service module or the manager module 400 is added / deleted according to the command of 300). In addition, the system performance (CPU occupancy, main memory usage, etc.) and service state data of the collaboration device is grasped and transmitted to the mediator module 300.

The service agent module is a module that makes the collaboration application module that directly controls the collaboration device controlled by the service module. Such a service agent module grasps the service capability based on the description of the collaborative application module, collects the service context received during execution, and transmits the service context to the mediator module 300 and the manager module 400. In addition, the function of the cooperative application module is controlled according to the service operation command received from the mediator module 300 or the manager module 400.

The service module utilizes the ontology to describe the data model and implements the service interface based on the agent communication protocol (eg, query, request, contract network, etc.) to practice the application model.

Various functions provided by such a service agent module are implemented with a predetermined API, as shown in the table shown in FIG. 5.

6 is a diagram for describing service synthesis of a multi-party collaboration system according to an example of the present invention.

As shown, the multi-party collaboration system includes a mediator module of a first node that intends to newly participate in a plurality of collaboration nodes and a mediator module of a second node that participates in a plurality of collaborations.

Such a method of constructing a multilateral collaboration system includes a mediator module of a first node that intends to newly participate in a plurality of collaboration nodes using a contract network-based technology, which is an interaction protocol, and a mediator of a second node that already participates in a plurality of collaborations. The module may further include a network synthesis step of combining the manager modules of the first node and the second node with each other.

In more detail, the network synthesis step may include a work notification step (1), a proposal step (2), a contract step (3) and a result reporting step (4).

The job notification step (1) searches for the manager module indicated by the vertex on the work graph in the first node, and the manager module of the second node to be synthesized with the found service manager module. A call for proposal is sent to the mediator module of the 2 node.

Proposal step (2) indicates that if there is a manager module in the second node to be synthesized with the manager module retrieved from the first node by the mediator module of the second node, the second node may be the target of connection. The notifier transmits a proposal message to the mediator module of the first node. In this case, if the mediator module of the first node to newly participate in the work does not meet a specific condition, it may add a clue that the proposed content may not be valid.

The contract step (3) selects a second node that the mediator of the first node can synthesize from the proposal message, and accepts the contract message (Accept) for synthesizing the manager module and the agent ID of the manager module of the first node. Send to the module so that the mediator module of the second node can update the contents of the service synthesis.

In the result reporting step (4), the mediator module of the second node adds / updates the manager module of the first node using the agent ID of the manager module of the first node, and then updates the fact to the existing manager module in the second node. send.

In addition, the network synthesis step may further include, after the contracting step, the mediator module of the second node transmitting the agent IDs of the manager modules already participating in the collaboration to the mediator module of the first node to participate in the collaboration. .

Through this network synthesis process, it is possible to maintain service connection information in each node of a multi-party collaboration environment, and as a result, an agent-to-agent network for service synthesis between mediator modules and manager modules of all nodes participating in a given task. Will be constructed.

Then, the serviceLinkAdded method of the manager module on the first node newly participating in the collaboration is called, so that the manager module of the first node can be known, and the manager module of the first node negotiates with the manager module on the second node. Link service modules on a first node, executed according to a given workflow, with service modules on a second node.

After that, the service modules connected to each other can interact with each other for the processing of shared objects while exchanging service contexts described by the ontology.

7 is a view for explaining an example of a node GUI of the mediator module according to the present invention.

7 shows the node GUI of the mediator module, where "GISTNode" is the name of the collaboration node and "broadband" is the name of the collaboration device in the collaboration node. And "MediaManager" represents the manager module for controlling the collaboration device, "svc: // ~" represents the name of the service module available in the collaboration node.

As shown, users participating in the collaboration can access various service agent modules running on the device agent through the node GUI. When the service discovery unit in the mediator module finds / discovers the device agent module mounted in the collaboration devices in the node, the node GUI shows the name of the collaboration device and a list of available service modules. Therefore, users participating in the collaboration through the node GUI can see the connected devices and the services that can be supported.

In addition, using the node GUI, when a user adds / deletes service agent modules operating on the device agent module, the service setting / executing unit adds / deletes a service module including the corresponding service agent module according to the user's selection. .

Therefore, when the service agent module is added, the service module including the service agent module is registered in the mediator module and is managed according to the manipulation command of the mediator module.

All agents in the collaboration node communicate using HTTP-based distributed agent communication protocol. Since message transfer between agents is based on a FIPA compatible agent communication protocol, it is possible to communicate with an ontology-based collaboration context.

8 is a view for explaining an example of the space GUI of the mediator module according to the present invention.

As shown, for collaboration between collaboration nodes, users participating in the collaboration via the space GUI can access the collaboration session directory agent. In the space GUI, a list of selectable task data is presented to enable the user to perform a targeted task using basic collaboration devices in an attached collaboration session.

In FIG. 8, for example, video conference task data is supported. When the user selects this task data, the manager module corresponding to this task data graph is automatically selected, and a connection / synthesis is made between the manager modules.

Each manager module finds the service module to be synthesized according to the preset workflow in the node to which the manager module belongs, sets the service module according to the predefined service parameter, and executes the service module.

As such, the video conference task data is processed by the media manager module. The media manager module connects the media provider service module and the media consumer service module to each other so that video conferencing between users who participated in the collaboration can be performed.

9 is an example of a media manager GUI showing an execution screen of a media manager module in a multi-party collaboration system according to the present invention.

As shown, the GUI of the media manager module shows the execution screen of the media manager module, and a service having the ability to send an image (ie, role is provider and type media) by pressing the "Send" button. Search for modules and mark them for user selection. The main contents of the selected media provider service module are listed on the GUI of the media manager module of all participating nodes, and when the user selects it and presses the "Play" button, the user can select the ability to receive a video stream from the selected media provider service module. Search for and display the media consumer service modules that have it. The user can select one of them to receive an image.

Meanwhile, the above-described embodiments of the present invention can be written as a program that can be executed in a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. The computer-readable recording medium may be a magnetic storage medium (for example, a ROM, a floppy disk, a hard disk, a magnetic tape, etc.), an optical reading medium (for example, a CD-ROM, a DVD, an optical data storage device, etc.). And storage media such as carrier waves (eg, transmission over the Internet).

The above description is merely illustrative of the technical idea of the present invention, and various modifications, changes, and substitutions may be made by those skilled in the art without departing from the essential characteristics of the present invention. will be. Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical spirit of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by the embodiments and the accompanying drawings. . The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

1 is a table for explaining a conceptual model of a multi-party collaboration system and its configuration method according to an example of the present invention.

2A is a diagram for explaining a concept of a hierarchical structure of a multilateral collaboration system and a method of configuring the same according to an embodiment of the present invention.

2B is a diagram for explaining an example of connection between manager modules;

Fig. 2C is an example of creating a task data graph in an XML document to explain task data.

2D is a diagram for explaining an interaction between an administrator module and a service module.

2E is an example of an audio service description for explaining service data.

3 is a view for explaining an example of a multi-party collaboration system and a configuration method according to the present invention.

4 is a diagram illustrating an example of an agent-based management toolkit structure in a multi-party collaboration system and a method of configuring the same according to the present invention.

5 is an example of an API provided by a service agent module according to the present invention.

6 is a view for explaining an example of service synthesis of a multi-party collaboration system according to the present invention.

7 is a view for explaining an example of a node GUI of the mediator module according to the present invention.

8 is a view for explaining an example of the space GUI of the mediator module according to the present invention.

9 is a view for explaining an example of a media manager GUI showing an execution screen of a media manager module in a multi-party collaboration system according to the present invention.

<Description of reference numerals for the main parts of the drawings>

300: mediator module 310: service discovery unit

320: service execution / setting unit 330: service synthesis unit

340: Node GUI unit 350: Space GUI unit

400: manager module

Claims (26)

The mediator module may include: a task step of providing task data to at least one manager module to enable a user to perform a target task using basic collaboration devices at each of the plurality of collaboration nodes; The at least one manager module controlled by the task data provides a service command step of providing service command data to at least one service module in a preset manner for processing the task data; The at least one service module controlled by the service command data includes a service step of providing service data for executing the collaboration devices; And The mediator module may include a system setup step of discovering / setting / synthesizing the collaboration devices and the service module available in the collaboration node; How to configure a multi-party collaboration system comprising a. The method of claim 1, The system setting step A service discovery step of searching for the service module capable of interfacing with the collaboration devices and the manager module and storing the state data of the service module; A service setting step of setting the manager module and the service module to execute / end the searched service module; And A service synthesis step of synthesizing the manager modules through mutual interface binding between the manager modules to process the task data; How to configure a multi-party collaboration system comprising a. The method of claim 2, The system setting step A node status display step of displaying the collaboration devices retrieved in the service discovery step, the service module available in the collaboration devices, and the manager module controlling the service module in a node graphical user interface (GUI); Method of configuring a multi-party collaboration system further comprising. The method of claim 3, wherein The service setting step When the collaboration device and the service module available in the collaboration device displayed in the node status display step are selected by the user, the mediator module enables the manager module and the service module to execute / end the service module. Comprising the step of setting How to configure a multi-party collaboration system characterized in that. The method of claim 4, wherein The service discovery step Searching for a device agent module mounted in the collaboration devices to search the collaboration devices, and displaying the collaboration device in the node GUI through the searched device agent module; The service setting step Adding / deleting the service module or the manager module available through the searched collaboration device by the mediator module according to the user's selection; How to configure a multi-party collaboration system comprising a. The method of claim 2, The system setting step Displaying a manager module set in the service setting step and the task data available through the service module in a space graphical user interface (GUI); Method of configuring a multi-party collaboration system further comprising. The method of claim 6, The service synthesis step Automatically selecting / synthesizing at least one manager module for processing the task data if any one of the displayed task data is selected by the user; Retrieving / setting at least one service module to be synthesized by the selected / synthesized manager module to process the task data; Method of configuring a multi-party collaboration system further comprising. The method of claim 1, The manager module manages media, networking, display, multimodal interactions and related service modules classified into common functional groups such as data and task sharing and movement, Transmitting, by the manager module, service command data corresponding to each of the function groups to the at least one service module in the service command step. How to configure a multi-party collaboration system characterized in that. The method of claim 1, The service phase Transmitting the service data to a service agent module such that a collaboration application module, wherein the at least one service module directly controls the collaboration devices, is controlled by the service data; Transmitting, by the service agent module, application programming interfaces (APIs) preset according to the service data to the collaborative application module; And Directly controlling the execution / termination of the collaboration devices by the collaboration application module. How to configure a multi-party collaboration system characterized in that. The method of claim 1, How to configure the multi-party collaboration system The first node through a mediator module of a first node to newly participate in the plurality of collaboration nodes using a contract network-based technology, which is an interaction protocol, and a mediator module of a second node already participating in the plurality of collaborations. And a network synthesis step of synthesizing the manager modules of the second node with each other. How to configure a multi-party collaboration system characterized in that. 11. The method of claim 10, The network synthesis step The mediator module of the first node searches for a manager module in the first node and sends a notification message to the mediator module of the second node to find the manager module of the second node to be synthesized with the found manager module. Notification step; If there is a manager module in the second node in which the mediator module of the second node is to be synthesized with the searched manager module, a proposal message indicating that the second node may be a connection target is transmitted to the mediator module of the first node. Proposal step to; A contract that the mediator of the first node selects a second node that can be synthesized from the proposal message, and transmits a contract message for synthesizing the manager module and an agent ID of the manager module of the first node to the mediator module of the second node. step; And The mediator module of the second node adds / updates the manager module of the first node using the agent ID of the manager module of the first node and then transmits the update fact to an existing manager module in the second node. Including a result reporting step; How to configure a multi-party collaboration system characterized in that. The method of claim 11, The network synthesis step After the contracting step, the mediator module of the second node further comprising transmitting an agent ID of the manager module that is already participating in the collaboration to the mediator module of the first node. How to configure a multi-party collaboration system characterized in that. A mediator module that provides task data for each user performing a target task using basic collaboration devices in each of the plurality of collaboration nodes; At least one manager module controlled by the task data and providing service command data in a preset manner for processing the task data; At least one service module controlled by the service command data and providing service data for executing the collaboration devices. The mediator module discovers / sets / synthesizes the collaboration devices and the at least one service module available within the collaboration node; Multi-party collaboration system characterized by. The method of claim 13, The mediator module Search for the collaboration devices available in the collaboration node and the service module that can interface with the manager module in the respective collaboration devices, and store in the collaboration context storage to store state data of the service module Service discovery department; A service setting / executing unit configured to set the manager module and the service module to execute / end the service module and to transmit the task data to the manager module; A service synthesizer for synthesizing the manager module through mutual interface binding between the manager modules to process the task data; Multilateral collaboration system comprising a. The method of claim 14, wherein the mediator module A graphical user interface (GUI) unit for displaying the collaboration devices retrieved by the service discovery unit, a service module available in the collaboration devices, and the manager module controlling the service module; Multilateral collaboration system comprising a. The method of claim 15, The service setting / executing unit The manager module and the service module are configured to enable / disable the service module when the collaboration devices displayed in the node GUI unit and the service module available in the collaboration device are selected by the user. Sending the task data to the manager module for execution / shutdown of the module Multi-party collaboration system characterized by. The method of claim 14, The service discovery unit searches a device agent module mounted in the collaboration devices to search the collaboration devices, displays the collaboration device in the node GUI through the searched device agent module, The service setting / executing unit adds / deletes the service module or the administrator module available for the searched collaboration device according to the user's selection. Multi-party collaboration system characterized by. The method of claim 14, The mediator module A space graphical user interface (GUI) unit for displaying an administrator module configured in the service setting / executing unit and the task data available through the service module; Multi-party collaboration system, characterized in that it further comprises. The method of claim 18, The service synthesis unit Automatically selecting / synthesizing at least one manager module for processing the task data when any of the displayed task data is selected by the user in the space GUI Multi-party collaboration system characterized by. The method of claim 19, The automatically selected / synthesized manager module Search / set at least one service module to be synthesized for processing the task data, and transmit service command data to the searched / set service module in a preset manner according to the task data transmitted from the service setup / execution unit; To do Multi-party collaboration system characterized by. The method of claim 13, The manager module manages media, networking, display, multimodal interactions and related service modules classified into common functional groups such as data and task sharing and movement, Wherein each of the manager modules transmits service command data corresponding to each of the functional groups to the at least one service module. Multi-party collaboration system characterized by. The method of claim 13, Each of the at least one service module A service agent module configured to receive the service data so that a collaboration application module which directly controls the collaboration devices is controlled by the service data; Multilateral collaboration system comprising a. The method of claim 13, The multilateral collaboration system A mediator module of a first node to newly participate in the plurality of collaboration nodes and a second mediator module participating in the plurality of collaborations, The mediator modules of the first and second nodes are synthesized with each other using a contract network based technology which is an interaction protocol. Multi-party collaboration system characterized by. The method of claim 14, The task data Component data for setting the manager module in the service setting / executing unit and synthesis data for synthesizing the manager module in the service synthesizing unit, The component data includes a name, a type of the manager module, and the service command data; The synthesis data includes service synthesis data necessary for synthesis of the manager module. Multi-party collaboration system characterized by. The method of claim 22, The service data Wherein the service module includes specification data of a name, role, type, maximum instance number or setting of a service agent module for controlling the service agent module. Multi-party collaboration system characterized by. In a computer-readable recording medium, A recording medium on which a program for implementing the method according to any one of claims 1 to 12 is recorded.

Images