KR20050056041A - Method and system for controlling object in order to share resource between heterogeneous devices - Google Patents

Abstract

According to the present invention, when a sharing participating terminal receives a raw screen image corresponding to an application program from a communication server and displays the raw screen image on a display unit, when an event occurrence is detected, the terminal participates in generating a predetermined form of event information corresponding to the event and transmitting the generated event information to the communication server. And, when partial image information corresponding to the event information is received from the communication server, updating the region corresponding to the event generated in the raw screen image by using the partial image data and the image coordinate information. And it relates to a system, by converting the minimum area image corresponding to the changed position to the image of the DIB (Device Interface Bitmap) form, and then compressing and transmitting to enable application program sharing between heterogeneous devices.

Description

Method and system for controlling object in order to share resource between heterogeneous devices}

The present invention relates to an object control method and system for resource sharing between heterogeneous devices. In particular, the present invention relates to resource sharing between heterogeneous devices for enabling collaboration through resource sharing of application programs and data, regardless of an operating system environment of a user terminal. An object control method and system for the same.

Due to the development of science and technology and the improvement of economic level, various user terminals (eg, computers, Unix systems, personal digital assistants (PDAs), etc.) are being developed for automating or convenient human tasks. These terminals are provided with any one of various operating systems, such as Windows, and have various application programs that operate under the operating system. By using the application programs provided as described above, a user can conveniently work on a document or surf the web.

And as the exchange of human society becomes closer, the development of communication networks also enables the sharing of information (eg, work documents) among geographically dispersed users. Scott McNealy of Sun Microsystems, USA, is over. It has professed to be a platform-independent network era, and this trend suggests that users will no longer rely solely on PCs for Internet-based telecommunications networks. As such, in order to pursue a platform-independent network era, not only simple data sharing but also application sharing should be enabled. In order to enable sharing of applications, heterogeneous input event hooking, emulation technology that emulates hooked event information in a shared application on heterogeneous operating systems, and executes the same result, and heterogeneous application programs. However, there is a need for DIB compression and decompression using a GUI screen sharing technique, and multi-object object synchronization and wired / wireless communication in the form of collaboration. However, the conventional PC-based application sharing technology requires application programs in the same environment. It is a sharing technique. In addition, MC Global, Netmeeting, PC Anyware, and WebX are the products that use application sharing technology under the same OS. These products have limitations that allow applications to be shared only on the Windows operating system.

In addition, conventional application program sharing technologies include a capture board, a graphic device interface (GDI) hooking technology, a virtual driver installation method, and a GUI screen capture method. Using the capture board is the fastest and most reliable screen capture method. However, there is a problem in that separate hardware must be used and different vendors provide different drives. In this case, screen capture is possible independently of the CPU, so there is no system load, but hardware has to be installed, and there is a problem in that it does not have generality by using a separate application program interface (API) that supports the hardware.

In addition, GDI hooking technology is a technology for hooking at the operating system level that processes user events for a shared application, and has a small amount of data and a high processing speed, but interoperability is not possible in heterogeneous platforms.

The virtual drive installation method is a method in which the virtual drive intercepts functions that are passed to the video graphic adapter (VGA) driver at the operating system level when performing a user event for a shared application. This method has excellent performance in terms of execution speed and data volume, but it has difficulty in implementation and platform independence because a virtual drive corresponding to VGA for each user's system must be implemented. The result of the user's input event is to capture the screen and send it unilaterally to all users to share the same screen. The problem is that the screen capture time and the amount of data to be transferred are large.

Accordingly, in order to solve the above-described problem, an object of the present invention is to share an application program between heterogeneous devices by converting a minimal area image corresponding to a changed position into a DIB (Device Interface Bitmap) type image, performing compression, and then transmitting the same. An object control method and system for sharing resources between heterogeneous devices is provided.

Another object of the present invention is to escape from the limited working area of each user terminal that was limited to a specific platform to perform only a limited task for maximizing the efficiency of the work and for cost sharing between heterogeneous devices Another object of the present invention is to provide an object control method and system for resource sharing between heterogeneous devices capable of interactive education regardless of the type of a user terminal in remote education.

It is still another object of the present invention to provide a method and system for controlling an object for resource sharing between heterogeneous devices where a remote patient and a doctor can perform remote medical care using a user terminal under a separate and different operating system.

It is still another object of the present invention to provide a method and system for controlling an object for resource sharing between heterogeneous devices that enables dynamic presentation through application sharing in the case of teleconferencing using video conferencing.

It is still another object of the present invention to provide a method and system for controlling an object for sharing resources between heterogeneous devices to diagnose and repair a failure or an error of a specific user terminal regardless of the type of an administrator terminal at a remote location.

Another object of the present invention is a heterogeneous type that can be used or tested in advance through application sharing in a test bed in which the application is installed, without having to install the application in a user terminal for testing a specific application developed. An object control method and system for sharing resources between devices is provided.

In order to achieve the above object, according to an aspect of the present invention, in a method for sharing an application installed in a source providing terminal in a sharing participating terminal via a communication network, wherein the sharing participating terminal is the source via a communication server; Coupled to a providing terminal and a communication network, and not including the application program; receiving a raw screen image corresponding to the application program from a communication server and displaying the raw screen image on the display unit, wherein the raw screen image is stored in the application program. It is a work screen image according to execution; Generating event information of a predetermined form corresponding to the event when an event occurrence is detected, wherein the event is at least one of a mouse event and a keyboard event; Transmitting the event information to the communication server through a communication network; Receiving partial image information corresponding to the event information from the communication server, wherein the partial image information includes at least partial image data and image coordinate information corresponding to the partial image data; And updating the area corresponding to the generated event in the raw screen image by using the partial image data and the image coordinate information, wherein the source providing terminal receives the event information from the communication server, The application program is executed using the event information, the execution result screen is captured, the partial image information is generated, and the partial image information is transmitted to the communication server through a communication network. A sharing method is provided, and a system, apparatus, and recording medium are provided for enabling the execution of an application program sharing method using the communication network.

When the mouse event occurs, generating the predetermined type of event information comprises: detecting the occurrence of the mouse event; Generating a mouse pointer position corresponding to the mouse event as mouse coordinate information; And generating packet data corresponding to the mouse event by using a mouse event indicator and the mouse coordinate information.

In addition, when the keyboard event occurs, generating the predetermined type of event information comprises: detecting the occurrence of the keyboard event; Extracting an input key button value corresponding to the keyboard event; And generating packet data corresponding to the keyboard event by using a keyboard event indicator and the input key button value. The packet data may include an event type field indicating the mouse event or the keyboard event; It may include a data field including mouse coordinate information corresponding to a mouse event or an input key button value corresponding to the keyboard event.

The sharing participating terminal and the source providing terminal may be heterogeneous terminals, respectively.

The updating of an area corresponding to the generated event in the raw screen image may include: determining an area corresponding to the image coordinate information among the raw screen image; And replacing the determined area with the partial image data.

The mouse event may include at least one of mouse movement, mouse click, double click, and drag and drop.

In a preferred embodiment of the present invention, in order to share an application installed in a source providing terminal with a sharing participating terminal through a communication network, a screen image providing method performed by the source providing terminal, wherein the communication server is the sharing through the communication network Coupled with a participant terminal and the source providing terminal, wherein the shared participant terminal does not include the application program; executing the application program; Generating a raw work screen image according to execution of the application program and transmitting the raw work screen image to the communication server through a communication network; Receiving event information from the communication server, wherein the event information includes an event type field indicating a mouse event or a keyboard event, mouse coordinate information corresponding to the mouse event, or an input key button value corresponding to the keyboard event. Including a data field comprising; Operating the application program according to the event information; Generating an update work screen image corresponding to an operation result of the application program; Generating a partial image corresponding to a different area of the original work screen image and the update work screen image; Generating partial image information using the partial image, wherein the partial image information includes at least partial image data and image coordinate information corresponding to the partial image data; And transmitting the partial image information to the communication server through a communication network, wherein the sharing participating terminal receives and displays the raw work screen image, and an event occurs in any region of the raw work screen image. If detected, generates the event information corresponding to the generated event and transmits it to the communication server, and updates the area corresponding to the event occurrence using the partial image information received from the communication server. Provided is an application program sharing method using the present invention, and a system, apparatus, and a recording medium for enabling the application program sharing method using the communication network are provided. The generating of the partial image corresponding to a different region of the raw work screen image and the update work screen image may include extracting a partial image of a region corresponding to the mouse coordinate information.

In addition, when the display screen of the source providing terminal is divided into a plurality of first regions, each of the first regions is divided into a plurality of second regions, and each of the first regions is managed by an independent change management unit. The display screen may be a full screen or an active window in which the application program is executed. The generating of the update work screen image and the generating of the partial image may include: a first area managed by the independent change manager; Capturing an area image of a; Determining, by the change management unit, whether the area image is changed from an image of a corresponding area of the raw work screen image; Requesting generation of a partial image by a change manager for managing a corresponding area when changed; And generating the changed region image as a partial image. When the occurrence of an event is detected, an application program sharing method using the communication network according to the present invention operates the application program according to the generated event. It may further comprise a step.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram schematically illustrating an overall system configuration for performing an object control method for resource sharing between heterogeneous devices according to a preferred embodiment of the present invention.

Referring to FIG. 1, the entire system for performing an object control method for resource sharing between heterogeneous devices includes a source providing terminal 110, a communication server 130, and a sharing participating terminal 160. The source providing terminal 110 and the sharing participating terminal 160 may be the same type of user terminal under the same operating system or different types of user terminal under different operating systems. The user terminal may be, for example, a computer, a personal digital assistant (PDA), a mobile communication terminal, a Unix system terminal, or the like. However, in this case, the source providing terminal 110 is a user terminal having a shared resource (for example, an electronic file such as an application program or an image), and the sharing participating terminal 160 is provided as a terminal having no shared resource. The shared resource provided by the terminal 110 will be defined as a user terminal to collaborate. Of course, although only one sharing participating terminal 160 is illustrated in FIG. 1, the source providing terminal 110 may be combined with a plurality of sharing participating terminals 160 through a communication network.

The source providing terminal 110 and the sharing participating terminal 160 share a resource held by the source providing terminal 110 and share a screen corresponding to an event generated in the source providing terminal 110 or the sharing participating terminal 160. And a screen management module (115a, 115b hereinafter referred to as 115) and an object management module (120a, 120b hereinafter referred to as 120). The screen management module 115 and the object management module 120 may be configured in a hardware configuration combined with a control unit (eg, a CPU) of the source providing terminal 110 and / or the sharing participating terminal 160, or may perform respective functions. The screen management module 115 included in the source providing terminal 110 may periodically perform an event (eg, a mouse click or a key button input) on the source providing terminal 110. When a screen display status update occurs or an event signal is received from the sharing participation terminal 160, a specific screen (for example, a full screen, an active window screen, and a partial screen corresponding to a changed part) is captured. The image data (for example, image data in the form of a bitmap) is transmitted to the communication server. The screen management module 115 included in the sharing participating terminal 160 restores the image data received from the source providing terminal 110 via the communication server 130 and displays the image data on the display unit of the sharing participating terminal 160. If the image data is a partial image corresponding to a part of the entire screen, only the corresponding region is updated using the partial image, and the zoom in and zoom out of the screen are managed.

In addition, the object management module 120 included in the source providing terminal 110 may generate an event (eg, a mouse movement, a mouse click, a double click, a drag and drop, etc.) generated in the source providing terminal 110. Keyboard events such as key button input, user access events such as connection, disconnection, role change, and screen display state updates such as banner advertisement window update) or events received from the sharing participating terminal 160 to be shared. Allows the function to be performed in an application. The object management module 120 included in the sharing participating terminal 160 may object by hooking the corresponding event when an event occurs in the sharing participating terminal 160, and convert the objectized event information to the communication server 130 through a communication network. To send). Since the object management module 120 must emulate mouse input, keyboard input, etc. in all user terminals, an operating system-specific processing routine for processing the event information received from the communication server 130 in the corresponding operating system. Should have

The communication server 130 transmits event information or image data from the source providing terminal 110 or the sharing participating terminal 160 to the counterpart so that application program sharing is performed between the source providing terminal 110 and the sharing participating terminal 160. And a transmitter / receiver 135, a user manager 140, an object manager 145, a screen synchronizer 150, and a controller 155. Of course, the communication server 130 may further include a storage unit for storing an operation program, subscriber information (eg, personal information), shared space information (eg, identifier, sharer, etc.), image data, and the like. The transmitter / receiver 135 receives image data from the source providing terminal 110 and transmits the image data to the sharing participating terminal 160, and transmits the event information received from the sharing participating terminal 160 to the source providing terminal 110. Perform the function. The transceiver 135 may be a TCP / IP processing module capable of transmitting and receiving data with a plurality of user terminals. The transmitter / receiver 135 transmits user information, shared objects, and image data to all users connected to each shared space by using a multicast communication method.

The user manager 140 maintains dynamic participation information of users accessing individual shared spaces (ie, the user of the source providing terminal 110 and the user of the sharing participating terminal 160), and transmits the information to the terminals of the corresponding users. Perform the function of sending. The participation information may be transmitted through the transmission / reception unit 135. The object manager 145 manages shared object information (for example, full image, partial image, event information, etc.) for each shared space, All the shared object information is transmitted to the shared participating terminal 160 connected while the sharing is in progress, thereby ensuring the same view for the intermediate access users. The shared object information may be stored in the object manager 145 or may be stored in a separate storage unit.

The screen synchronizer 150 controls asynchronous transmission of image data received from the source providing terminal 110. The screen synchronizer 150 creates a communication thread corresponding to the users accessing the shared space. That is, when a request for connection to the shared space, event information or partial image data is received by the communication server 130, the screen synchronizer 150 needs to be transmitted to each user accessing the shared space. And / or retrieve image information. Each control information and / or image information includes a communication flag corresponding to each user. For example, if a communication flag of specific image information is 0 for any user, the transmission is started, and when the transmission is completed, the communication flag is set to 1. Accordingly, the screen synchronizer 150 may determine control information and / or image information that has been completed or need to be transmitted using the communication flag, and each user may be provided with only the latest information. Using this method, a user with a high communication speed can receive the latest information faster than other users, and can guarantee a quality of service (QoS) for all users. The controller 155 controls the user manager 140, the object manager 145, and the screen synchronizer 150 so that the communication server 130 may perform the above functions.

2 is a flowchart illustrating a process of performing an object control method for sharing resources between heterogeneous devices according to an embodiment of the present invention, and FIG. 3A is a diagram illustrating event information transmission from a sharing participating terminal according to an embodiment of the present invention. FIG. 3B is a diagram illustrating screen area classification according to an exemplary embodiment of the present invention, and FIG. 3C is a diagram illustrating an event information transmission packet according to an exemplary embodiment of the present invention. 4A illustrates a program sharing procedure in a source providing terminal according to an exemplary embodiment of the present invention, and FIG. 4B illustrates an image data transmission packet according to an exemplary embodiment of the present invention.

Referring to FIG. 2, in step 205, the source providing terminal 110 shares the resources (for example, an application program, an electronic file, or the like, which will be described below with respect to an application program). Sends a request for creating a shared space and a raw screen image through a communication network. The request for creating a shared space may be similar to the process of creating a chat room in a chat service system. The raw screen image may be a raw screen that is displayed when a specific application is executed, or the corresponding application (for example, Microsoft Excel) may have a specific electronic file ( For example, this may be an execution screen when share.xls) is executed. The shared space creation request and the raw screen image may be transmitted sequentially or at different times. The communication server 130 generates a shared space corresponding to the shared space creation request from the source providing terminal 110 in step 210 and stores the corresponding raw screen image. Of course, if the shared participant information (eg, ID) is included in the shared space creation request received from the source providing terminal 110, the shared participant may be sent to the shared participant to induce access to the shared space. have.

In step 215, the sharing participating terminal 160 transmits shared space selection information for accessing a specific shared space to the communication server 130 through a communication network.

The communication server 130 adds the user to the shared participation terminal 160 in the shared space corresponding to the shared space selection information in step 220, and proceeds to step 225 to store the shared participating terminal 160 connected in the shared space. Send the raw screen image.

Although not shown in FIG. 2, an authentication step for allowing the user of the source providing terminal 110 and the user of the sharing participating terminal 160 to be connected may be further included.

The sharing participating terminal 160 displays the raw screen image received from the communication server 130 on the display unit in step 230, and the flow proceeds to step 235 in which an event (for example, a mouse click or a double click is performed) on the sharing participating terminal 160. When a mouse event such as a drag and drop, a keyboard event such as a key button input, etc. occurs, the corresponding event is converted into object information and transmitted to the communication server 130.

In operation 240, the communication server 130 receives event information from the sharing participating terminal 160 and transmits the received event information to the source providing terminal 110 through the communication network.

The source providing terminal 110 receives event information from the communication server 130 in step 245, operates the application program according to the received event information, and then generates partial image data corresponding to the operation result. In operation 250, the generated partial image data is transmitted to the communication server 130 through the communication network.

In operation 255, the communication server 130 receives the partial image data from the source providing terminal 110 and transmits the received partial image to the sharing participating terminal 160 through the communication network. By storing the partial image data, the communication server 130 may transmit the current screen image to a user who accesses the shared space later so that all shared space accessors share the same screen image.

In operation 260, the sharing participation terminal 160 updates the raw screen image displayed on the display unit by using the partial image data received from the communication server 130. That is, only the screen image of the region corresponding to the event information is updated in the sharing participating terminal 160.

So far, referring to FIG. 2, in performing resource sharing between heterogeneous devices, the source providing terminal 110 performs an operation corresponding to the event information generated in the sharing participating terminal 160 to transfer the resultant image data to the sharing participating terminal 160. The method of sharing resources of the source providing terminal 110 by the sharing participating terminal 160 by transmitting them has been described. Of course, when a mouse event or a keyboard event occurs in the source providing terminal 110 or when the display screen of an application program installed and driven in the source providing terminal 110 is changed (for example, the banner advertisement window is shared while the web browser is shared). When the display state is updated), it is natural that the partial image data corresponding to the event or the change screen is automatically generated and transmitted to the sharing participating terminal 160 to enable active resource sharing by the source providing terminal 110.

3A to 3C, an event generated in the sharing participating terminal 160 is transmitted to the source providing terminal 110 through the communication server 130, and partial image data is received from the source providing terminal 110 to receive the raw screen image. Flowcharts for generating the same are illustrated.

The screen management module 115 and the object management module 120 included in the source providing terminal 110 and the sharing participating terminal 160 may include a system control module, a shared program management module, an event management module, an image management module, a communication module, It can be subdivided into thread management modules. Each module will be easily understood from the name of each module and the following description.

Referring to FIG. 3A, the sharing participation terminal 160 displays the raw screen image on the display unit in step 305. In operation 310, the sharing participation terminal 160 determines whether an event (eg, a mouse event such as a mouse click, a double click, a drag and drop, or a keyboard event such as a key button input) occurs in the raw screen image area. do. The raw screen image area may be divided into 16 areas as illustrated in FIG. 3B, and each area is divided into 16 sub-regions. Of course, the number of division areas can be changed as necessary.

If no event occurs, the process proceeds to step 315 where the sharing participation terminal 160 determines whether partial image data is received from the communication server 130. The partial image data may be, for example, partial image data provided when an event occurs in the source providing terminal 110 or a display state is changed. If the partial image data is not received, the flow proceeds to step 310. However, if partial image data is received, the flow advances to step 320 to update the raw screen image using the partial image. In this case, the partial image data received from the communication server 130 may be a partial image corresponding to one or more detailed areas shown in FIG. 3B, and when the partial image data is received, only the detailed area corresponding to the corresponding area is updated. Through this method, the amount of data transmitted and received can be minimized, and as a result, the load on the communication server 130 and the user terminal can be minimized.

However, if an event occurs, the process proceeds to step 325 where the sharing participating terminal 160 drives the thread management module. The thread management module driven in operation 330 determines whether the event has occurred or the event caused by the input of a key button according to a specific area, and generates event information in a predetermined manner. In operation 335, the generated event information is transmitted to the communication server 130 through the communication network. The configuration of packet data of event information transmitted to the communication server 130 is illustrated in FIG. 3C. Referring to FIG. 3C, the packet data of the event information includes an event type field, a coordinate value, or an ASCII value field. Of course, the header or cyclic redundancy check (CRC) may be further included.

Hereinafter, the contents described with reference to FIGS. 3A to 3C will be briefly described again.

As described above, the screen management module 115 and the object management module 120 included in the source providing terminal 110 and the sharing participating terminal 160 are a system control module, a shared program management module, an event management module, and an image management. It can be subdivided into modules, communication modules, and thread management modules. The system control module manages all events occurring in the source providing terminal 110 or the sharing participating terminal 160 to drive the thread management module and to control the shared program management module, the image management module, and the communication module. The thread management module captures the screen at the start time (in the case of the source providing terminal 110) and automatically detects the converted part. That is, the thread management module manages each area where the screen size is divided into 16, runs the threads for capturing the screen, and checks whether there is a changed area (or detail area) among the areas (or detail areas) managed by each thread. After comparison, the system controller is informed. That is, each thread automatically captures the allocated area every time or event occurs in the system and converts it into DIB (Device Independent Bitmap) format. Determine if there is a change. Compared to the detail area unit, the changed detail area is informed to the thread management module. The shared program management module is in charge of driving and handling an application program designated by a user for sharing among many application programs installed in the source providing terminal 110. The event management module hooks mouse events (eg, mouse movements, mouse clicks, double clicks, etc.) and / or keyboard events generated in the course of executing the shared program, and records each event received from the communication server 130. Emulate with a management module. The image management module transfers images (eg, a whole image, a partial image, etc.) transferred from the thread management module to the system controller to the communication module or restores and transfers partial image data received from the communication server 130. The communication module performs wired and / or wireless communication processing of the corresponding terminal, and is coupled to the communication server 130 through a communication network to transmit and receive data. The object management module 120 transmits event information corresponding to the generated event. It generates and transmits to the communication server 130 through the communication network. The packet data of the event information transmitted to the communication server 130 includes information on the event type (ie, mouse event or keyboard event) and input values corresponding to each event type (eg, mouse movement in case of a mouse event). , Mouse click (up, down), mouse double click, mouse drag drop, and the like, and in the case of a keyboard event, input key button information). In the case of a keyboard event, a kit value corresponding to each key button may be needed, and a key table for processing thereof may exist.

4A and 4B, partial image data corresponding to an event generated in the source providing terminal 110 or event information received from the sharing participating terminal 160 through the communication server 130 is generated, and the generated portion A flow chart for transmitting image data is illustrated. Referring to FIG. 4A, the source providing terminal 110 determines whether an event has occurred in the source providing terminal 110 in step 405. If no event occurs, the process proceeds to step 407.

In operation 407, the source providing terminal 110 determines whether the display state of the display unit is updated. For example, the source providing terminal 110 may recognize that the display state is updated when all or part of the display state of the executed application is updated (for example, when the banner advertisement on the web browser screen is updated). Can be. That is, it is possible to actively monitor whether or not the display information is changed without generating an event. If the display state is updated, the process proceeds to step 415; otherwise, the process proceeds to step 410.

In operation 410, the source providing terminal 110 determines whether event information is received from the communication server 130. If no event information is received, the flow proceeds to step 405. However, if an event occurs in the source providing terminal 110 or event information is received from the communication server 130, the process proceeds to step 415. Thereafter, in step 415, the source providing terminal 110 captures an area (or a detailed area) corresponding to an event or received event information or an area whose display state is updated. When an event occurs in the source providing terminal 110 or event information is received, each of the coordinate information corresponding to the event and the coordinates of the region in which the display information is updated can be identified. Can be determined whether or not to be scanned. Of course, when the event information received from the communication server 130 is intended to execute the application program provided in the source providing terminal 110, the screen corresponding to the execution result after performing the operation corresponding to the corresponding event information (I.e., specific area or detail area).

In operation 420, the source providing terminal 110 may determine whether the partial image captured in operation 415 matches the partial image (or the previous partial image) of the corresponding region of the full screen image that was originally displayed. If there is a match, it is not necessary to update the full screen image by transmitting the partial image, and the flow proceeds to step 405 without transmitting the partial image data. However, if the partial image does not match the image that was originally displayed, the flow advances to step 425 to transmit the partial image data to the communication server 130. When the partial image data is transmitted to the communication server 130, The configuration of the packet data is illustrated in FIG. 4B. Referring to FIG. 4B, the packet data includes a bitmap control flag field for indicating whether the image data corresponds to the entire screen or the image data corresponding to a specific area, an area corresponding to the image data, and / or a detail area. An area value field and a detail area value field for indicating, a rectangular coordinate field including coordinate value information corresponding to the image data, and a partial image data field (or an entire image data field) including captured image information. Thus, since the packet data is composed of a bitmap image, the packet data can be applied regardless of the operating system or the type of the user terminal.

Hereinafter, a process of controlling (or executing) an Excel program installed in the source providing terminal 110 in the shared participation terminal 160 in which the Excel program is not installed will be described with reference to the above-described process.

First, a user of a source providing terminal 110 who wants to share an installed Excel program connects to the communication server 130 using the source providing terminal 110 and establishes a shared space.

Subsequently, a user of the sharing participating terminal 160 who wants to share an Excel program installed in the source providing terminal 110 accesses the communication server 130 using the sharing participating terminal 160, and accesses the opened shared space. .

The communication server 130 stores the raw screen image data received from the source providing terminal 110 when the shared space is opened (or after opening) (ie, a bitmap image corresponding to an Excel sheet screen displayed by execution of an Excel program). Data) is transmitted to the sharing participating terminal 160 through the communication network. The sharing participating terminal 160 receives the raw screen image data received from the communication server 130 and displays it on the display unit. In this state, it is assumed that a user of the sharing participating terminal 160 wants to input data that the user of the sharing participation terminal 160 goes to column A row 1 of the Excel sheet (ie, the bitmap image corresponding to the Excel sheet).

First, a user of the sharing participating terminal 160 clicks a position of row A of column A of an Excel sheet by using a mouse. The sharing participating terminal 160 generates event information corresponding to the generated event and transmits the generated event information to the source providing terminal 110, and the source providing terminal 110 operates the Excel program using the received event information and outputs the result. The corresponding partial image data is generated and transmitted to the sharing participating terminal 160. That is, the source providing terminal 110 performs a mouse click operation on a cell of the corresponding coordinate (ie, any cell in the Excel sheet) corresponding to the mouse click and the clicked coordinate information included in the event information. The Excel program will activate the cell, and the source providing terminal 110 generates partial image data of the activated cell and transmits the partial image data to the sharing participating terminal 160. In this case, the partial image may be an image corresponding to a detailed area corresponding to the coordinate information. This is to solve the problem (for example, mouse temporary stoppage, a large amount of image data transmission and reception) that takes a lot of load on the system if the source providing terminal 110 captures and transmits the full screen image in all cases. Therefore, the screen area to be captured is divided into a plurality of parts, and each area is processed as one thread (that is, one thread is in charge of one area and a plurality of detailed areas included in the area), and only the changed part is processed. Applies a method of detecting and transmitting only the changed detailed region (or region). The screen image that needs to be captured in this way can be divided into areas and detailed areas to minimize the load on the system.

Thereafter, the display unit of the sharing participating terminal 160 is to display a screen image of a state in which the corresponding cell is activated (confirming which region corresponds to which region the image corresponds to by using coordinate information and updating only the corresponding region). Input data to be input in order (ie, a, ㅏ, ㅏ, ㅏ, ,, ㅏ), and each time data is inputted, the corresponding event information is transmitted to the source providing terminal 110, and the updated area of The partial image is transmitted to the sharing participating terminal 160 so that the user of the sharing participating terminal 160 has a feeling of using the corresponding program even though there is no corresponding program.

As described above, the present invention provides a technique for dealing with an input event having the same effect as installing an application remotely to users who do not have an application and circumventing the limitation of application sharing in a specific operating system. Multi-users are guaranteed to be consistent with views through a communication server that can manage shared screens and shared objects. It is not possible to share heterogeneous application programs by automatically converting DIB images that can be supported by all terminals and comparing captured images in areas and sub-areas included in the areas. It uses a method of automatically notifying each area of threads. This approach is designed to reduce the load on shared applications so that they do not interfere with your work.

In addition, it defines an event structure for handling input events (keyboard, mouse) in a shared application by a user who does not have a shared application, sends it to a terminal that has a shared application, interprets it, and executes the shared application for processing. I can do it. In addition, a communication server is provided for efficient sharing of multi-users, and the communication server manages image data and control data. In particular, the image data does not necessarily have to be transmitted to all users. Users who transmit the current image properly and have a slow network may be looking at the past image. This is because a slow network of one user can seriously impede the performance of all users. Therefore, communication takes a centralized structure. Of course, it can be provided in a similar form through the P2P connection.

As described above, the object control method and system for resource sharing between heterogeneous devices according to the present invention converts the minimum area image corresponding to the changed location into a DIB (Device Interface Bitmap) type image, performs compression, and then transmits the same. Enable application sharing between heterogeneous devices.

In addition, the present invention can escape from the limited working area of each user terminal that was dependent on a specific platform to perform only limited work, thereby maximizing work efficiency and reducing costs.

In addition, the present invention can perform interactive training regardless of the type of the user terminal during the remote training.

In addition, according to the present invention, a remote patient and a doctor who are located at a remote location may perform remote medical care using a user terminal under a separate and different operating system.

In addition, the present invention enables a dynamic presentation through application sharing in the case of teleconferencing using video conferencing.

In addition, the present invention makes it possible to diagnose and repair a failure or error of a specific user terminal regardless of the type of the administrator terminal at a remote location.

In addition, the present invention can be used or tested in advance through the application sharing in the test bed in which the application is installed, without having to install the application on its user terminal for testing the developed specific application.

Although the above has been described with reference to a preferred embodiment of the present invention, those skilled in the art to which the present invention pertains without departing from the spirit and scope of the present invention as set forth in the claims below It will be appreciated that modifications and variations can be made.

1 is a diagram schematically illustrating an overall system configuration for performing an object control method for resource sharing between heterogeneous devices according to a preferred embodiment of the present invention.

2 is a flowchart illustrating a process of performing an object control method for resource sharing between heterogeneous devices according to an exemplary embodiment of the present invention.

3A is a flowchart illustrating an event information transmission procedure in a sharing participation terminal according to an embodiment of the present invention.

3B is a view illustrating screen area division according to an exemplary embodiment of the present invention.

3C illustrates an event information transport packet according to an embodiment of the present invention.

4A illustrates a program sharing procedure in a source providing terminal according to an exemplary embodiment of the present invention.

4B illustrates an image data transmission packet according to a preferred embodiment of the present invention.

<Description of the symbols for the main parts of the drawings>

110: source providing terminal

115a, 115b: Screen Management Module

120a, 120b: Object Management Module

130: communication server

135: transceiver

140: user management unit

145: object management unit

150: screen synchronization unit

155 control unit

160: sharing participating terminals

Claims (13)

A method for sharing an application installed in a source providing terminal through a communication network in a sharing participating terminal, wherein the sharing participating terminal is coupled to the source providing terminal through a communication network via a communication server and does not include the application program. No, Receiving a raw screen image corresponding to the application program from a communication server and displaying the raw screen image on the display unit, wherein the raw screen image is a work screen image according to execution of the application program; Generating event information of a predetermined form corresponding to the event when an event occurrence is detected, wherein the event is at least one of a mouse event and a keyboard event; Transmitting the event information to the communication server through a communication network; Receiving partial image information corresponding to the event information from the communication server, wherein the partial image information includes at least partial image data and image coordinate information corresponding to the partial image data; And updating an area corresponding to the generated event in the raw screen image by using the partial image data and the image coordinate information. Including, The source providing terminal receives the event information from the communication server, executes the application program using the event information, captures an execution result screen to generate the partial image information, and converts the partial image information into a communication network. Transmitting to said communication server via Application program sharing method using a communication network characterized in that. The method of claim 1, When the mouse event occurs, generating the event information of the predetermined form Detecting the occurrence of the mouse event; Generating a mouse pointer position corresponding to the mouse event as mouse coordinate information; And Generating packet data corresponding to the mouse event by using a mouse event indicator and the mouse coordinate information Application program sharing method using a communication network comprising a. The method of claim 1, When the keyboard event occurs, generating the event information of the predetermined form Detecting the occurrence of the keyboard event; Extracting an input key button value corresponding to the keyboard event; And generating packet data corresponding to the keyboard event by using a keyboard event indicator and the input key button value. The method of claim 1, The sharing participating terminal and the source providing terminal are heterogeneous terminals, respectively. Application program sharing method using a communication network characterized in that. The method according to claim 2 or 3, The packet data includes an event type field indicating the mouse event or the keyboard event, and a data field including mouse coordinate information corresponding to the mouse event or an input key button value corresponding to the keyboard event. Application program sharing method using a communication network characterized in that. The method of claim 1, The updating of the region corresponding to the generated event in the raw screen image may include: Determining an area corresponding to the image coordinate information in the raw screen image; And Replacing the determined region with the partial image data Application program sharing method using a communication network comprising a. The method of claim 1 The mouse event includes at least one of mouse movement, mouse click, double click, drag and drop. Application program sharing method using a communication network characterized in that. In the method for providing a screen image performed by the source providing terminal to share an application installed in the source providing terminal with the sharing participating terminal through a communication network, wherein the communication server is connected to the shared participating terminal and the source providing terminal through the communication network. Coupled, the sharing involvement terminal not including the application program; executing the application program; Generating a raw work screen image according to execution of the application program and transmitting the raw work screen image to the communication server through a communication network; Receiving event information from the communication server, wherein the event information includes an event type field indicating a mouse event or a keyboard event, mouse coordinate information corresponding to the mouse event, or an input key button value corresponding to the keyboard event. Including a data field comprising; Operating the application program according to the event information; Generating an update work screen image corresponding to an operation result of the application program; Generating a partial image corresponding to a different area of the original work screen image and the update work screen image; Generating partial image information using the partial image, wherein the partial image information includes at least partial image data and image coordinate information corresponding to the partial image data; And transmitting the partial image information to the communication server through a communication network. Including, The sharing participation terminal receives and displays the raw work screen image, and when an occurrence of an event in any region of the raw work screen image is detected, generates and transmits the event information corresponding to the generated event to the communication server. And updating the area corresponding to the occurrence of the event by using the partial image information received from the communication server. Application program sharing method using a communication network characterized in that. The method of claim 8, Generating a partial image corresponding to a different region of the raw work screen image and the update work screen image comprises extracting a partial image of a region corresponding to the mouse coordinate information. Application program sharing method using a communication network characterized in that. The method of claim 8, When the display screen of the source providing terminal is divided into a plurality of first regions, each of the first regions is divided into a plurality of second regions, and each of the first regions is managed by an independent change management unit. The display screen is a full screen or an active window in which the application is executed; The generating of the update work screen image and the generating of the partial image may include: Capturing a region image of a first area managed by the independent change manager; determining whether the change image is changed from an image of a corresponding area of the raw work screen image; Requesting generation of a partial image by a change manager for managing a corresponding area when changed; And Generating the changed region image as a partial image Application program sharing method using a communication network comprising a. The method of claim 8, Operating the application program according to the generated event when an event occurrence is detected; Application program sharing method using a communication network characterized in that it further comprises. In a recording medium that is tangibly embodied in a program that can be executed in a shared participant terminal to perform an application program sharing method using a communication network, wherein the shared participant terminal Does not include the application, and is coupled with a source providing terminal in which the application is installed via a communication server; Receiving a raw screen image corresponding to the application program from a communication server and displaying the raw screen image on the display unit, wherein the raw screen image is a work screen image according to execution of the application program; Generating event information of a predetermined form corresponding to the event when an event occurrence is detected, wherein the event is at least one of a mouse event and a keyboard event; Transmitting the event information to the communication server through a communication network; Receiving partial image information corresponding to the event information from the communication server, wherein the partial image information includes at least partial image data and image coordinate information corresponding to the partial image data; And Updating an area corresponding to the generated event in the raw screen image by using the partial image data and the image coordinate information Run the The source providing terminal receives the event information from the communication server, executes the application program using the event information, captures an execution result screen to generate the partial image information, and converts the partial image information into a communication network. Recording medium recording a program, characterized in that for transmitting to the communication server through. In the recording medium that can be read by the source providing terminal, the program of instructions that can be executed in the source providing terminal to implement the application program sharing method using a communication network is tangibly implemented, wherein the source providing terminal is The application is installed, and is coupled with a shared participating terminal where the application is not installed via a communication server; Generating a raw work screen image according to execution of the application program and transmitting the raw work screen image to the communication server through a communication network; Receiving event information from the communication server, wherein the event information includes an event type field indicating a mouse event or a keyboard event, mouse coordinate information corresponding to the mouse event, or an input key button value corresponding to the keyboard event. Including a data field comprising; Operating the application program according to the event information; Generating an update work screen image corresponding to an operation result of the application program; Generating a partial image corresponding to a different area of the original work screen image and the update work screen image; Generating partial image information using the partial image, wherein the partial image information includes at least partial image data and image coordinate information corresponding to the partial image data; And Transmitting the partial image information to the communication server through a communication network. Run the The sharing participation terminal receives and displays the raw work screen image, and when an occurrence of an event in any region of the raw work screen image is detected, generates and transmits the event information corresponding to the generated event to the communication server. And updating the area corresponding to the occurrence of the event by using the partial image information received from the communication server. Recording medium recording a program characterized in that.