JP5676921B2 - Server apparatus, control method thereof, and system - Google Patents

Description

  The present invention relates to a technology related to a screen sharing system in which a plurality of information processing apparatuses share a screen.

In recent years, screen sharing systems that share a screen among a plurality of information processing apparatuses have been used in various scenes. This screen sharing system is used in the following cases.
-When remotely operating an information processing device in a remote location-When displaying the screen of an information processing device such as a laptop computer brought into the conference room on a large display installed in the conference room via a network In addition to screen sharing among a plurality of information processing apparatuses, there are also cases where a user operates a shared screen of a client information processing apparatus to transfer a file between memory spaces of different information processing apparatuses. Here, the shared screen is a screen that virtually displays the screen of another information processing apparatus by the screen sharing function. For example, the display screen of the server information processing apparatus is displayed on the shared screen of the client information processing apparatus by the screen sharing function.

  In the screen sharing system described above, only files can be transferred by operations on the shared screen. However, the object data of the application can also be transferred by transferring the image data displayed in the application to the memory space of the information processing apparatus different from the information processing apparatus in which the application is activated.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses an apparatus that avoids an input collision by performing exclusive control on an information processing apparatus serving as a server when there are inputs from a plurality of clients.

JP 2004-5430 A

  However, in the above-described screen sharing system, if a drag operation is performed from the client while the pointing device is operated on the server side, the operation on the server side and the operation on the client collide, and a desired operation cannot be performed. There is a problem.

  Further, according to the exclusive control described in Patent Document 1, although operation collision can be avoided, there is a problem that a desired operation cannot be performed.

  Due to this problem, the user cannot use the image data displayed in the application, for example, by transferring the image data displayed in the application to a memory space of an information processing apparatus different from the information processing apparatus in which the application is activated.

  An object of the present invention is to enable data transfer from another information processing apparatus to a shared area even when a user is operating on the shared area in the shared server.

The present invention is an information processing apparatus connected to another information processing apparatus for executing a first process on a first image displayed on a first display means controlled by the information processing apparatus. A first unit that inputs a first instruction and a second image displayed on a second display unit that is controlled by the other information processing apparatus. Acquisition means for acquiring, from the other information processing apparatus, a second instruction for causing the second process to be executed on the first image displayed on the first display means; and , A transmission unit that transmits an image obtained by performing the second process on the first image to the other information processing apparatus, and the first image displayed on the first display unit. In the first period during which the first process is being executed, the acquisition means performs the second process. When the display is acquired, the first image is displayed on the first display unit in the first period while the first process is performed on the first image and the second process is not performed. Display control means for causing the transmission means to add a fifth image obtained by adding a fourth image corresponding to the second instruction acquired by the acquisition means to the third image. It transmits to the said other information processing apparatus during the period .

  ADVANTAGE OF THE INVENTION According to this invention, even if it is a case where the user is operating with respect to a shared area in a shared server, the data transfer from another information processing apparatus to a shared area can be performed.

It is a figure which shows the network structure of the screen sharing system by this embodiment. It is the schematic which shows the hardware constitutions of the client information processing apparatus and server information processing apparatus which comprise the screen sharing system by this embodiment. It is the schematic which shows the function structure of the information processing apparatus by this embodiment. 3 is a flowchart showing data transfer processing in a client information processing apparatus 101. FIG. 3 is a diagram illustrating a screen display example of a client information processing apparatus 101 and a server information processing apparatus 102. It is a figure which shows the operation screen management table which the screen sharing application program 222 manages. 3 is a flowchart showing data reception processing in a server information processing apparatus 102. 4 is a flowchart showing transfer data conversion processing in the client information processing apparatus 101.

(First embodiment)
The best mode for carrying out the invention will be described below in detail with reference to the drawings.

  Here, a screen sharing system will be described as an example as an example in which a plurality of users operate the shared area of the shared server.

  FIG. 1 is a diagram showing a network configuration of the screen sharing system according to the present embodiment. As shown in FIG. 1, the screen sharing system is a system in which a client information processing apparatus 101, a server information processing apparatus 102, and a server information processing apparatus 103 are connected to a network 111, each having a communication function. In the present embodiment, a system sharing the screen of the server information processing apparatus 102 will be described.

  FIG. 2 is a schematic diagram illustrating a hardware configuration of the client information processing apparatus and the server information processing apparatus that configure the screen sharing system according to the present embodiment. Note that both the client information processing apparatus and the server information processing apparatus are realized by a computer system such as a personal computer, and therefore will be described as the information processing apparatus 201 in FIG.

  The information processing apparatus 201 includes a CPU 211 that controls the entire apparatus, a ROM 212 that stores a boot program and fixed data, and a RAM 213 that functions as a main memory and provides a work area for the CPU 211. Furthermore, the information processing apparatus 201 has the following configuration.

  An HDD 214 is a hard disk device, and stores an operating system (OS) 221, a screen sharing application program 222, a document editing application program 223, and the like. A VRAM 215 is a memory for developing image data to be displayed, and displays the image data on the display 231 which is an example of a display device by expanding the image data.

  The keyboard 232, mouse 233, and digitizer 234 as input devices are connected to the keyboard controller 216, mouse controller 217, and digitizer controller 218, respectively, and the input is detected as an interrupt signal by the CPU 211. A network interface (I / F) 219 is also provided and can be connected to the network 111.

  In the above configuration, the screen sharing application program 222 and the document editing application program 223 are activated in response to specific instruction events from the keyboard 232, mouse 233, and digitizer 234. At the time of activation, the screen sharing application program 222 and the document editing application program 223 are first loaded into the RAM 213 and executed by the CPU 211.

  The screen sharing application program 222 described above may be linked depending on the OS 221. The screen sharing application program 222 realizes a screen sharing function described later.

  FIG. 3 is a schematic diagram illustrating a functional configuration of the information processing apparatus according to the present embodiment. In FIG. 3, the information processing apparatus 301 corresponds to the information processing apparatus 201 shown in FIG. The display device 302 corresponds to the display 231 shown in FIG. The input device 303 corresponds to the keyboard 232, mouse 233, and digitizer 234 shown in FIG.

  The information processing apparatus 301 includes a communication unit 311, a data control unit 312, a data storage unit 313, an output unit 314, and an input unit 315. The communication unit 311 is connected to the network 111 and performs information communication control processing for exchanging data with other information processing apparatuses. The data control unit 312 controls processing such as screen sharing and data transfer. The data storage unit 313 stores data exchanged through screen sharing or data transfer.

  The output unit 314 outputs display screen information to a projector, a liquid crystal display, a rear projection display, or the like. The input unit 315 inputs information detected by the input device 303 and transmits the information to the data control unit 312.

  Next, data transfer in the screen sharing system according to the present embodiment will be described.

  First, data transfer in the client information processing apparatus 101 will be described with reference to FIG. An example of data transfer by a drag operation (drag and drop) on the shared screen and the local screen of the client information processing apparatus 101 will be described.

  Here, the local screen is a screen displaying the local environment without using the screen sharing function. In addition to the drag operation, it goes without saying that a case where data transfer is performed by a copy instruction & paste instruction or transfer instruction by a clipboard on the shared screen is also included. The copy instruction & paste instruction and the transfer instruction for the shared screen are as follows. For example, when an object included in a file or application is selected and instructed using a displayed menu, or an object included in the file or application is instructed by double-clicking.

  FIG. 4 is a flowchart showing data transfer processing in the client information processing apparatus 101. First, a screen sharing function is started between the server information processing apparatuses 102 and 103 and the client information processing apparatus 101, and the client information processing apparatus 101 shares a screen with the server information processing apparatus 102. A display example of the client information processing apparatus 101 at this time is shown in FIG.

  In FIG. 5A, reference numeral 501 is a display screen example of the client information processing apparatus 101, and 502 is a display screen example of the server information processing apparatus 102. Here, the screen 501 of the client information processing apparatus 101 will be described. Reference numeral 511 denotes a display example of the document editing application program 223 activated by the client information processing apparatus 101.

  Reference numeral 512 denotes a display example of the screen sharing application program 222 activated on the client information processing apparatus 101 (the screen 502 of the server information processing apparatus 102 is displayed on the client information processing apparatus 101 by the screen sharing function).

  In step S401, the data control unit 312 of the client information processing apparatus 101 acquires coordinate information input from the input device 303 such as the mouse 233 or the digitizer 234 via the input unit 315, and stores the data in the client information processing apparatus 101. Store in the unit 313. Hereinafter, the processing in FIG. 4 is performed by the client information processing apparatus 101. The data control unit 312 and the data storage unit 313 in each step are the data control unit 312 and the data storage unit 313 of the client information processing apparatus 101, respectively. It is.

  Next, in step S <b> 402, the data control unit 312 determines the information processing device An to be operated based on the coordinate information stored in the data storage unit 313. An represents the current operation target device.

  FIG. 6 is a diagram showing an operation screen management table managed by the screen sharing application program 222. This operation screen management table is stored in the data storage unit 313.

  In step S402, an information processing apparatus to be operated is determined from the operation screen management table based on the coordinate information stored in the data storage unit 313 in step S401. At this time, the data control unit 312 determines that “the operation target is the client information processing apparatus 101” for an area not included in the operation screen management table.

  Next, in step S403, the data control unit 312 determines whether or not the operation target information processing device An determined in step S402 is the same as the information processing device Ap of the operation target device flag. Ap represents an operation target device flag, which is set in step S406 to be described later and stored in the data storage unit 313.

  If the result of determination in step S403 is the same, the process proceeds to step S407, and if different, the process proceeds to step S404. In step S404, the data control unit 312 requests transfer data from the information processing device set in the information processing device Ap of the operation target device flag stored in the data storage unit 313.

  Here, the transfer data literally indicates “data to be transferred”. The transfer data includes file data, image data included in an arbitrary application program, moving image data, text data, object data, and the like.

  If the information processing device Ap of the operation target device flag is local in step S <b> 404 described above, the data control unit 312 acquires the transfer data stored in the data storage unit 313.

  In step S405, the data control unit 312 transmits the transfer data to the information processing apparatus An that is the operation target. If the information processing device An to be operated is local in step S405, the data control unit 312 of the client information processing device 101 expands the data acquired in step S404 and stores it in the data storage unit 313 as transfer data. Remember.

  Through steps S404 and S405 described above, the transfer data is transferred from the memory space of the information processing device An to the memory space of the information processing device Ap of the operation target device flag.

  Next, in step S <b> 406, the data control unit 312 sets the operation target information processing device An to the information processing device Ap of the operation target device flag, and stores it in the data storage unit 313.

  In step S407, the data control unit 312 determines whether the operation target information processing device An stored in the data storage unit 313 is local (in this case, the client information processing device 101). . If the operation target information processing apparatus An is local, the process proceeds to step S409. If the operation target information processing apparatus An is not local, the process proceeds to step S408.

  In step S408, the data control unit 312 transmits the input information stored in the data storage unit 313 in step S401 to the operation target information processing device An, and the process returns to step S401.

  On the other hand, in step S409, the data control unit 312 continues the local operation. Next, in step S410, the data control unit 312 determines whether or not the user ends the operation. Here, when the operation is finished, this process is finished. If the operation is not terminated, the process returns to step S401.

  If the user performs a drop operation or a paste operation after requesting and transmitting the transfer data in steps S404 and S405, the transfer data is transferred to the application or file system to be operated. That is, when the user performs a drop operation or a paste operation on the display screen of the screen sharing application program 222, the transfer data is passed to any application or file system running on the server information processing apparatus. Here, the display screen is obtained by displaying the display screen of the server information processing apparatus on the client information processing apparatus using the screen sharing function.

  With the above-described processing, the data transfer indicated by the arrow “A” shown in FIG. Here, the arrow “A” is data transfer from the memory space of the client information processing apparatus 101 to the memory space of the server information processing apparatus 103. An arrow “B” indicates data transfer within the local area of the server information processing apparatus 102.

  In this way, at least part of the screen information of the server is transferred to the client via the network, and the transferred screen information is displayed as a shared area on the client screen. Then, the input event information generated in the shared area by the client is transferred to the server, and the transferred input event information is reproduced as the input event information of the server.

  Therefore, in a screen sharing system in which a server remotely operates from a client, data can be transferred between memory spaces of different information processing apparatuses when a user operates the shared screen on the client information processing apparatus.

  While the server information processing apparatus 102 is performing drawing on a paint application or dragging a file, a file drag operation from the client information processing apparatus 101 to the server information processing apparatus may be performed. is there. In this case, in order to avoid the collision of the mouse operation, the operation of the client information processing apparatus 101 may be prohibited, or the drag operation may fail due to the collision of the mouse operation.

  Hereinafter, processing of the server information processing apparatus 102 in the case where such a collision of mouse operation drag operations has occurred will be described.

  FIG. 7 is a flowchart showing data reception processing in the server information processing apparatus 102. First, in step S <b> 1001, the data control unit 312 of the server information processing apparatus 102 receives data transmitted from the client information processing apparatus 101 and stores it in the data storage unit 313. This data reception process is a process corresponding to the data transmission to the server information processing apparatus 102 performed by the client information processing apparatus 101 in steps S404, S405, and S408 shown in FIG. Hereinafter, the processing in FIG. 7 is performed by the server information processing apparatus 102, and the data control unit 312 and the data storage unit 313 in each step are the server information processing apparatus 102 data control unit 312 and the data storage unit 313. is there.

  In step S <b> 1002, at the start of a drag operation in the shared area (shared screen) from the client information processing apparatus 101, it is determined whether another apparatus is performing a drag operation in the shared area. If a drag operation is being performed by another device, the D flag in the data storage unit 313 is set in step S1003. Here, the other apparatus represents an apparatus other than the client information processing apparatus 101 received in step S1001 and includes the server information processing apparatus 102. The D flag is a flag indicating whether a drag operation is being performed by another device.

  This is because, for example, when an operation of the arrow “A” on the screen 501 of the client information processing apparatus 101 in FIG. 5A is performed, the server information processing apparatus 102 performs a drag operation of the arrow “B”. If so, the D flag is set.

  If it is determined in step 1002 that another device is not performing a drag operation in the shared area, the process advances to step S1004.

  In step S1004, the data control unit 312 analyzes the data received in step S1001 and determines whether reception of transfer data is requested. As a result of the determination, if reception of transfer data is requested, the process proceeds to step S1005, and if reception of transfer data is not requested, the process proceeds to step S1010.

  In step S <b> 1005, the transfer data is received from the client information processing apparatus 101 and transferred to the data control unit 312 of the server information processing apparatus 102. Processing in the client information processing apparatus 101 at this time will be described later with reference to FIG.

  In step S1006, the data control unit 312 converts the transfer data received in step S1005 into a state that can be used by an application or the like, and stores the converted data in the data storage unit 313.

  In step S1007, the D flag in the data storage unit 313 is confirmed. If a drag operation is being performed by another device, the process proceeds to step S1008. In step S1008, processing target data such as a file or object data at the start of dragging, and current mouse information such as absolute coordinates and button information are stored in the data storage unit 313. In step S1009, the D2 flag in the data storage unit 313 is set, and the process returns to step S1001.

  By setting the D2 flag, it is used for determination of a drag operation in another device such as an input from another client information processing device or an input from the console of the server information processing device 102.

  If it is determined in step S1007 that the drag operation is not being performed by another device, in step S1010, the shared information area drag operation is started by the client information processing apparatus 101, and the process returns to step S1001.

  On the other hand, in step S1011, the data control unit 312 determines whether the data stored in the data storage unit 313 in step S1004 is a drag process. As a result of the determination, if the data is a drag process, the process proceeds to step S1012. If the data is not a drag process, the process proceeds to step S1015.

  In step S1012, the D flag in the data storage unit 313 is confirmed. If the drag operation is being performed by another device, the process proceeds to step S1013. In step S1013, the current absolute coordinates and mouse information such as button information are stored in the data storage unit 313, and the process returns to step S1001. If the coordinate information is relative coordinate data, the current absolute coordinate is calculated.

  If it is determined in step S1012 that the drag operation is not being performed by another device, in step S1014, the shared information area is dragged by the client information processing device 101, and the process returns to step S1001.

  On the other hand, in step S1015, the data control unit 312 determines whether the data stored in the data storage unit 313 in step S1001 is a drop process. As a result of the determination, if the data is a drop process, the process proceeds to step S1016. If the data is not a drop process, the process proceeds to step S1019.

  In step S1016, the D flag in the data storage unit 313 is confirmed. If a drag operation is being performed by another device, the process proceeds to step S1017. In step S1017, the data control unit 312 uses the data stored in the data storage unit 313 in step S1008 and the data usable by the application and the mouse information such as the current absolute coordinates and button information as drop data. Save to 313. If the coordinate information is relative coordinate data, the current absolute coordinate is calculated.

  For example, since the operation of the arrow “A” of the client information processing apparatus 101 on the screen 501 of the client information processing apparatus 101 in FIG. 5A has ended, the client information processing apparatus 101 can start the next operation. However, since the drag operation of the arrow “B” in the server information processing apparatus 102 is performed on the screen 502 of the server information processing apparatus 102, the drop operation of the arrow “A” is not executed in the server information processing apparatus 102.

  At this time, the server information processing apparatus 102 creates an image that emulates the operation of the client information processing apparatus 101 from the drag and drop information received from the client information processing apparatus 101, and creates an image to be transmitted to the client information processing apparatus 101. to add. Thus, the screen is displayed on the screen 501 of the client information processing apparatus 101 in FIG. 5A as if the client information processing apparatus 101 is being operated. However, it is not actually executed like the screen 502 of the client information processing apparatus 101 in FIG.

  If it is determined in step S1016 that the drag operation is not being performed by another device, the process proceeds to step S1018. In step S1018, the data control unit 312 drops data that can be used by the application or the like stored in the data storage unit 313 in step S1008 as drop data at the current mouse pointer position, and returns to step S1001.

  In step S1019, the data control unit 312 performs a local operation (server information processing apparatus 102) using the data stored in the data storage unit 313 in step S1001. For example, the local mouse cursor is moved based on the data stored in the data storage unit 313 in step S1001.

  Next, in step S1020, the data control unit 312 determines whether to end the operation. If the operation is to be terminated, the process proceeds to step S1021, and if the operation is not terminated, the process returns to step S1001.

  In step S1021, the D flag in the data storage unit 313 is confirmed. If a drag operation is being performed by another device, the process proceeds to step S1022. If it is determined in step S1021 that the drag operation is not being performed by another device, the process ends.

  In step S22, the process waits until the drag operation is completed on the other device, and proceeds to step S1023 when the drag operation is completed on the other device.

  In step S1023, data that can be used by an application or the like saved in the data storage unit 313 in step S1017 is dropped on the current absolute coordinate position saved in the data storage unit 313 in step S1017, and the process proceeds to step S1024. .

  For example, in the screen 501 of the client information processing apparatus 101 in FIG. 5C, the operation of the arrow “A” of the client information processing apparatus 101 that has already been completed is executed after the drag operation in the server information processing apparatus 102 is completed. The After the drag operation on the server information processing apparatus 102 ends, the operation of the arrow “A” of the client information processing apparatus 101 is executed, and the object 551 is dropped on the screen 502 of the server information processing apparatus 102.

  In step S1024, the D flag and the D2 flag are cleared, and the process ends.

  FIG. 8 is a flowchart showing transfer data conversion processing in the client information processing apparatus 101. Here, as shown by an arrow “A” in FIG. 5B, data transfer from the client information processing apparatus 101 to the server information processing apparatus 102 will be described as an example. In FIG. 5B, reference numeral 531 denotes a cursor of the mouse 233 of the client information processing apparatus 101. Here, the information processing apparatus set in the operation target apparatus flag Ap is the server information processing apparatus 102, and the operation target information processing apparatus An determined by the data control unit 312 of the client information processing apparatus 101 is local (that is, client information). Processing device 101 itself). Reference numeral 532 denotes a cursor of the mouse 233 of the server information processing apparatus 102, which is present at the position shown in FIG.

  First, in step S801, the data control unit 312 of the client information processing apparatus 101 generates a window 541 under the cursor 531 illustrated in FIG. Hereinafter, the processing in FIG. 8 is performed by the client information processing apparatus 101, and the data control unit 312 and the data storage unit 313 in each step are the data control unit 312 and the data storage unit 313 of the client information processing apparatus 101, respectively. It is. Next, in step S802, the data control unit 312 generates a mouse event. Next, in step S803, the data control unit 312 processes the mouse event generated in step S802 in the window 541. Through this process, the transfer data stored in the data storage unit 313 is acquired. In step S804, the data control unit 312 converts the transfer data acquired in step S803 into a transferable state.

  Next, in step S805, the transfer data converted in step S804 is transmitted to the server information processing apparatus 102, and this process ends.

  As described above, there is a shared area for displaying part of the server's screen information on the client's screen, the input event information generated for the shared area is played back by the server, and the server can be operated remotely. I do. At this time, when the client tries to drag a file or the like on the client terminal to the shared area by dragging, it is determined whether another user is dragging on the server. When the drag operation is being performed, all the drag processes of the client are received and stored, and after the drag process of another user is completed, the stored drag process is executed.

Needless to say, the network configuration of the screen sharing system according to the present invention is not limited to the configuration described above, but may be the following configuration.
・ Server information processing device: multiple units, client information processing device: 1 unit ・ Server information processing device: 1 unit, client information processing device: 1 unit ・ Server information processing unit: 1 unit, client information processing device: multiple units ・ Server Information processing apparatus: plural units, client information processing apparatus: plural units Further, the server information processing apparatus may share a server application as a Web server.

Furthermore, not only screen sharing but also operations in the shared area may be used.
In the present embodiment, image data, moving image data, text data, and object data included in a file or an arbitrary application are taken up as data transferred between memory spaces of different information processing apparatuses. However, it goes without saying that the present invention can also handle other data.

  Even if the present invention is applied to a system composed of a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), it is applied to an apparatus (for example, a copier, a facsimile machine, etc.) composed of a single device. It may be applied.

  In addition, a recording medium in which a program code of software for realizing the functions of the above-described embodiments is recorded is supplied to the system or apparatus, and the computer (CPU or MPU) of the system or apparatus stores the program code stored in the recording medium. Read and execute. It goes without saying that the object of the present invention can also be achieved by this.

  In this case, the program code itself read from the computer-readable recording medium realizes the functions of the above-described embodiments, and the recording medium storing the program code constitutes the present invention.

  As a recording medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  In addition, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also the following cases are included. That is, based on the instruction of the program code, an OS (operating system) running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing. .

  Further, the program code read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. After that, based on the instruction of the program code, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing, and the function of the above-described embodiment is realized by the processing. Needless to say.

Claims (11)

An information processing apparatus connected to another information processing apparatus,
An input means for inputting a first instruction for executing a first process on the first image displayed on the first display means controlled by the information processing apparatus;
A second process corresponding to the process executed by the other information processing apparatus on the second image displayed on the second display means controlled by the other information processing apparatus is applied to the first display means. Obtaining means for obtaining a second instruction to be executed on the displayed first image from the other information processing apparatus;
Transmitting means for transmitting an image obtained by performing the first process and the second process on the first image to the other information processing apparatus;
When the acquisition unit acquires the second instruction during a first period in which the first process is performed on the first image displayed on the first display unit, the first instruction Display control means for causing the first display means to display a third image in which the first processing is performed on the image and the second processing is not performed, in a first period;
The transmission unit adds a fifth image obtained by adding a fourth image corresponding to the second instruction acquired by the acquisition unit to the third image, and the other information processing apparatus in the first period. An information processing apparatus that transmits to the information processing apparatus. Wherein the display control unit, said other instructions by said first image drag operation on the been conducted have such have field coupling from the apparatus other than the information processing apparatus, the drag operation in response to an instruction from the other information processing apparatus The information processing apparatus according to claim 1, wherein: A method for controlling an information processing apparatus connected to another information processing apparatus,
An input step of inputting a first instruction for executing a first process on the first image displayed on the first display means controlled by the information processing apparatus;
A second process corresponding to the process executed by the other information processing apparatus on the second image displayed on the second display means controlled by the other information processing apparatus is applied to the first display means. An acquisition step of acquiring, from the other information processing apparatus, a second instruction to be executed on the displayed first image;
A transmission step of transmitting, to the other information processing apparatus, an image obtained by performing the first process and the second process on the first image;
When the second instruction is acquired during a first period in which the first process is performed on the first image displayed on the first display unit, the first image is displayed. A display control step of causing the first display means to display a third image that executes the first process and does not execute the second process in a first period;
In the transmission step, a fifth image obtained by adding a fourth image corresponding to the second instruction acquired by the acquisition unit to the third image is added to the other information processing apparatus in the first period. The control method characterized by transmitting to. In the display control step, the other directed by the first image drag operation on the been conducted have such have field coupling from the apparatus other than the information processing apparatus, the drag operation in response to an instruction from the other information processing apparatus control method according to claim 3, wherein the benzalkonium to perform. To a computer connected to another information processing device,
An input procedure for inputting a first instruction for executing a first process on the first image displayed on the first display means controlled by the computer;
A second process corresponding to the process executed by the other information processing apparatus on the second image displayed on the second display means controlled by the other information processing apparatus is applied to the first display means. An acquisition procedure for acquiring from the other information processing apparatus a second instruction to be executed on the displayed first image;
A transmission procedure for transmitting, to the other information processing apparatus, an image obtained by performing the first process and the second process on the first image;
When the second instruction is acquired during a first period in which the first process is performed on the first image displayed on the first display unit, the first image is displayed. A display control procedure for executing the first process and displaying the third image on which the second process is not performed on the first display unit in a first period;
In the transmission procedure, the fifth image obtained by adding the fourth image according to the second instruction to the third image is transmitted to the other information processing apparatus in the first period. A program to make it run. An information processing system having a first information processing apparatus and a second information processing apparatus connected to the first information processing apparatus,
First input means for inputting, to the first information processing apparatus, a first instruction for executing processing for the first image displayed on the first display means controlled by the first information processing apparatus. When,
A second instruction for executing a second process on the second image displayed on the second display means controlled by the second information processing apparatus is input to the second information processing apparatus. A second input means;
A third instruction for causing the third image corresponding to the second process to be executed on the first image displayed on the first display means from the second information processing apparatus to the first. First transmission means for transmitting to one information processing apparatus;
Obtaining means for obtaining the third instruction;
An image obtained by executing processing corresponding to the first instruction and processing corresponding to the third instruction on the first image is transferred from the first information processing apparatus to the second information processing apparatus. A second transmitting means for transmitting;
When the acquisition unit acquires the third instruction during a first period in which processing according to the first instruction is performed on the first image displayed on the first display unit, The first display unit executes a process according to the first instruction on the first image, and causes the first display unit to display a third image that has not been processed according to the third instruction. Display control means,
A fifth image obtained by adding a fourth image corresponding to the third instruction acquired by the acquisition unit in the first period to the third image is displayed in the second period in the first period. An information processing system comprising: second display control means for displaying on the means. The display control means may control, when a drag operation on the first image by an instruction from a device other than pre-Symbol another information processing apparatus is not performed, the drag operation in response to an instruction from the other information processing apparatus The system according to claim 6 , wherein the system is executed. The other information processing apparatus, according to claim, characterized in that a third transmission means for transmitting the fourth image corresponding to the third instruction and the third instruction to the information processing apparatus 6 Alternatively, the information processing system according to claim 7 . When an instruction to execute the second display control in the shared area is acquired from the second apparatus while the first display control according to the instruction from the first apparatus is being executed in the shared area on the screen of the display means. Control means for executing the second display control in the shared area in the second period after executing the first display control in the first period;
An image that is displayed in the shared area and in which the first display control is performed and an image corresponding to the second display control is added to the image that is not subjected to the second display control Transmitting means for transmitting to the second device during a first period;
An information processing apparatus comprising: When an instruction to execute the second display control in the shared area is acquired from the second apparatus while the first display control according to the instruction from the first apparatus is being executed in the shared area on the screen of the display means. A control step of executing the second display control in the shared area in a second period after executing the first display control in the first period;
An image that is displayed in the shared area and in which the first display control is performed and an image corresponding to the second display control is added to the image that is not subjected to the second display control Transmitting to the second device during a first period; and
A method for controlling an information processing apparatus, comprising: On the computer,
When an instruction to execute the second display control in the shared area is acquired from the second apparatus while the first display control according to the instruction from the first apparatus is being executed in the shared area on the screen of the display means. A control procedure for executing the second display control in the shared area in the second period after executing the first display control in the first period;
An image that is displayed in the shared area and in which the first display control is performed and an image corresponding to the second display control is added to the image that is not subjected to the second display control A transmission procedure for transmitting to the second device during a first period;
A program for running

Images