JP6554041B2 - Information processing method and information processing apparatus - Google Patents

Description

  The present technology relates to an information processing method and an information processing system for sharing and observing the same image among users of a plurality of information processing apparatuses.

  In order to support a remote conference among a plurality of users, a system that shares a common screen as a shared information resource between terminals of each user, for example, a video conference system is known. Further, also in the field of pathological diagnosis, it is possible to improve efficiency of diagnosis by sharing images for common pathological diagnosis among terminals of a plurality of doctors and exchanging opinions between terminals while performing diagnosis. It has been done.

  In addition, there is known a technology in which a pointing cursor can be displayed on a screen shared among a plurality of terminals, and a terminal which has acquired the right to speak between the plurality of terminals obtains an operation right of the pointing cursor. (See, for example, Patent Document 1).

  In addition, there is a cooperative medical assistance method that allows two doctors to observe the same image by displaying the same medical information (chest X-ray photograph) on each display device of two medical terminals. In this cooperative medical treatment support method, one medical treatment terminal displays the result of performing high-frequency emphasis file processing on the image on the display device of the own terminal to clarify the lesion site, and edit and process the information in the other medical treatment. Transmit to the terminal. The other medical terminal performs high-frequency emphasis file processing on the image based on the received editing process information, and displays the result on the display device of its own terminal. Thereby, the same editing process result of each doctor of two medical treatment terminals can be shared. (See, for example, Patent Document 2).

Japanese Patent Application Publication No. 07-298235 Japanese Patent Laid-Open No. 08-147391, FIG. 2, paragraphs [0027-0033])

  However, in a system in which opinions are exchanged while sharing the same image among a plurality of terminals, various problems to be solved remain. For example, when the resolution of the entire image is higher than the resolution of the screen, the user proceeds with observation while moving a notable range in the entire image. However, in the meanwhile, even if a user who does not have the operation right discovers a place he / she wants to observe independently, he / she can not visit the place at any time unless the operation right is exchanged. In addition, when the user is allowed to freely exchange the operation right, the image observation from the viewpoint of each user is interrupted one by one, which may lower the efficiency.

  In view of the circumstances as described above, an object of the present technology is to provide an information processing method and an information processing system capable of improving efficiency when an image is shared and observed among users of a plurality of information processing apparatuses. It is to do.

  In order to achieve the above object, an information processing method according to an aspect of the present technology corresponds to each of the plurality of information processing devices in a state where an exclusive operation right is given to one of the plurality of information processing devices. The images synchronized with each other are displayed on a plurality of display units, and when the first request is input from the user to the information processing apparatus which does not have the operation right, the synchronization state of the images of the information processing apparatus is cancelled. When the second request is input from the user to the information processing apparatus for which the synchronization state has been released, the synchronization state is restored.

  In the information processing method according to an aspect of the present technology, on / off of the synchronization state of the image of the information processing apparatus to which the operation right is not granted is appropriately set according to an instruction from the user of the information processing apparatus to which the operation right is not given. It can be switched. As a result, the user of the information processing apparatus to which the operation right is not granted not only passively observes the result of the image operation by the user of the information processing apparatus to which the operation right is granted, but also actively It is possible to perform observation from another viewpoint by operating. In addition, since it is possible to restore the synchronized state of the image at any time, the information processing apparatus to which the operation right is not granted allows the user to return to the original state or the user of the information processing apparatus to which the operation right has been granted. It is possible to freely manipulate the image without being aware of the latest manipulation result of the image by the.

  The information processing method uses the first request to generate an input from a user for an operation on an image displayed on the display unit of the information processing apparatus to which the operation right is not given. When the request is input, the synchronization state of the image of the information processing apparatus may be canceled. As a result, when an input from the user for an operation on an image to which no exclusive operation right is granted is generated, the user does not use the image to which the operation right is assigned as a private image in the information processing apparatus. be able to. In addition, operations such as movement of a private image and change of zoom magnification can be performed simultaneously with the start of an operation for an image to which the operation right is not given. That is, the user's operability is further improved because the user operation only for shifting to the operation on the private image is not required.

  This information processing method includes granting the operation right to the information processing apparatus when an operation right acquisition request is input from the user to the information processing apparatus to which the operation right is not given. You may Thereby, the operation right of the image can be freely moved according to the instruction from the user of each information processing apparatus among the plurality of information processing apparatuses. That is, it is possible to switch at any time between the side that operates the image and the side that only observes the operated image.

  This information processing method displays a plurality of images as images asynchronous with one another and images synchronized with each other among the plurality of information processing devices for each display unit, and synchronizes among the plurality of information processing devices. The operation right may be given to one of a plurality of information processing apparatuses for each combination of images to be performed.

 As a result, each information processing apparatus can display, for example, one image as an operable image and the other image as an image which can be observed only, for the same or different two images. Also, this makes it possible to efficiently exchange, for example, two-way views between two or more users using two images for which different users are given the operation right.

  In this information processing method, the operation right of one of a plurality of images displayed for each display unit is given to one of the information processing apparatuses, and the operation right of the other image is given to the other information processing apparatus When the third request is input from the user to the one information processing apparatus, the one image may be replaced with the other image.

  For example, when the user of one information processing apparatus is a student and the user of the other information processing apparatus is a teacher, the teacher presents an image that has an operation right on the student side on the display side of the information processing apparatus on the student side. You can immediately replace the image with the operation right. For this reason, the student can check the periphery and details of the image for which the teacher has the operation right by operating the image for which the student has the operation right. For this reason, the student can promptly move to the work of confirming the periphery and details of the image that the teacher is currently paying attention to using the image having the operation right on the student side.

  An information processing system according to another embodiment of the present technology includes a plurality of information processing apparatuses. Each of the plurality of information processing devices is provided with an exclusive operation right for one of the display unit capable of displaying an image, an input unit capable of receiving an instruction from a user, and the plurality of information processing devices. A control unit that performs control so that images synchronized with each other are displayed on the plurality of display units corresponding to each of the plurality of information processing devices. The control unit cancels the synchronization state of the image of the information processing apparatus when the first request is input from the user to the information processing apparatus to which the operation right is not given, and the synchronization state is canceled. When the second request is input from the user to the information processing apparatus, the control is performed to restore the synchronization state.

  As described above, according to the present technology, it is possible to improve the efficiency in the case of performing observation while sharing images among users of a plurality of information processing apparatuses.

It is a figure showing composition of an information processing system concerning a 1st embodiment concerning this art. It is a block diagram which shows the functional structure of the synchronous server of FIG. It is a figure which shows the structure of the synchronous management table at the time of single window display mode. It is a figure which shows the structure of the synchronous management table at the time of multi window display mode. It is a figure which shows a structure of the synchronous management table at the time of the multi-window display mode in case the number of windows displayed on each viewer is three. It is a block diagram which shows the functional structure of the viewer of FIG. It is a sequence diagram which shows the flow of the synchronous process including the operation | movement of synchronous / asynchronous switching between viewers. It is a sequence diagram which shows the flow of the synchronous process including the movement process of the operation right between viewers. FIG. 10 is a sequence diagram showing a flow of synchronization processing in a multi-window display mode including copying of an image between windows. It is a figure which shows the hardware constitutions of a typical computer. It is a figure for explaining a modification of an embodiment concerning this art. It is a figure which shows the example of the window displayed on each viewer in single window display mode. It is a figure which shows the state by which the asynchronous button was operated with respect to the image in the 1st window which is a presenter. It is a figure which shows the state by which operation of movement was performed with respect to the image displayed on the 1st window of the presenter. It is a figure which shows the state by which operation of movement was performed with respect to the image displayed on the 1st window of the presenter continuously. It is a figure which shows the state by which operation of movement was performed with respect to the image displayed on the 2nd window of the audience. It is a figure which shows the state which the synchronous relationship of the 1st window and the 2nd window restored. It is a figure which shows the example of a display of the window which shows the transfer process of the operation right between viewers. FIG. 14 is a view showing a display example of a window showing the transfer processing of the operation right between viewers in the same manner. It is a figure which shows the example of a display of multi-window display mode. It is a figure which shows the state by which operation of movement was performed with respect to the image in the 2nd right window (presenter) in a 2nd viewer. It is a figure which shows the state in which the image synchronization process was performed between the 2nd right window in a 2nd viewer, and the 1st right window of a 1st viewer. It is a figure which shows the state by which the inter-window copy button was operated in the left window of a 1st viewer. It is a figure which shows the state of the copy process of the image between windows. It is a figure which shows the synchronous process after the copy process of the image between windows.

Hereinafter, embodiments of the present technology will be described with reference to the drawings.
First Embodiment
[Information processing system]
FIG. 1 is a diagram showing a configuration of an information processing system according to a first embodiment of the present technology.

  The information processing system 100 includes an image server 10, a synchronization server 20, and a plurality of viewers 30A and 30B (information processing devices). These can be connected to each other through the network 40. The network 40 may be a wide area network (WAN) such as the Internet or a local area network (LAN). Also, the network 40 may be wired or wireless. Here, for simplicity of explanation, the case where the two viewers 30A and 30B are connected is shown, but the number of viewers may be three or more. In this specification, when the viewers are individually specified, they are distinguished from “first viewer 30A” and “second viewer 30B”, and when the viewers are not individually specified, “viewers 30A and 30B”. It shall be written as

The image server 10 may be, for example, a typical computer. The image server 10 has an image storage for storing image data, and responds to requests from the viewers 30A and 30B by reading out corresponding image data from the image storage. Here, the image stored in the image storage may be, for example, a microscope image of a pathological sample. The microscope image of the pathological sample has a resolution larger than the screen of the display unit of the viewers 30A and 30B. The image server 10 receives an image data request including, for example, an image name and image position information in the entire image from the viewers 30A and 30B, reads out the corresponding image data from the image storage, and responds to the viewers 30A and 30B. . The image position information may be, for example, coordinate data of an image in the entire image. Alternatively, when the entire image is divided and managed in a unit of a predetermined number of vertical and horizontal pixels called “tiles” (for example, 256 × 256 (pixels), 236 × 512 (pixels)), each unit is previously registered. The assigned number can be used as the image position information.

[Synchronization server 20]
The synchronization server 20 performs synchronization processing to display the same image between the viewers 30A and 30B. Here, "the same image" refers to the same part of the same subject image, and further, the same zoom magnification and the same direction. That is, an image which looks the same between viewers 30A and 30B is referred to as "the same image" herein.

The synchronization server 20 may be configured, for example, by a typical computer system.
FIG. 2 is a block diagram showing the functional configuration of the synchronization server 20 implemented using a typical computer system. As shown in the figure, the synchronization server 20 includes a communication unit 21, a synchronization management table storage unit 23, a synchronization processing unit 25, and the like.

The communication unit 21 receives an image data request, an operation right acquisition request, an asynchronous request, and a synchronization return request with the viewers 30A and 30B through the network 40, and transmits responses to the respective requests.
The synchronization management table storage unit 23 stores a synchronization management table which is management information necessary for synchronization processing of images between the viewers 30A and 30B.

FIG. 3 is a diagram showing the configuration of the synchronization management table.
In the synchronization management table, records for each window are registered as many as the number of windows. The record for each window includes information such as a viewer ID, a window ID, an operation right flag, and an asynchronous flag. Each information will be described below.

The viewer ID is information for identifying the viewer.
The window ID is an ID uniquely given to a combination of a plurality of windows in a corresponding relationship in the sense that they are targets of image synchronization processing among a plurality of viewers. FIG. 3 shows the case where one window is displayed in each of two viewers (single window display mode), so the type of window ID is 1. When a plurality of windows are displayed in each viewer (multi-window display mode), as shown in FIG. 4, there are a plurality of combinations of windows having a corresponding relationship among the plurality of viewers. Therefore, there are a plurality of types of window IDs. Note that the number of windows displayed in each viewer may be three or more. An example of the synchronization management table in this case is shown in FIG. The type of window ID is 3 or more.

  The operation right flag is a flag for specifying one window to which an exclusive operation right (hereinafter simply referred to as "operation right") is given in a combination of a plurality of windows having a corresponding relationship among a plurality of viewers. is there. The operation right flag is set to any one window in a combination of a plurality of windows corresponding to a plurality of viewers. The operation right can perform operations such as movement, change of zoom magnification, rotation, etc. (hereinafter referred to as “image operation”) on an image displayed in a window by inputting an instruction from the user. It is a right. Therefore, the operation by the instruction from the user on the image displayed in the window to which the operation right is not given is invalid. However, when the temporary asynchronous state is set, the restriction is released. The temporary asynchronous state will be described later in detail.

  The operation result of the image in the window to which the operation right is given is also reflected in the image of the window to which the operation right is not given. As a result, the synchronized state of the image displayed in each of the plurality of windows corresponding to the plurality of viewers is secured. In the following description, a window to which an operation right is given is called a “presenter window”, and a window to which no operation right is given is called an “audience window”.

  The asynchronous flag is a flag that is set to a value of “1” when the synchronous state of a plurality of windows in a corresponding relationship among a plurality of viewers is temporarily canceled to be in an asynchronous state. The asynchronous flag is set by an asynchronous request from the viewer. The asynchronous flag is reset by a synchronous return request from the viewer.

  The synchronization processing unit 25 refers to the synchronization management table and performs synchronization processing of images displayed in a plurality of windows that have a correspondence relationship between a plurality of viewers. The synchronization processing unit 25 obtains the image position information, the viewer ID, and the window ID of the window that is the presenter each time an instruction from the user for operating the image in the window that is the presenter is generated. It is possible to receive the synchronization request including using the communication unit 21. When the synchronization processing unit 25 receives the synchronization request, the synchronization processing unit 25 transmits the synchronization request including the respective pieces of information included in the received synchronization request to another viewer using the communication unit 21.

  The synchronization processing unit 25 communicates the operation authority acquisition request including the viewer ID transmitted from the viewer having the window when the operation authority acquisition button in the window of the audience is operated, and the window ID of the window of the audience 21 can be used for reception. The synchronization processing unit 25 sets the operation authority flag for the window of the audience on the synchronization management table based on the viewer ID and the window ID included in the received operation authority acquisition request, and the window and window ID of the audience Reset the operation right flag of the window of the matching presenter. As a result, the operation right is transferred between a plurality of windows in correspondence.

  The synchronization processing unit 25 uses the communication unit 21 for an asynchronous request including the viewer ID transmitted from the viewer having the window when the asynchronous button in the audience window is operated, and the window ID of the window that is the audience. It is possible to receive. Based on the viewer ID and window ID included in the received asynchronous request, the synchronization processing unit 25 sets the asynchronous flag for the audience window on the synchronization management table, thereby bringing the audience window into an asynchronous state. Set

  The synchronization processing unit 25 performs synchronous recovery including the viewer ID transmitted from the viewer having the window when the synchronization button in the audience window set in the asynchronous state is operated, and the window ID of the window that is the audience. The request can be received using the communication unit 21. The synchronization processing unit 25 resets the asynchronous flag for the window of the audience on the synchronization management table based on the viewer ID and the window ID included in the received synchronization return request.

  Since the information processing system 100 according to the present embodiment employs a server / client system, the synchronization server 20 is configured to perform synchronization processing. However, the essence of this technology is independent of the server / client system. Assuming a peer-to-peer system, the synchronization processing by the synchronization server 20 can be performed by at least one of the plurality of viewers 30A, 30B.

[Viewers 30A and 30B]
The viewers 30A, 30B may be configured by, for example, a typical computer system.
FIG. 6 is a block diagram showing a functional configuration of viewers 30A and 30B implemented using a typical computer system. In addition, since the structure of two viewers 30A and 30B is the same, only the structure of 1st viewer 30A is demonstrated.
As shown in the figure, the first viewer 30A includes an input unit 31A, a communication unit 33A, a display unit 35A, a viewer control unit 37A, and the like.

  The input unit 31A is a unit that receives input from the user, such as a mouse, a keyboard, a touch panel, and a voice input unit. The user uses the input unit 31A to issue instructions for performing various operations on the image in the window, and for issuing various requests such as an operation right acquisition request, an asynchronous request, a synchronous return request, and an inter-window copy request. It can be input.

The communication unit 33A transmits various requests such as an image data request, a synchronization request, an operation right acquisition request, an asynchronous request, and a synchronous return request to the synchronous server 20 through the network 40, and receives a response to each request. Do.
The display unit 35A is a device that visually outputs an image, such as, for example, a cathode ray tube (CRT), a plasma display, a liquid crystal display, or an organic EL (electroluminescence) display.

  The viewer control unit 37A (control unit) is an image necessary for updating the image in the window based on the operation information input by the user using the input unit 31A in order to update the image in the window of the presenter. Calculate location information. The viewer control unit 37A transmits an image data request including the image position information, the viewer ID, and the window ID of the presenter window to the image server 10 using the communication unit 33A. Further, the viewer control unit 37A transmits a synchronization request including the image position information, the viewer ID, and the window ID of the presenter window to the synchronization server 20 using the communication unit 33A.

  The viewer control unit 37A invalidates the operation information input by the user using the input unit 31A in order to update the image in the audience window, unless the asynchronous state is set in the audience window. As a result, it is possible to avoid the failure of the synchronization relationship between the presenter and the audience.

  The viewer control unit 37A updates the image in the window based on the image data sent from the image server 10 in response to the transmitted image data request. When the viewer control unit 37A receives the synchronization request including the image position information from the synchronization server 20, the viewer control unit 37A uses the communication unit 33A to transmit the image data request including the image position information included in the synchronization request. , And update the image in the window with the image data sent from the image server 10.

  Like the first viewer 30A, the second viewer 30B also includes an input unit 31B, a communication unit 33B, a display unit 35B, a viewer control unit 37B, and the like. Description of this will be omitted because it will be redundant.

[GUI related to windows]
Next, GUI (Graphical User Interface) related to the window will be described.
FIG. 12 is a diagram showing an example of two windows synchronized with each other between two viewers 30A and 30B. The left side in the drawing is a first window 51A set as a presenter in the first viewer 30A, and the right side is a second window 51B set as an audience in the second viewer 30B.

  The first window 51A and the second window 51B are provided with image operation GUIs 52A and 52B that receive an instruction from a user for operating an image therein. The image operation GUIs 52A and 52B include thumbnail maps 53A and 53B and thumbnail map operation GUIs 54A and 54B. The thumbnail maps 53A and 53B are frames that equivalently show the reduced images 55A and 55B of the entire subject and the range of images displayed in the first window 51A and the second window 51B on the reduced images 55A and 55B. 56A, 56B. The frames 56A and 56B can be moved on the reduced images 55A and 55B by an arbitrary amount in an arbitrary direction according to an instruction from the user. Based on the operation information of the movement of the frames 56A and 56B, the viewer control unit 37A calculates the image position information. On the other hand, the thumbnail map operation GUIs 54A and 54B are configured by GUI elements such as a plurality of buttons for receiving an instruction from the user for operating the image, such as image movement, change of zoom magnification, rotation, and the like. In the thumbnail maps 53A and 53B, the operation of moving the frames 56A and 56B can also be performed by a drag operation using a mouse or the like. The above is description of image operation GUI52A, 52B for manipulating an image.

  In addition, an operation right acquisition button 57B and an asynchronous / synchronous switching button 58B are displayed in the audience window, for example, the second window 51B.

  The operation right acquisition button 57B is a button used when it is desired to acquire the operation right in the second window 51B of the audience. When the operation right is given by operating the operation right acquisition button 57B, the operation right acquisition button 57B disappears from the second window 51B. When the operation right acquisition button 57B disappears from the second window 51B, the user can recognize that he / she has been given the operation right and has become a presenter, and can proceed to image operation.

  The asynchronous / synchronous switching button 58B is a button used when the second window 51B of the audience is temporarily made asynchronous with respect to the first window 51A of the presenter to be used privately. When the asynchronous / synchronous switching button 58B is operated to temporarily enter an asynchronous state, the notation of the asynchronous / synchronous switching button 58B changes from “asynchronous” to “synchronous”, and a second window is displayed by input to the image operation GUI 52B. It becomes possible to operate the 51B image. When the notation is “synchronous”, the asynchronous / synchronous switching button 58B functions as a button for returning from the asynchronous state to the synchronous state.

Also, when the second window 51B of the audience is in synchronization with the first window 51A of the presenter, the user can start some input operation on the image operation GUI 52B of the second window 51B using the input unit 31B. Also in this case, the second window 51B of the audience is out of synchronization and becomes asynchronous. Also in this case, the notation of the asynchronous / synchronous switching button 58B is switched from "synchronous" to "asynchronous". Furthermore, according to the switching method by starting the input operation to the image operation GUI 52B, the actual operation on the image of the second window 51B is performed from the first input operation to the image operation GUI 52B. As a result, compared with the method of operating the asynchronous / synchronous switching button 58B, it is possible to shift to the private operation of the image more quickly.
Note that the present technology is not limited to that the means for inputting an instruction from the user for operating an image is the image operation GUI 52B. For example, a pointer cursor is displayed directly on the image of the second window 51B, and the image is operated by an input operation (click, double click, right click, left click, drag, etc.) using this pointer cursor. Also good. Also in this case, the actual operation may be performed on the image of the second window 51B from the first input operation.

[Operation of Information Processing System 100]
Next, the operation of the information processing system 100 of this embodiment will be described.
The explanation of the operation is
1. 1. Synchronous processing in single window display mode and synchronous / asynchronous switching processing 2. Operation right transfer processing in single window display mode Perform in the order of synchronization processing in multi-window display mode.

[1. Synchronous processing in single window display mode and synchronous / asynchronous switching processing]
Next, the operation of synchronization processing in the single window display mode will be described with reference to FIGS. 7 and 12 to 17. FIG. 7 is a sequence diagram showing a flow of synchronous processing including a synchronous / asynchronous switching operation. 12 to 17 show display examples of two windows corresponding to each other in the single window display mode.

  In FIG. 12, the left side in the drawing is the first window 51A of the presenter in the first viewer 30A, and the right side is the second window 51B of the audience in the second viewer 30B. The first window 51A and the second window 51B are in synchronization with each other. Therefore, the same image is displayed on the first window 51A and the second window 51B. Further, in the second window 51B of the audience, an operation right acquisition button 57B and an asynchronous / synchronous switching button 58B are displayed. The notation of the asynchronous / synchronous switching button 58B is "asynchronous".

  In the first viewer 30A, an arbitrary operation is performed on the image displayed in the first window 51A of the presenter by the input from the user (A1 in FIG. 7). The viewer control unit 37A of the first viewer 30A calculates the image position information based on the operation information input by the user through the input unit 31A. The viewer control unit 37A performs control for updating the image of the first window 51A based on the image position information, and transmits a synchronization request including the image position information to the synchronization server 20 using the communication unit 33A. Control is performed to transmit (A2 in FIG. 7).

  When receiving the synchronization request, the synchronization server 20 extracts image position information from the synchronization request, generates a synchronization request including the image position information, and transmits the synchronization request to the second viewer 30B (A3 in FIG. 7).

When the viewer control unit 37B of the second viewer 30B receives with the communication unit 33A of the synchronization request, extracts the image position information from the synchronization request, the image of the second window 51B to the image position information based on Control for updating is performed (A4 in FIG. 7).

  The above operation is repeated each time an image in the first window 51A is operated in the first viewer 30A as long as the first viewer 30A has the operation right.

  For example, as shown in FIG. 13, when a movement operation is performed on an image in the first window 51A as a presenter, this operation is reflected in the image in the second window 51B as an audience.

  As shown in FIGS. 12 and 13, an operation right acquisition button 57B and an asynchronous / synchronous switching button 58B are displayed in the second window 51B of the audience. Here, an operation when the asynchronous / synchronous switching button 58B is operated by the user in the second window 51B of the audience or when the image operation GUI 52B is operated will be described.

  When the user operates the asynchronous / synchronous switching button 58B or the image operation GUI 52B in the second window 51B of the audience (B1 in FIG. 7), the viewer control unit 37B of the second viewer 30B sends the viewer ID and the viewer ID to the synchronous server 20. An asynchronous request including the window ID is transmitted (B2 in FIG. 7).

Synchronization Synchronization processing unit 25 of the server 20 receives the asynchronous request, and sets the non-synchronization flag of the second window 51B of the synchronization management table (B3 in Fig. 7). Thereby, the synchronization state between the first window 51A and the second window 51B is released. Thereafter, the synchronization processing unit 25 sends an asynchronous response back to the second viewer 30B (B4 in FIG. 7). When receiving the asynchronous response, the viewer control unit 37B of the second viewer 30B changes the notation of the asynchronous / synchronous switching button 58B of the second window 51B to "synchronous", and the second by the input to the image operation GUI 52B. Is set to an asynchronous state in which the operation on the image of the window 51B is possible (B5 in FIG. 7).

  Thereafter, the synchronization processing unit 25 of the synchronization server 20 prohibits the transmission of the image position information from the first viewer 30A to the second viewer 30B until the synchronization recovery request is received from the second viewer 30B. Thereby, the image of the second window 51B of the second viewer 30B becomes an image that can be operated privately to the second viewer 30B, and the user can perform any operation.

  For example, in FIG. 13, it is assumed that the asynchronous / synchronous switching button 58B or the image operation GUI 52B is operated in the second window 51B of the audience. As a result, the second window 51B becomes asynchronous with respect to the first window 51A of the presenter, and the notation of the asynchronous / synchronous switching button 58B changes from “asynchronous” to “synchronous”. Thereafter, as shown in FIGS. 14 and 15, even if an operation is performed on the image displayed in the first window 51A of the presenter (C1 in FIG. 7), the second viewer 30B includes the operation information. The operation is not reflected in the image of the second window 51B, since the synchronization request including the based image position information is not given. In FIGS. 14 and 15, while the operation on the image displayed in the first window 51A of the presenter is repeated, the user does not perform the operation on the image of the second window 51B of the audience. The case is shown.

Then, as shown in FIG. 16, when the operation information is input from the user using the input unit 31A to the image displayed in the second window 51B of the audience, the second of the audience is based on the operation information. The image in the window 51B is updated (B6 in FIG. 7).
As described above, since the image of the second window 51B of the audience can be made asynchronous with the image of the first window 51A of the presenter, the image in the second window 51B of the audience is free. It becomes possible to perform operations.
When the second window 51B of the audience is switched to the asynchronous state by the operation of the image operation GUI 52B, the actual operation is performed on the image displayed in the second window 51B by the first operation of the image operation GUI 52B. Will be executed.

  Next, a case (A1 in FIG. 7) in which the asynchronous / synchronous switching button 58B denoted as “synchronous” is operated in FIG. 16 will be described. In this case, the viewer control unit 37B of the second viewer 30B transmits a synchronization return request including the viewer ID and the window ID of the second window 51B of the audience to the synchronization server 20 using the communication unit 33A (FIG. 7). E2).

Synchronization processing unit 25 of the synchronization server 20 receives the synchronization recovery request, based on the viewer ID and the window ID included in the synchronization recovery request, resets the non-synchronization flag of the second window 51B in the synchronization management table ( E3 in FIG. In addition, the synchronization processing unit 25 sends a synchronization request including the latest image position information calculated by the latest operation (D1 in FIG. 7) to the image of the second window 51B, which is the presenter, as the second source of the synchronization return request. The data is transmitted to the viewer 30B (E4 in FIG. 7).

  When receiving the synchronization request, the viewer control unit 37B of the second viewer 30B performs control to update the image of the second window 51B based on the image position information included in the received synchronization request, and is asynchronous. The notation of the / synchronization switching button 58B is changed from "synchronous" to "asynchronous" (E5 in FIG. 7). Thereby, as shown in FIG. 17, the synchronization state of the first window 51A and the second window 51B is restored.

[2. Operation right transfer processing in single window display mode]
Next, the process of moving the operation right in the single window display mode will be described with reference to FIGS. 8, 18 and 19. FIG. 8 is a sequence diagram showing a flow of synchronization processing including processing for moving the operation right between viewers. FIG. 18 and FIG. 19 are diagrams showing display examples of the window showing the transfer processing of the operation right.

  In FIG. 8, an operation performed by the user on the image displayed in the first window 51A of the presenter is reflected in the image displayed in the second window 51B of the audience (see FIG. 8). A1 to A4) are the same as the operations from A1 to A4 in FIG.

  As shown in FIG. 18, when the operation right acquisition button 57B is operated by the user in the second window 51B of the audience (F1 in FIG. 8), the viewer control unit 37B of the second viewer 30B displays the viewer ID and the audience. The operation right acquisition request including the window ID of the second window 51B is transmitted to the synchronization server 20 (F2 in FIG. 8).

  When receiving the operation right acquisition request, the synchronization processing unit 25 of the synchronization server 20 generates an operation right abandonment request including the viewer ID and the window ID extracted from the operation right acquisition request and transmits the request to the first viewer 30A ( F3 of FIG. 8).

  When receiving the operation right abandonment request, the viewer control unit 37A of the first viewer 30A displays the operation right acquisition button 57A and the asynchronous / synchronous switching button 58A on the first window 51A as shown in FIG. The control is performed (F4 in FIG. 8), and the operation authority abandonment response is sent back to the synchronization server 20 (F5 in FIG. 8).

  When the synchronization server 20 receives the operation right abandonment response from the first viewer 30A, it resets the operation right flag of the first window 51A in the synchronization management table and sets the operation right flag of the second window 51B (see FIG. F6 in FIG. 8). Thereafter, the synchronization processing unit 25 of the synchronization server 20 transmits an operation right acquisition response including at least the window ID of the second window 51B to the second viewer 30B (F7 in FIG. 8).

  When receiving the operation right acquisition response, the second viewer 30B deletes the operation right acquisition button 57B and the asynchronous / synchronous switching button 58B from the second window 51B as shown in FIG. 19 (F8 in FIG. 8).

  Thus, the movement of the operation right from the first window 51A to the second window 51B is completed. Thereafter, the operation performed by the user on the image displayed on the first window 51A that has become a new presenter is reflected in the image displayed on the second window 51B that has newly become an audience. (G1 to G4 in FIG. 8).

[3. Synchronization processing in multi-window display mode]
Next, synchronization processing in the multi-window display mode will be described.
In the case where a plurality of windows are displayed on each viewer (multi-window display mode), since there are a plurality of combinations of windows corresponding to each other, the operation right is 1 for each combination of windows corresponding to each other. Given for one window.

  FIG. 20 is a view showing a display example of a multi-window corresponding to the synchronization management table of FIG. In this example, there are a first viewer 30A with a viewer ID "0001" and a second viewer 30B with a viewer ID "0002". In the first viewer 30A, there is a first left window 61A with a window ID of "01" and a first right window 62A with a window ID of "02". In the second viewer 30B, there is a second left window 61B of window ID = "01" and a second right window 62B of window ID = "02". Then, the operation right is in the first left window 61A in the combination of the window ID = 01 and the operation right is in the second right window 62B in the combination of the window ID = 02.

  Synchronous processing in multi-window display mode, synchronous / asynchronous switching processing, and operation right transfer processing are units of window combinations that correspond to each other, and the same operations in single window display mode are independent of each other. There is no particular difference in being performed. Therefore, these descriptions are omitted.

  An educational system can be considered as an effective use of the multi-window display mode. In the education system, among the combinations of the two windows corresponding to each other, the operation right of one combination is given to the viewer on the teacher side, and the operation right of one other combination is given to the viewer on the student side. The image displayed in the window given the operation right to the viewer on the teacher side is manipulated by the teacher, and the student can refer to the image manipulated by the teacher through the viewer on the student side. On the other hand, the window for which the student's viewer has been given the operation right is related to the image displayed in the window for which the teacher's viewer has the operation right. Used to confirm through However, in such a case, first, the student himself manually adjusts the position of the image in the window for which the student's viewer is given the operation right to the position of the image displayed in the window for which the teacher's viewer has the operation There is a need. Such a fitting operation can be a heavy burden on the user.

  In contrast, in the information processing system 100 according to the present embodiment, in the two windows displayed in the multi-window display mode, the image of the presenter window is directly replaced with a copy of the image of the audience window in accordance with an instruction from the user. It is possible to This eliminates the need to manually align the position of the image in the window that the student viewer has been given the operating right with the position of the image displayed in the window that has the operating right on the teacher's viewer. It is possible to quickly shift to an operation for confirming the periphery and details of the image displayed in the window for which the teacher side has the operation right.

  Next, with reference to FIG. 9 and FIGS. 20 to 25, the synchronization processing in the multi-window display mode including the image copy between the windows will be described.

  In FIG. 20, a first viewer 30A (ID = 0001) is a student viewer, and a second viewer 30B (ID = 0002) is a teacher viewer. In the combination of windows with window ID = 01, the operation right is on the first left window 61A, that is, on the student side, and on the combination of windows with window ID = 02, the operation right is on the second right window 62B, that is, on the teacher side. .

  The presenter's window (first left window 61A, second right window 62B) to which the operation right is given is an inter-window copy button 63A that accepts an instruction from the user for copying an image between the windows. , 63B is displayed.

FIG. 9 is a sequence diagram showing a flow of synchronization processing in the multi-window display mode including copying of images between windows.
Here, the operation performed by the teacher on the image in the second right window 62B (presenter) in the second viewer 30B is the image in the first right window 62A (audience) of the first viewer 30A. The operations (A1 to A4 in FIG. 9) until it is reflected in the same are the same as the operations from A1 to A4 in FIG.

  FIG. 21 shows the result of the movement operation by the teacher on the image in the second right window 62B (presenter) in the second viewer 30B. By the operation of moving the image in the second right window 62B (presenter), as shown in FIG. 22, the image in the first right window 62A (audience) of the first viewer 30A is updated by the synchronization process. Ru.

  Here, when the student wants to check the periphery and details of the image displayed on the first right window 62A using the first left window 61A (presenter) of the first viewer 30A, as shown in FIG. The student operates the inter-window copy button 63A in the first left window 61A using the input unit 31A (H1 in FIG. 9).

  When determining that the inter-window copy button 63A is operated, the viewer control unit 37A of the first viewer 30A obtains a copy of the image data being displayed in the first right window 62A (H2 in FIG. 9). As shown in FIG. 24, the image in the first left window 61A is replaced with this image data (H3 in FIG. 9).

In this way, it is possible to immediately display the image in the window where the teacher currently has the operation right, ie, the part of the image that the teacher is explaining at the current moment, in the window where the student has the operation right . For this reason, the student can immediately shift to a task of confirming the periphery and details of the image currently being watched by the teacher using a window where the student has the operation right.
The image data being displayed in the first right window 62A can be acquired by copying the image data in the memory of the first viewer 30A. Alternatively, the image server 10 may be accessed to acquire image data.

  Further, after updating the image in the first left window 61A, the viewer control unit 37A of the first viewer 30A calculates the image position information of the image after the update, and this image position information, the viewer ID, and the A synchronization request including the window ID of the left window 61A of 1 is transmitted to the synchronization server 20 (H4 in FIG. 9).

  The synchronization server 20 extracts the image position information, the viewer ID, and the window ID from the received synchronization request, and transmits the synchronization request including the information to the second viewer 30B (H5 in FIG. 9).

  The viewer control unit 37B of the second viewer 30B updates the image of the second left window 61B as shown in FIG. 25 based on the position information and the window ID included in the received synchronization request (FIG. 9). H6).

[Typical computers]
Next, a typical computer configuration that can be used for the image server 10, the synchronization server 20, and the viewers 30A and 30B will be described.

FIG. 10 is a diagram illustrating a hardware configuration of a typical computer 200.
As shown in the figure, the computer 200 includes a central processing unit (CPU) 201, a read only memory (ROM) 202, and a random access memory (RAM) 203. The computer 200 further includes an input device 204, an output device 205, a storage device 206, a media interface device 207, a network connection device 208, and a bus 209 connecting them.

  The CPU 201 functions as an arithmetic processing unit and a control unit, and controls the overall operation of the computer 200 according to various programs. The ROM 202 stores programs used by the CPU 201, calculation parameters, and the like. The RAM 203 temporarily stores a program executed by the CPU 201, parameters and the like that appropriately change in the execution.

  The synchronization processing unit 25 of the synchronization server 20 and the viewer control units 37A and 37B of the viewers 30A and 30B are, for example, a CPU 201, a program stored in the ROM 202, a working area of the RAM 203, etc. in the hardware configuration of the computer 200 Is realized.

  The input device 204 generates an input signal based on an input by the user such as a mouse, a keyboard, a touch panel, a button, a microphone, a switch, and a lever, and an input control circuit that generates an input signal based on the input by the user. Etc. The user of the computer 200 can input various data to the CPU 201 and instruct processing operations by operating the input device 204.

  The output device 205 includes, for example, a display device such as a cathode ray tube (CRT) display device, a liquid crystal display (LCD) device, or an organic light emitting diode (OLED) device. Furthermore, the output device 205 includes an audio output device such as a speaker and headphones.

The storage device 206 is a device for storing programs and user data. The storage device 206 includes a storage medium, and a read / write device that writes data to and reads data from the storage medium. The storage system 2 06, for example, a like HDD (Hard Disk Drive), SSD (Solid State Drive).

  The media interface device 207 is a storage medium reader / writer. The media interface device 207 reads and writes data from the mounted removable recording medium 2A such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

  The network connection device 208 is an interface for connecting to the network 40, for example. The network connection device 208 may be a device compatible with wireless LAN (Local Area Network), a device compatible with wireless USB, or a wire communication device performing wired communication.

<Modification 1>
The image synchronization processing between the two viewers has been described so far, but it is needless to say that the present technology can also be applied when three or more viewers are connected. That is, the operation right is given to one window in the combination of three or more windows which correspond to each other among three or more viewers to be the presenter, and the operation on the image of the presenter's window is the window of all other audiences. Should be reflected.

<Modification 2>
In addition, when three or more viewers are connected, it is not necessary to display windows of all viewers on each viewer. For example, when the viewer on the teacher side and the viewers on the student side are connected, as shown in FIG. 11, a window on which the teacher side has the operation right for the teacher and a plurality of all the students as audience windows A window may be displayed, and the viewer on the student side may display a window that the student himself / herself has control over and a teacher's window as an audience window. Such selective control of window display for each viewer uses, for example, a table for managing information for selecting a window to be displayed as an audience for each viewer (for example, a combination of viewer ID and window ID). Can be realized.

In addition, this technique can also take the following structures.
(1) displaying an image synchronized with each other on a plurality of display units corresponding to each of the plurality of information processing devices in a state where an exclusive operation right is given to one of the plurality of information processing devices;
When the first request is input from the user to the information processing apparatus to which the operation right is not given, the synchronization state of the image of the information processing apparatus is canceled.
An information processing method, comprising: recovering the synchronization state when a second request is input from the user to the information processing apparatus whose synchronization state has been canceled.
(2) The information processing method according to (1),
When the first request is input with the occurrence of an input from the user for the operation on the image displayed on the display unit of the information processing apparatus to which the operation right is not given as the first request An information processing method for canceling the synchronization state of the image of the information processing apparatus.
(3) The information processing method according to (1) or (2) above,
An information processing method for giving the operation right to the information processing apparatus when an operation right acquisition request is input from the user to the information processing apparatus to which the operation right is not given.
(4) The information processing method according to any one of (1) to (3), wherein a plurality of images are formed as asynchronous images for each of the display units, and among the plurality of information processing devices. A method of displaying information as an image to be synchronized, and giving the operation right to one of a plurality of information processing devices for each combination of images to be synchronized among the plurality of information processing devices.
(5) The information processing method according to (4),
When the operation right of one image of the plurality of images displayed for each display unit is given to one of the information processing apparatus and the operation right of the other image is given to the other information processing apparatus, An information processing method, comprising replacing the one image with the other image when a third request is input from a user to the one information processing apparatus.

DESCRIPTION OF SYMBOLS 10 ... Image server 20 ... Synchronization server 23 ... Synchronization management table storage part 25 ... Synchronization processing part 30A ... 1st viewer 30B ... 2nd viewer 31A ... Input part 31B ... Input part 35A ... Display part 35B ... Display part 37A ... Viewer control unit 37B ... Viewer control unit 40 ... Network 100 ... Information processing system

Claims (16)

On the screen of the first information processing apparatus exclusive operation right is not granted, to display the image to be synchronized with the second information processing apparatus exclusive the operation right has been granted,
A synchronization switching button for switching the synchronization state of the image and an operation authority switching button for switching the operation authority are displayed on the screen of the first information processing apparatus,
When an input operation is performed by the user of the first information processing apparatus with respect to the synchronous switching button, the synchronous state of the image is canceled to be in an asynchronous state,
The synchronization state is restored when a synchronization request is input from the user to the first information processing apparatus whose synchronization state has been canceled,
The synchronous switching button are the same in the asynchronous state and the synchronous state, Ri different display formats der according to the synchronous state or the asynchronous state,
When the user performs an input operation on the operation authority switching button, an operation authority acquisition request is transmitted to the second information processing apparatus,
When an operation right abandonment request for the operation right acquisition request is received from the second information processing apparatus, the operation right is switched to the first information processing apparatus, and the synchronization switch button and the operation right switch button are disabled,
The resolution of the image is larger than the resolution of the screen of the first information processing apparatus;
Information processing method. An information processing method according to claim 1 ,
20. An information processing method, comprising: canceling the synchronization state of the image when an image operation is performed on the image by a user. An information processing method according to claim 2 ,
The image manipulation is at least one of movement, zoom, change of magnification, and rotation. The information processing method according to any one of claims 1 to 3 ,
An information processing method that determines that the synchronization request is input when the user performs the input operation on the switching button; The information processing method according to any one of claims 1 to 4 , wherein:
In the screen of the first information processing apparatus, an image synchronized between the first information processing apparatus and the second information processing apparatus, and of the first information processing apparatus and the second information processing apparatus An information processing method that displays images that are asynchronous with each other. An information processing method according to claim 5 ,
Displaying a replacement button for replacing one of the plurality of images with the other image on the screen of the first information processing apparatus;
When there is an input operation from the user for the replacement button, it is determined that an image replacement request is input, and the one image is replaced with the other image. The information processing method according to any one of claims 1 to 6 , wherein
An information processing method comprising: displaying a thumbnail image for each of the images on a screen of the first information processing apparatus. The information processing method according to any one of claims 1 to 7 , wherein
The information processing method, wherein the image is a medical image. A first information processing apparatus to which an exclusive operation right is not given ,
Wherein an image to be synchronized with the second information processing apparatus exclusive the operation right is granted, the synchronization switching button for switching the synchronization state of the image, and an operation right switch button for switching the operation authority The image is displayed on the screen of the first information processing apparatus, and when the user of the first information processing apparatus receives an input operation to the synchronous switching button, the synchronous state of the image is canceled to make the image asynchronous. The control unit is configured to restore the synchronization state when a synchronization request is input from the user while the state is released.
The synchronous switching button are the same in the asynchronous state and the synchronous state, Ri different display formats der according to the synchronous state or the asynchronous state,
When the user performs an input operation on the operation authority switching button, an operation authority acquisition request is transmitted to the second information processing apparatus,
When an operation right abandonment request for the operation right acquisition request is received from the second information processing apparatus, the operation right is switched to the first information processing apparatus, and the synchronization switch button and the operation right switch button are disabled,
The resolution of the image is larger than the resolution of the screen of the first information processing apparatus;
Information processing device. The information processing apparatus according to claim 9 ,
An information processing apparatus, wherein the control unit cancels the synchronization state of the image when there is an image operation by a user input to the image. The information processing apparatus according to claim 10 ,
The image operation is at least one of movement, zoom, change of magnification, and rotation. The information processing apparatus according to any one of claims 9 to 11 ,
An information processing apparatus, wherein the control unit determines that the synchronization request is input when the user performs the input operation on the switching button in a state where the synchronization state is canceled. The information processing apparatus according to any one of claims 9 to 12 ,
The control unit is configured to display an image synchronized between the first information processing device and the second information processing device on the screen of the first information processing device, the first information processing device, and the second information processing device. An information processing apparatus that displays images that are asynchronous with each other. An information processing apparatus according to claim 13 ,
The control unit
Displaying a replacement button for replacing one of the plurality of images with the other image on the screen of the first information processing apparatus;
An information processing apparatus that determines that an image replacement request has been input when a user performs an input operation on the replacement button, and replaces the one image with the other image. The information processing apparatus according to any one of claims 9 to 14 ,
An information processing apparatus, wherein the control unit displays a thumbnail image for each of the images on a screen of the first information processing apparatus. The information processing apparatus according to any one of claims 9 to 15 ,
The information processing apparatus is an image for medical use.

Images