JP2018005634A - Screen sharing system, screen sharing method, terminal device, and screen sharing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To share a display range between a plurality of terminals in an entire celestial sphere.SOLUTION: A screen sharing system including a plurality of terminal devices has: selection means that selects entire celestial sphere image data to be shared between the plurality of terminal devices; creation means that creates display image data indicative of a display range to be displayed on a screen of the plurality of terminal devices, of an image area of an entire celestial sphere image based on the entire celestial sphere image data selected by the selection means; display means that displays a display image based on the display image data created by the creation means on a screen of the plurality of terminal devices; and operation means that receives change operation of the display range to be displayed on the screen of the plurality of terminal devices, of the image area of the entire celestial sphere. The creation means is configured to when the change operation is received by the operation means, create display image data indicative of the changed display area, of the image area of the entire celestial sphere.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a screen sharing system, a screen sharing method, a terminal device, and a screen sharing program.

  An image obtained by imaging 360 degrees of the imaging device is also called an omnidirectional image. Such an omnidirectional image can be displayed using dedicated application software, a Web application, or the like. At this time, the user displays a desired display range in the omnidirectional image by performing a display range changing operation (for example, pan operation, tilt operation, zoom operation, etc.) on the omnidirectional image. Can do.

  In addition, a technique is known that enables content data such as image data to be shared and reproduced simultaneously between a plurality of terminals (see, for example, Patent Document 1).

  However, in the above-described conventional technology, there are cases where the display range in the omnidirectional image cannot be shared among a plurality of terminals. That is, there are cases where each terminal displays a different display range in the omnidirectional image while sharing the same omnidirectional image among a plurality of terminals.

  Embodiments of the present invention have been made in view of the above points, and an object of the present invention is to share a display range in an omnidirectional image among a plurality of terminals.

  In order to achieve the above object, an embodiment of the present invention is a screen sharing system including a plurality of terminal devices, and selection means for selecting omnidirectional image data shared between the plurality of terminal devices; Generating means for generating display image data indicating display ranges displayed on the screens of the plurality of terminal devices, out of the image area of the omnidirectional image based on the omnidirectional image data selected by the selection means; Display means for displaying a display image based on the display image data generated by the generation means on a plurality of terminal device screens, and a plurality of terminal device screens among the image areas of the omnidirectional image. An operation unit that accepts a change operation of the displayed display range, and the generation unit, when the change operation is accepted by the operation unit, of the image area of the omnidirectional image, It generates display image data indicating the display range of Sarago.

  According to the embodiment of the present invention, the display range in the omnidirectional image can be shared among a plurality of terminals.

It is a figure which shows the system configuration | structure of an example of the screen sharing system which concerns on this embodiment. It is a figure which shows the hardware constitutions of an example of the terminal device which concerns on this embodiment. It is a figure explaining the outline | summary of the synchronization of the display range in an omnidirectional image. It is a figure which shows the function structure of an example of the terminal device which concerns on this embodiment. It is a sequence diagram showing an example of sharing and display processing of the omnidirectional image. It is a figure which shows the initial screen on which the selection column for selecting a share start, participation in a share, a share end, and a share pause was displayed. It is a figure explaining sharing of the initial display range in a spherical image. It is a sequence diagram which shows the modification of sharing and a display process of a omnidirectional image. It is a sequence diagram which shows an example of the change process of the display range of an omnidirectional image. It is a figure explaining reflection of a change of a display range. It is a sequence diagram which shows the modification of the change process of the display range of an omnidirectional image. It is a sequence diagram which shows an example of the drawing process with respect to the display range of a spherical image. It is a figure explaining reflection of drawing information. It is a figure which shows the other example of the screen where the omnidirectional image which reflected drawing information was displayed. It is a sequence diagram which shows the modification of the drawing process with respect to the display range of a spherical image.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<System configuration>
First, the system configuration of the screen sharing system 1 according to the present embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating a system configuration of an example of a screen sharing system 1 according to the present embodiment.

  As shown in FIG. 1, a screen sharing system 1 according to the present embodiment includes a plurality of terminal devices 10 and is connected to be communicable via a network N such as the Internet or an in-house LAN (Local Area Network). Yes. Hereinafter, when a plurality of terminal devices 10 are distinguished from each other, they are expressed as “terminal device 10A”, “terminal device 10B”, “terminal device 10C”, and the like.

  The terminal device 10 is various information processing devices such as a notebook PC (personal computer), a desktop PC, a smartphone, a tablet terminal, a mobile phone, and a game machine. The terminal device 10 is installed (installed) with a screen sharing program 20 for sharing a screen on which the omnidirectional image is displayed with another terminal device 10.

  Thereby, in the screen sharing system 1 which concerns on this embodiment, the screen on which the omnidirectional image was displayed can be shared between several terminal devices 10 (namely, the display range in an omnidirectional image is synchronized). . Therefore, in the screen sharing system 1 according to the present embodiment, the user of each terminal device 10 can browse the same display range in the same omnidirectional image.

  The display range in the omnidirectional image is a range (a part of the omnidirectional image) displayed by the terminal device 10 in the omnidirectional image. Therefore, hereinafter, an image corresponding to the display range in the omnidirectional image (that is, an image within the display range) is also referred to as “omnidirectional display partial image”.

<Hardware configuration>
Next, the hardware configuration of the terminal device 10 included in the screen sharing system 1 according to the present embodiment will be described with reference to FIG. FIG. 2 is a diagram illustrating a hardware configuration of an example of the terminal device 10 according to the present embodiment.

  As illustrated in FIG. 2, the terminal device 10 according to the present embodiment includes an input device 101, a display device 102, an external I / F 103, and a RAM (Random Access Memory) 104. The terminal device 10 according to the present embodiment includes a ROM (Read Only Memory) 105, a CPU (Central Processing Unit) 106, a communication I / F 107, and a storage device 108. Each of these hardware is connected by a bus B.

  The input device 101 is a touch panel or the like, and is used by the user of the terminal device 10 to perform various operations. Note that the input device 101 may be a keyboard, a mouse, or the like.

  The display device 102 is a display or the like, and displays, for example, an omnidirectional display partial image shared between the terminal devices 10.

  The communication I / F 107 is an interface for connecting the terminal device 10 to the network N. Accordingly, the terminal device 10 can share the omnidirectional display partial image with other terminal devices 10 via the communication I / F 107.

  The storage device 108 is, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like, and stores various programs and data. The programs and data stored in the storage device 18 include an OS (Operating System) that is basic software for controlling the entire terminal device 10 and programs that provide various functions on the OS (for example, the screen sharing program 20 and the like). is there.

  The storage device 108 manages stored programs and data by a predetermined file system or DB (database).

  The external I / F 103 is an interface with an external device. The external device includes a recording medium 103a. Thereby, the terminal device 10 can read and write the recording medium 103 a via the external I / F 103. The recording medium 103a includes, for example, a flexible disk, CD, DVD, SD card, USB memory, and the like.

  The ROM 105 is a non-volatile semiconductor memory that can retain programs and data even when the power is turned off. The ROM 105 stores programs and data such as BIOS (Basic Input / Output System), OS settings, and network settings that are executed when the terminal apparatus 10 is started. The RAM 104 is a volatile semiconductor memory that temporarily stores programs and data.

  The CPU 106 is an arithmetic device that realizes control and functions of the entire terminal device 10 by reading a program and data from the ROM 105, the storage device 108, and the like onto the RAM 104 and executing processing.

  The terminal device 10 according to the present embodiment has the hardware configuration shown in FIG.

<Overview of display range synchronization in spherical images>
Here, an outline of the case where the display range in the omnidirectional image is synchronized between the plurality of terminal devices 10 included in the screen sharing system 1 according to the present embodiment (that is, the case where the omnidirectional display partial image is shared). Will be described with reference to FIG. FIG. 3 is a diagram for explaining the outline of synchronization of the display range in the omnidirectional image.

  As shown in FIG. 3A, it is assumed that the user performs an operation for changing the display range (for example, pan operation) in the omnidirectional image in the terminal device 10A.

  In this case, the omnidirectional display partial image after the display range changing operation is displayed on the screen G110 of the terminal device 10A by the screen sharing program 20 (that is, the display range in the omnidirectional image is changed). ). Further, as shown in FIG. 3A, the screen sharing program 20 synchronizes the change of the display range with the screen G120 of the terminal device 10B, and the omnidirectional display partial image after the change of the display range is displayed. Is displayed.

  Thus, in the screen sharing system 1 according to the present embodiment, the same omnidirectional display partial image is displayed between the terminal device 10A and the terminal device 10B (that is, the terminal device 10A and the terminal device 10B are displayed). Between the celestial sphere display partial images). In other words, in the screen sharing system 1 according to the present embodiment, changes in the display range are synchronized between the terminal device 10A and the terminal device 10B.

  Sharing of the omnidirectional display partial image as described above can be performed by pressing the share button G600 to start sharing the screen among the plurality of terminal devices 10. In FIG. 3, as an example of the display range changing operation, the panning operation has been performed on the omnidirectional image. However, the present invention is not limited to this. The display range changing operation may be, for example, a tilt operation or a zoom operation.

  In addition, a pen button G200, a stamp button G300, an eraser button G400, and a pointer button G500 are also displayed on the screen of the terminal device 10. By pressing these buttons, various drawing information drawn on the omnidirectional display partial image can be shared among a plurality of terminal devices 10.

  For example, by pressing the pen button G200, the user can perform an operation (pen operation) for drawing drawing information such as an arbitrary straight line, curve, or character on the omnidirectional display partial image. Similarly, for example, by pressing the stamp button G300, the user can perform an operation (stamp operation) for drawing drawing information such as an image (icon) called a stamp on the omnidirectional display partial image.

  In addition, for example, by pressing the eraser button G400, the user can perform an operation (eraser operation) of deleting drawing information drawn by a pen drawing operation or a stamp drawing operation from the omnidirectional display partial image. Further, by pressing the pointer button G500, the user can perform an operation (pointer operation) for displaying drawing information such as a pointer (for example, an arrow-shaped icon) on the omnidirectional display partial image.

  The drawing information drawn by the pen operation and the stamp operation, the deletion of the drawing information by the eraser operation, the pointer display by the pointer operation, and the like can be shared among the plurality of terminal devices 10. Hereinafter, the pen operation, stamp operation, eraser operation, and pointer operation are also referred to as “drawing operation”.

  As described above, in the screen sharing system 1 according to the present embodiment, the omnidirectional display partial image can be shared among the plurality of terminal devices 10 and the drawing information (for example, straight line) for the omnidirectional display partial image is shared. , Curves, characters, icons, etc.). That is, in the screen sharing system 1 according to the present embodiment, content such as an omnidirectional image, a change in the display range of the omnidirectional image, and drawing information on the omnidirectional image is shared among a plurality of terminal devices 10. Can do.

  Here, in the screen sharing system 1 according to the present embodiment, one terminal device 10 functioning as a server can change the display range in the omnidirectional image with another terminal device 10 functioning as a client. By synchronizing, omnidirectional display partial image sharing is realized.

  In the screen sharing system 1 according to the present embodiment, the description will be made assuming that the omnidirectional display partial image, which is a partial area in the omnidirectional image, is shared among the plurality of terminal devices 10, but the present invention is not limited thereto. I can't. In the screen sharing system 1 according to the present embodiment, for example, some images (images corresponding to the display range) in the panoramic image that can be changed in the display range may be shared among the plurality of terminal devices 10. .

<Functional configuration>
Next, the functional configuration of the terminal device 10 included in the screen sharing system 1 according to the present embodiment will be described with reference to FIG. FIG. 4 is a diagram illustrating a functional configuration of an example of the terminal device 10 according to the present embodiment.

  As illustrated in FIG. 4, the terminal device 10 according to the present embodiment includes a sharing control unit 11, an operation reception unit 12, a display image generation unit 13, a display control unit 14, and a communication unit 15. Each of these functional units is realized by processing that the screen sharing program 20 causes the CPU 106 to execute.

  The terminal device 10 according to the present embodiment includes a data storage unit 16. The storage unit 16 can be realized using the storage device 108, for example.

  The sharing control unit 11 controls the sharing start, sharing end, sharing participation, sharing suspension, and the like with other terminal devices 10. That is, for example, when the sharing start operation is received by the operation receiving unit 12, the sharing control unit 11 causes the terminal device 10 to function as a server. Further, for example, when the sharing participation operation is received by the operation receiving unit 12, the sharing control unit 11 transmits a sharing participation request to another terminal device 10 functioning as a server. Furthermore, for example, when the operation accepting unit 12 accepts a sharing end operation, the sharing control unit 11 transmits a sharing end request to another terminal device 10 functioning as a server.

  The operation reception unit 12 receives various operations by the user. For example, the operation accepting unit 12 accepts a sharing start operation or a sharing participation operation for starting sharing of an omnidirectional image or the like with another terminal device 10. Further, for example, the operation receiving unit 12 receives a sharing end operation for ending sharing of the omnidirectional image or the like with another terminal device 10. Further, for example, the operation reception unit 12 receives a shared pause operation for temporarily stopping synchronization such as a change in the display range in the omnidirectional image with another terminal device 10.

  The display image generation unit 13 generates omnidirectional display partial image data indicating the omnidirectional display partial image in response to a change operation of the display range in the omnidirectional image. That is, the display image generation unit 13 generates omnidirectional display partial image data by extracting a partial region of the omnidirectional image.

  Here, the omnidirectional display partial image data includes, for example, coordinates indicating the center of the omnidirectional display partial image and the outer edge of the omnidirectional display partial image, as described in JP-A-2015-18296. Can be expressed by an angle of view indicating. Therefore, the display image generation unit 13 extracts the range represented by the coordinates and the angle of view after the change operation from the omnidirectional image in accordance with the change operation of the display range, thereby performing the omnidirectional display partial image data. Is generated.

  Note that the coordinates of the center of the omnidirectional display partial image are the azimuth angle (horizontal coordinate) from the center point of the sphere when the point on the omnidirectional image is made to correspond to the point on the sphere, and the elevation angle ( Vertical coordinates). Further, the angle of view can be represented by the position of the camera device (pseudo camera device) with respect to the center point of the sphere showing the omnidirectional image and the camera angle of the camera device.

  The display control unit 14 displays an omnidirectional display partial image, drawing information, and the like. That is, for example, the display control unit 14 displays the omnidirectional display partial image generated by the display image generation unit 13. Further, for example, the display control unit 14 superimposes and displays various drawing information (for example, curves, straight lines, characters, stamps, etc.) on the omnidirectional display partial image.

  The communication unit 15 performs various data communications with other terminal devices 10. For example, the communication unit 15 transmits a sharing participation request to the terminal device 10 that functions as a server. Further, for example, the communication unit 15 transmits data (display range change data) related to a display range change operation in the omnidirectional image to the other terminal device 10. The display range change data is, for example, data represented by coordinates and a field angle after the display range change operation is performed.

  The data storage unit 16 stores omnidirectional image data indicating an omnidirectional image and data indicating drawing information (for example, image data of a stamp). The stamp image data is, for example, image data represented in a transparent PNG (Portable Network Graphics) format or the like.

<Details of processing>
Next, details of processing of the screen sharing system 1 according to the present embodiment will be described.

(Sharing of initial display range in spherical image)
First, omnidirectional image sharing is started among a plurality of terminal devices 10 (terminal device 10A, terminal device 10B, and terminal device 10C), and the same display range (initial display range) in the omnidirectional image is started. Will be described with reference to FIG. FIG. 5 is a sequence diagram illustrating an example of processing for sharing and displaying an omnidirectional image.

  First, the operation reception unit 12 of the terminal device 10A receives a sharing start operation (step S501). For example, the user of the terminal device 10A can perform a sharing start operation by pressing the sharing start button G111 in the selection field G610 displayed on the initial screen G100 illustrated in FIG. Accordingly, the sharing control unit 11 of the terminal device 10A causes the terminal device 10A to function as a server.

  Such a selection field G610 can be displayed by pressing the share button G600. Further, the initial screen G100 can be displayed, for example, when the user starts the screen sharing program 20.

  In addition, when the user of the terminal device 10 is sharing the omnidirectional image or the like with another terminal device 10, the user can perform the sharing end operation by pressing the sharing end button G113 in the selection field G610. It can be carried out. In addition, when the user of the terminal device 10 shares an omnidirectional image or the like with another terminal device 10, the user can press the share pause button G114 in the selection field G610 to share the image. A pause operation can be performed.

  Next, the operation reception unit 12 of the terminal device 10B receives a sharing participation operation (step S502). For example, the user of the terminal device 10B can perform a sharing start operation by pressing the sharing participation button G112 in the selection field G610 displayed on the initial screen G100 illustrated in FIG. Note that the user of the terminal device 10B presses the share participation button G112 after designating identification information (IP (Internet Protocol) address or the like) of the terminal device 10A functioning as a server, for example.

  The sharing control unit 11 of the terminal device 10B transmits a sharing participation request to the terminal device 10A via the communication unit 15 when the sharing participation operation is received by the operation receiving unit 12 (step S503). And the share control part 11 of 10 A of terminal devices will return a share participation response to the terminal device 10B via the communication part 15, if a share participation request is received. Thereby, the sharing control unit 11 of the terminal device 10B causes the terminal device 10B to function as a client.

  When the sharing control unit 11 of the terminal device 10A receives the sharing participation request, the sharing control unit 11 may request authentication for participating in sharing to the other terminal device 10 that has transmitted the sharing participation request.

  Next, the operation accepting unit 12 of the terminal device 10C accepts a sharing participation operation (step S504). Note that the user of the terminal device 10C can perform the sharing start operation in the same manner as in step S502.

  When the share accepting operation is accepted by the operation accepting unit 12, the share control unit 11 of the terminal device 10C transmits a share participation request to the terminal device 10A via the communication unit 15 (step S505). When the sharing control unit 11 of the terminal device 10A receives the sharing participation request, the sharing control unit 11 returns a sharing participation response to the terminal device 10C via the communication unit 15. Thereby, the sharing control unit 11 of the terminal device 10B causes the terminal device 10C to function as a client.

  Next, the operation accepting unit 12 of the terminal device 10A accepts an operation for selecting a omnidirectional image to be shared (step S506). The user of the terminal device 10A selects the omnidirectional image to be shared by selecting desired omnidirectional image data from the list of omnidirectional image data stored in the data storage unit 16, for example. The operation can be performed.

  Next, the display image generation unit 13 of the terminal device 10A, based on the omnidirectional image data indicating the omnidirectional image selected in step S506, displays the omnidirectional display partial image indicating the omnidirectional display partial image. Generate data. Then, the display control unit 14 of the terminal device 10A displays the omnidirectional display partial image based on the omnidirectional display partial image data generated by the display image generation unit 13 (step S507). Here, the display image generation unit 13 generates omnidirectional display partial image data based on, for example, initial coordinates and initial angle of view (that is, initial display range data) determined in advance for each omnidirectional image. It ’s fine.

  When receiving the selection operation of the omnidirectional image to be shared, the operation accepting unit 12 of the terminal device 10A receives the omnidirectional image data indicating the selected omnidirectional image and the initial display range via the communication unit 15. Data is transmitted to the terminal device 10B and the terminal device 10C (steps S508 and S509).

  Then, the display image generation unit 13 of the terminal device 10B generates omnidirectional display partial image data based on the received omnidirectional image data and initial display range data. Then, the display control unit 14 of the terminal device 10B displays the omnidirectional display partial image based on the omnidirectional display partial image data generated by the display image generation unit 13 (step S510).

  Similarly, the display image generation unit 13 of the terminal device 10C generates omnidirectional display partial image data based on the received omnidirectional image data and initial display range data. Then, the display control unit 14 of the terminal device 10C displays the omnidirectional display partial image based on the omnidirectional display partial image data generated by the display image generation unit 13 (step S511).

  As described above, the same omnidirectional image data and the same initial display range data can be shared among the terminal device 10A, the terminal device 10B, and the terminal device 10C. Accordingly, the same omnidirectional display partial image (that is, the same initial display range in the same omnidirectional image) can be displayed between the terminal device 10A, the terminal device 10B, and the terminal device 10C.

  That is, for example, as illustrated in FIG. 7, the terminal device 10 </ b> A displays a screen G <b> 110 on which the initial display range in the omnidirectional image is displayed. Similarly, for example, as illustrated in FIG. 7, the terminal device 10 </ b> B displays a screen G <b> 120 on which the initial display range in the omnidirectional image is displayed. Similarly, for example, as illustrated in FIG. 7, a screen G130 on which an initial display range in the omnidirectional image is displayed is displayed on the terminal device 10C.

  Thereby, when the omnidirectional image to be shared is selected in the terminal device 10 functioning as a server, the same omnidirectional display partial image can be shared among the plurality of terminal devices 10.

(Modification of sharing initial display range in omnidirectional image)
Here, sharing of the omnidirectional image is started between the plurality of terminal devices 10 (terminal device 10A, terminal device 10B, and terminal device 10C), and the same display range (initial display range) in the omnidirectional image is started. ) Will be described with reference to FIG. FIG. 8 is a sequence diagram showing a modification of the omnidirectional image sharing and display processing. In the omnidirectional image sharing and display process shown in FIG. 8, the processes in steps S501 to S505 are the same as those in FIG.

  Subsequent to step S505, the operation accepting unit 12 of the terminal device 10B accepts an operation for selecting a omnidirectional image to be shared (step S801). The user of the terminal device 10B selects the desired omnidirectional image by selecting desired omnidirectional image data from the list of omnidirectional image data stored in the data storage unit 16, for example. The operation can be performed.

  Next, the display image generation unit 13 of the terminal device 10B, based on the omnidirectional image data indicating the omnidirectional image selected in the above step S801, displays the omnidirectional display partial image indicating the omnidirectional display partial image. Generate data. Then, the display control unit 14 of the terminal device 10B displays the omnidirectional display partial image based on the omnidirectional display partial image data generated by the display image generation unit 13 (step S802).

  When receiving the selection operation of the omnidirectional image to be shared, the operation accepting unit 12 of the terminal device 10B receives the omnidirectional image data indicating the selected omnidirectional image and the initial display range via the communication unit 15. Data is transmitted to the terminal device 10A (step S803).

  When receiving the omnidirectional image data and the initial display range data, the communication unit 15 of the terminal device 10A transmits the omnidirectional image data and the initial display range data to the terminal device 10C (step S804). That is, when the terminal device 10 functioning as a server receives the omnidirectional image data and the initial display range data from the terminal device 10 functioning as a client, the terminal device 10 functions as a client. It transmits to the other terminal device 10.

  Next, the display image generation unit 13 of the terminal device 10A generates omnidirectional display partial image data based on the received omnidirectional image data and initial display range data. Then, the display control unit 14 of the terminal device 10A displays the omnidirectional display partial image based on the omnidirectional display partial image data generated by the display image generation unit 13 (step S805).

  Similarly, the display image generation unit 13 of the terminal device 10C generates omnidirectional display partial image data based on the received omnidirectional image data and initial display range data. Then, the display control unit 14 of the terminal device 10C displays the omnidirectional display partial image based on the omnidirectional display partial image data generated by the display image generation unit 13 (step S806).

  As described above, the same omnidirectional display partial image can be displayed between the terminal device 10A, the terminal device 10B, and the terminal device 10C. Thus, in the terminal device 10 that functions as a client (for example, the terminal device 10B), when the omnidirectional image to be shared is selected, the terminal device 10 sends the omnidirectional sphere to the terminal device 10 that functions as a server. Image data and initial display range data are transmitted. Further, when the terminal device 10 functioning as a server receives the omnidirectional image data and the initial display range data from the one terminal device 10 functioning as a client, the omnidirectional image data and the initial display range data are used as the client. It transmits to the other terminal device 10 which functions.

  Thereby, even when the omnidirectional image to be shared is selected in the terminal device 10 functioning as a client, the same omnidirectional display partial image can be shared among the plurality of terminal devices 10.

(Change of display range in spherical image)
Next, after the initial display range in the omnidirectional image is shared among the plurality of terminal devices 10 (the terminal device 10A, the terminal device 10B, and the terminal device 10C), the display range for the omnidirectional image is displayed. The processing when the change operation is performed will be described with reference to FIG. FIG. 9 is a sequence diagram illustrating an example of processing for changing the display range of the omnidirectional image.

  The operation accepting unit 12 of the terminal device 10A accepts a display range changing operation (step S901). That is, the operation accepting unit 12 accepts a display range changing operation such as a pan operation, a tilt operation, and a zoom operation on the omnidirectional image.

  Next, when the operation accepting unit 12 accepts a display range changing operation, the display control unit 14 of the terminal device 10A changes the display range in the omnidirectional image (step S902).

  That is, the display image generation unit 13 of the terminal device 10A generates omnidirectional display partial image data indicating the omnidirectional display partial image after changing the display range, based on the shared omnidirectional image data. . Then, the display control unit 14 of the terminal device 10 </ b> A displays the omnidirectional display partial image based on the omnidirectional display partial image data generated by the display image generation unit 13. Thereby, the display range in the omnidirectional image is changed, and the changed omnidirectional display partial image is displayed. Here, for example, the display image generation unit 13 generates the omnidirectional display partial image data based on the coordinates and the angle of view (that is, the display range change data) after being changed by the display range change operation. good.

  When receiving the operation for changing the display range, the operation accepting unit 12 of the terminal device 10A transmits display range change data to the terminal device 10B and the terminal device 10C via the communication unit 15 (steps S903 and S904).

  And the display control part 14 of the terminal device 10B will change the display range in an omnidirectional image, if display range change data are received (step S905).

  In other words, the display image generation unit 13 of the terminal device 10B is based on the shared omnidirectional image data and the display range change data, and shows an omnidirectional sphere that indicates the omnidirectional display partial image after the display range is changed. Display partial image data is generated. Then, the display control unit 14 of the terminal device 10 </ b> B displays the omnidirectional display partial image based on the omnidirectional display partial image data generated by the display image generation unit 13. Thereby, the display range in the omnidirectional image is changed, and the changed omnidirectional display partial image is displayed.

  Similarly, when receiving the display range change data, the display control unit 14 of the terminal device 10C changes the display range in the omnidirectional image (step S906).

  In other words, the display image generation unit 13 of the terminal device 10 </ b> C displays the omnidirectional sphere indicating the omnidirectional display partial image after the change of the display range based on the shared omnidirectional image data and the display range change data. Display partial image data is generated. Then, the display control unit 14 of the terminal device 10 </ b> C displays the omnidirectional display partial image based on the omnidirectional display partial image data generated by the display image generation unit 13. Thereby, the display range in the omnidirectional image is changed, and the changed omnidirectional display partial image is displayed.

  Note that the display control unit 14 of the terminal device 10B and the terminal device 10C does not change the display range in the omnidirectional image, for example, when the shared pause button G114 is pressed. Accordingly, the user of the terminal device 10B or the terminal device 10C can temporarily stop the synchronization of the display range change in the omnidirectional image. The user of the terminal device 10B and the terminal device 10C can restart the change synchronization of the display range in the omnidirectional image by pressing the shared pause button G114 again.

  As described above, the same omnidirectional display partial image can be displayed between the terminal device 10A, the terminal device 10B, and the terminal device 10C. In other words, in the screen sharing system 1 according to the present embodiment, the change of the display range in the terminal device 10A can be reflected in the terminal device 10B and the terminal device 10C.

  That is, for example, as shown in FIG. 10, it is assumed that a display range changing operation has been performed on the screen G110 of the terminal device 10A. In this case, in the screen sharing system 1 according to the present embodiment, the display range of the omnidirectional image displayed on the screen G110 of the terminal device 10A is changed. In the screen sharing system 1 according to the present embodiment, the change is also reflected in the omnidirectional image displayed on the screen G120 of the terminal device 10B and the omnidirectional image displayed on the screen G130 of the terminal device 10C. The

  Thereby, when a change operation of the display range is performed in the terminal device 10 functioning as a server, the same omnidirectional display partial image (after being changed by the change operation) between the plurality of terminal devices 10. Omnidirectional display partial image) can be displayed. That is, in the screen sharing system 1 according to the present embodiment, the omnidirectional display partial image displayed on the screens of the plurality of terminal devices 10 when the display range changing operation is performed in the terminal device 10 functioning as a server. Can be synchronized.

(Modification of changing the display range in the spherical image)
Here, after the initial display range in the omnidirectional image is shared among the plurality of terminal devices 10 (the terminal device 10A, the terminal device 10B, and the terminal device 10C), the display range for the omnidirectional image is displayed. A modification of the processing when the change operation is performed will be described with reference to FIG. FIG. 11 is a sequence diagram illustrating a modification of the processing for changing the display range of the omnidirectional image.

  The operation accepting unit 12 of the terminal device 10B accepts a display range changing operation (step S1101). That is, the operation accepting unit 12 accepts a display range changing operation such as a pan operation, a tilt operation, and a zoom operation on the omnidirectional image.

  Next, when the operation accepting unit 12 accepts a display range changing operation, the display control unit 14 of the terminal device 10B changes the display range in the omnidirectional image (step S1102).

  That is, the display image generation unit 13 of the terminal device 10B generates omnidirectional display partial image data indicating the omnidirectional display partial image after changing the display range, based on the shared omnidirectional image data. . Then, the display control unit 14 of the terminal device 10 </ b> B displays the omnidirectional display partial image based on the omnidirectional display partial image data generated by the display image generation unit 13. Thereby, the display range in the omnidirectional image is changed, and the changed omnidirectional display partial image is displayed.

  When receiving the operation for changing the display range, the operation accepting unit 12 of the terminal device 10B transmits display range change data to the terminal device 10A via the communication unit 15 (step S1103).

  When receiving the display range change data, the communication unit 15 of the terminal device 10A transmits the display range change data to the terminal device 10C (step S1104). That is, when receiving the display range change data from the terminal device 10 functioning as a client, the terminal device 10 functioning as a server transmits the display range change data to another terminal device 10 functioning as a client.

  Next, the display control unit 14 of the terminal device 10A changes the display range in the omnidirectional image based on the received display range change data (step S1105).

  That is, the display image generation unit 13 of the terminal device 10A displays the omnidirectional display indicating the omnidirectional display partial image after the change of the display range based on the shared omnidirectional image data and the display range change data. Partial image data is generated. Then, the display control unit 14 of the terminal device 10 </ b> A displays the omnidirectional display partial image based on the omnidirectional display partial image data generated by the display image generation unit 13. Thereby, the display range in the omnidirectional image is changed, and the changed omnidirectional display partial image is displayed.

  Similarly, the display control unit 14 of the terminal device 10C changes the display range in the omnidirectional image based on the received display range change data (step S1106).

  In other words, the display image generation unit 13 of the terminal device 10C displays the omnidirectional display indicating the omnidirectional display partial image after the change of the display range based on the shared omnidirectional image data and the display range change data. Partial image data is generated. Then, the display control unit 14 of the terminal device 10 </ b> C displays the omnidirectional display partial image based on the omnidirectional display partial image data generated by the display image generation unit 13. Thereby, the display range in the omnidirectional image is changed, and the changed omnidirectional display partial image is displayed.

  As described above, the same omnidirectional display partial image can be displayed between the terminal device 10A, the terminal device 10B, and the terminal device 10C. As described above, when a display range change operation is performed in the terminal device 10 that functions as a client (for example, the terminal device 10B), the terminal device 10 sends display range change data to the terminal device 10 that functions as a server. Send. Further, when receiving the display range change data from one terminal device 10 functioning as a client, the terminal device 10 functioning as a server transmits the display range change data to another terminal device 10 functioning as a client.

  As a result, even when a display range change operation is performed in the terminal device 10 functioning as a client, the same omnidirectional display partial image (after being changed by the change operation) among the plurality of terminal devices 10. Can be displayed. That is, in the screen sharing system 1 according to the present embodiment, the omnidirectional display portion displayed on the screens of the plurality of terminal devices 10 even when the display range is changed in the terminal device 10 that functions as a client. Images can be synchronized.

(Drawing on a spherical display partial image)
Next, FIG. 12 shows processing when a drawing operation is performed on a shared omnidirectional display partial image among a plurality of terminal devices 10 (terminal device 10A, terminal device 10B, and terminal device 10C). The description will be given with reference. FIG. 12 is a sequence diagram illustrating an example of a drawing process for the display range of the omnidirectional image.

  The operation reception unit 12 of the terminal device 10A receives a drawing operation for the omnidirectional display partial image (step S1201). That is, the operation reception unit 12 receives a pen operation, a stamp operation, an eraser operation, or a pointer operation with respect to the omnidirectional display partial image.

  Next, when the drawing operation is received by the operation receiving unit 12, the display control unit 14 of the terminal device 10A reflects the drawing (step S1202).

  That is, for example, when a pen operation is received by the operation receiving unit 12, the display control unit 14 displays drawing information such as a straight line, a curve, and a character drawn by the pen operation. Further, when a stamp operation is received by the operation receiving unit 12, the display control unit 14 displays the drawing information of the stamp drawn by the stamp operation. Note that the user can select a stamp drawn by the stamp operation from the image data stored in the data storage unit 16.

  Further, for example, when the eraser operation is accepted by the operation accepting unit 12, the display control unit 14 deletes the drawing information of the operation target by the eraser operation (that is, hides it). Furthermore, for example, when a pointer operation is received by the operation receiving unit 12, the display control unit 14 displays a predetermined pointer at a position (coordinates in the omnidirectional image) designated by the pointer operation.

  Next, when accepting a drawing operation, the operation accepting unit 12 of the terminal device 10A transmits drawing data to the terminal device 10B and the terminal device 10C via the communication unit 15 (steps S1203 and S1204).

  Here, for example, when the drawing operation is a pen operation or a stamp operation, the drawing data is a set of drawing data image data and coordinates at which the drawing information is drawn (coordinates in the omnidirectional image). is there. For example, when the drawing operation is an eraser operation, the drawing data is coordinates where drawing information of an operation target by the eraser operation is drawn. Further, for example, when the drawing operation is a pointer operation, the drawing data is a set of image data of the pointer and coordinates at which the pointer is drawn (coordinates of the omnidirectional image).

  When receiving the drawing data via the communication unit 15, the display control unit 14 of the terminal device 10B reflects the drawing based on the drawing data (step S1205).

  That is, for example, when the display control unit 14 receives drawing data of a pen operation or a stamp operation, the display control unit 14 draws the image data included in the drawing data at a position indicated by the coordinates included in the drawing data. Further, for example, when receiving the drawing data of the eraser operation, the display control unit 14 deletes the drawing information drawn at the position indicated by the coordinates included in the drawing data. Further, for example, when receiving the drawing data of the pointer operation, the display control unit 14 displays the image data included in the drawing data at the position indicated by the coordinates included in the drawing data.

  Similarly, when receiving the drawing data via the communication unit 15, the display control unit 14 of the terminal device 10C reflects the drawing based on the drawing data (step S1206).

  As described above, the same drawing information can be displayed on the same omnidirectional image among the terminal device 10A, the terminal device 10B, and the terminal device 10C. In other words, in the screen sharing system 1 according to the present embodiment, display and deletion of drawing information in the terminal device 10A can be reflected in the terminal device 10B and the terminal device 10C.

  That is, for example, as shown in FIG. 13, on the screen G110 of the terminal device 10A, a drawing operation (a stamp operation for drawing the stamp G710, a pen operation for drawing the character G720, a pointer for drawing the pointer G730) Operation) has been performed. In this case, in the screen sharing system 1 according to the present embodiment, the stamp G710, the character G720, and the pointer G730 are displayed on the omnidirectional display partial image displayed on the screen G110 of the terminal device 10A.

  In the screen sharing system 1 according to the present embodiment, the stamp G710, the character G720, and the pointer G730 are also reflected on the screen G120 of the terminal device 10B. Similarly, in the screen sharing system 1 according to the present embodiment, the stamp G710, the character G720, and the pointer G730 are reflected on the screen G130 of the terminal device 10C.

  Thereby, when drawing operation is performed in the terminal device 10 which functions as a server, the same drawing information can be displayed among the plurality of terminal devices 10. That is, in the screen sharing system 1 according to the present embodiment, when a drawing operation is performed on the terminal device 10 functioning as a server, the drawing information displayed on the screens of the plurality of terminal devices 10 can be synchronized.

  As shown in FIG. 10, the pointer G730 displayed on the terminal device 10A performing the pointer operation, and the pointer G730 displayed on the screens of the terminal devices 10B and 10C reflecting the drawing information of the pointer operation. May be displayed in different images.

  Here, for example, when the screen ratio (aspect ratio) of the terminal device 10A that performed the drawing operation is different from the screen ratio of the terminal device 10B to which the drawing information of the drawing operation is reflected, the terminal device The drawing information may not be reflected in the omnidirectional display partial image displayed by 10B.

  For example, when the screen ratio of the terminal device 10A is “16: 9” and the screen ratio of the terminal device 10B is “4: 3”, the drawing information drawn at the left end or the right end of the screen of the terminal device 10A is the terminal It may not be reflected in the omnidirectional display partial image of the device 10B.

  In such a case, the display control unit 14 of the terminal device 10B, as shown in FIG. 14A, is outside the omnidirectional display partial image displayed on the screen G120 (that is, outside the display range in the omnidirectional image). ) May be displayed with a notification arrow G740 indicating that the drawing information is drawn. Accordingly, the user of the terminal device 10B can know that the drawing information is reflected outside the omnidirectional display partial image.

  Further, as shown in FIG. 14B, the display control unit 14 of the terminal device 10B displays the omnidirectional display partial image in the screen G120 at the same screen ratio as the terminal device 10A. May be switched. Thereby, the user of the terminal device 10B can know the reflection of the drawing information even when the drawing information is reflected outside the omnidirectional display partial image.

(Variation of drawing for omnidirectional display partial image)
Here, regarding a modified example of processing when a drawing operation is performed on a shared omnidirectional display partial image among a plurality of terminal devices 10 (terminal device 10A, terminal device 10B, and terminal device 10C), This will be described with reference to FIG. FIG. 15 is a sequence diagram illustrating a modification of the drawing process for the display range of the omnidirectional image.

  The operation receiving unit 12 of the terminal device 10B receives a drawing operation for the omnidirectional display partial image (step S1501). That is, the operation reception unit 12 receives a pen operation, a stamp operation, an eraser operation, or a pointer operation with respect to the omnidirectional display partial image.

  Next, when a drawing operation is received by the operation receiving unit 12, the display control unit 14 of the terminal device 10B reflects the drawing (step S1502).

  When receiving the drawing operation, the operation receiving unit 12 of the terminal device 10B transmits the drawing data to the terminal device 10A via the communication unit 15 (step S1503).

  When receiving the drawing data, the communication unit 15 of the terminal device 10A transmits the drawing data to the terminal device 10C (step S1504). That is, when receiving the drawing data from the terminal device 10 that functions as a client, the terminal device 10 that functions as a server transmits the drawing data to another terminal device 10 that functions as a client.

  Next, when receiving the drawing data via the communication unit 15, the display control unit 14 of the terminal apparatus 10A reflects the drawing based on the drawing data (step S1505).

  Similarly, when receiving the drawing data via the communication unit 15, the display control unit 14 of the terminal device 10C reflects the drawing based on the drawing data (step S1506).

  As described above, the same drawing information can be displayed on the same omnidirectional image among the terminal device 10A, the terminal device 10B, and the terminal device 10C. As described above, when a drawing operation is performed in the terminal device 10 (for example, the terminal device 10B) that functions as a client, the terminal device 10 transmits drawing data to the terminal device 10 that functions as a server. Further, when receiving the drawing data from one terminal device 10 functioning as a client, the terminal device 10 functioning as a server transmits the drawing data to another terminal device 10 functioning as a client.

  Thereby, when drawing operation is performed in the terminal device 10 which functions as a client, the same drawing information can be displayed among the plurality of terminal devices 10. That is, in the screen sharing system 1 according to the present embodiment, when a drawing operation is performed in the terminal device 10 functioning as a client, the drawing information displayed on the screens of the plurality of terminal devices 10 can be synchronized.

<Summary>
As described above, in the screen sharing system 1 according to the present embodiment, the same celestial sphere image can be shared among a plurality of terminal devices 10, and the entire sky performed by one terminal device 10. The change of the display range for the spherical image can be synchronized with the other terminal devices 10. That is, in the screen sharing system 1 according to the present embodiment, the display range of the omnidirectional image (the omnidirectional display partial image) can be shared among the plurality of terminal devices 10.

  In the screen sharing system 1 according to the present embodiment, the drawing information for the omnidirectional image performed by one terminal device 10 can also be synchronized with other terminal devices 10. That is, in the screen sharing system 1 according to the present embodiment, the drawing information on the omnidirectional image can be shared among the plurality of terminal devices 10.

  The present invention is not limited to the specifically disclosed embodiments, and various modifications and changes can be made without departing from the scope of the claims.

DESCRIPTION OF SYMBOLS 1 Screen sharing system 10 Terminal device 11 Sharing control part 12 Operation reception part 13 Display image generation part 14 Display control part 15 Communication part 16 Data storage part

JP 2007-13704 A

Claims (9)

A screen sharing system including a plurality of terminal devices,
Selecting means for selecting omnidirectional image data shared among a plurality of the terminal devices;
Generating means for generating display image data indicating display ranges displayed on the screens of the plurality of terminal devices, out of the image area of the omnidirectional image based on the omnidirectional image data selected by the selection means;
Display means for displaying a display image based on the display image data generated by the generation means on a plurality of screens of the terminal devices;
An operation means for receiving a change operation of the display range displayed on the screens of the plurality of terminal devices out of the image area of the omnidirectional image;
Have
The generating means includes
A screen sharing system that generates display image data indicating a display range after change in the image area of the omnidirectional image when the change operation is received by the operation means. The generating means includes
The screen sharing system according to claim 1, wherein the display image data is generated based on a coordinate indicating the center of the display range in the omnidirectional image and an angle of view indicating an outer edge of the display range. The generating means includes
The screen sharing system according to claim 2, wherein when the change operation is received by the operation unit, the display image data after change is generated based on the coordinates after change and the angle of view. Drawing means for drawing drawing information on the display image;
The display means includes
The said drawing information is drawn on the said display image currently displayed on the screen of the said some terminal device, when the said drawing information is drawn by the said drawing means. Screen sharing system. The display means includes
When the drawing information is drawn by the drawing means, the drawing information is displayed on the screens of the plurality of terminal devices based on image data for drawing the drawing information and drawing coordinates for drawing the image data. The screen sharing system according to claim 4, wherein the drawing information is drawn on the display image.   The screen sharing system according to any one of claims 1 to 5, wherein the display range changing operation is a pan operation, a tilt operation, or a zoom operation on the omnidirectional image. A screen sharing method used in a screen sharing system including a plurality of terminal devices,
A selection procedure for selecting omnidirectional image data shared among a plurality of the terminal devices;
A generation procedure for generating display image data indicating display ranges displayed on the screens of the plurality of terminal devices, out of the image area of the omnidirectional image based on the omnidirectional image data selected by the selection procedure;
A display procedure for displaying a display image based on the display image data generated by the generation procedure on a plurality of screens of the terminal devices;
An operation procedure for accepting a change operation of the display range displayed on the screens of a plurality of the terminal devices in the image area of the omnidirectional image;
Have
The generation procedure is as follows:
A screen sharing method for generating display image data indicating a display range after change in the image area of the omnidirectional image when the change operation is accepted by the operation procedure. A terminal device that shares a screen with other terminal devices,
Selection means for selecting omnidirectional image data shared between the other terminal devices;
Image transmitting means for transmitting the omnidirectional image data selected by the selecting means to the other terminal device;
Display image data indicating a display range displayed on the screen of the terminal device and the other terminal device is generated from the image region of the omnidirectional image based on the omnidirectional image data selected by the selection unit. Generating means;
Display means for displaying a display image based on the display image data generated by the generation means on a screen of the terminal device;
An operation means for accepting an operation for changing the display range displayed on the screens of the terminal device and the other terminal device in the image area of the omnidirectional image;
Have
The generating means includes
When the change operation is received by the operation means, or when the operation to change the display range is performed on the other terminal device, the display range after the change in the image area of the omnidirectional image A terminal device that generates display image data indicating A terminal device that shares a screen with other terminal devices,
Selecting means for selecting omnidirectional image data shared between the other terminal devices;
Image transmitting means for transmitting the omnidirectional image data selected by the selecting means to the other terminal device;
Display image data indicating a display range displayed on the screen of the terminal device and the other terminal device is generated from the image region of the omnidirectional image based on the omnidirectional image data selected by the selection unit. Generating means,
Display means for displaying a display image based on the display image data generated by the generation means on a screen of the terminal device;
An operation means for accepting an operation for changing the display range displayed on the screens of the terminal device and the other terminal device out of the image area of the omnidirectional image,
Function as
The generating means includes
When the change operation is received by the operation means, or when the operation to change the display range is performed on the other terminal device, the display range after the change in the image area of the omnidirectional image A screen sharing program for generating display image data indicating