JP2023109839A - Communication terminal, communication method, and program - Google Patents

Abstract

To provide a communication terminal, communication system, communication method, and program that cope with the needs for a site on a receiving side of image data to grasp the situation of an entire site on a transmitting side.SOLUTION: In an image communication system consisting of a plurality of photographing devices, video conference terminals, communication management systems, PCs, and smart phones, a video conference terminal 3, which transmits omnidirectional image data so that a smart phone 9 receives it, transmits a display restriction parameter for restricting a display area within a predetermined area of part of an omnidirectional image when the smart phone displays the omnidirectional image so that other communication terminals receive it. Further, display restriction release instruction information indicating an instruction to release the restriction of the display area is transmitted so that the smart phone receives it S125.SELECTED DRAWING: Figure 30

Description

The present disclosure relates to communication terminals, communication systems, communication methods, and programs.

2. Description of the Related Art A conferencing system for holding a remote conference with a remote location via a communication network such as the Internet has become widespread. In this conference system, in a conference room where one of the parties, such as attendees, who conducts a remote conference, is present, the communication terminal of the remote conference system is used to shoot and collect images of the conference room and voices such as remarks of the parties of the conference. Then, they are converted into digital data and transmitted to the other party's communication terminal. As a result, images can be displayed on the display of the other party's conference room, and audio can be output from the speaker, making it possible to conduct a video call. (see Patent Document 1).

In addition, a camera capable of acquiring omnidirectional panoramic images in real time is connected to the communication terminal, and the omnidirectional (panorama) image obtained from this camera is distributed to each communication terminal of the other party, and each communication of the other party In a terminal, a technique is known in which an omnidirectional image is sequentially converted into a rectangular planar image and displayed on a display or the like (see Patent Document 2).

In addition, a communication terminal and a device that can acquire image data in real time are placed in the office (room) where employees usually work, and a connection is established between the communication terminal placed at home, etc. by employees who are working remotely. A system is known in which the status of an office can be constantly grasped by doing so, and at the same time, the state of an employee who is working remotely from the office can also be constantly grasped (see Patent Document 3).

However, when the image data to be sent and received is omnidirectional image data, the site on the receiving side of the omnidirectional image data can display the site on the transmitting side in a wide range, so confidentiality at the site on the transmitting side is high. It even shows things. Therefore, conventionally, from the image taken in a wide range such as a panorama image at the site of the sending side, the prohibition area that you do not want to send or display from the viewpoint of security etc. is specified, and the image is displayed as wide as possible within the displayable area. A method for displaying is disclosed (see Patent Document 4).

However, in the conventional method, the image data receiving site may be rough, so there is a problem that it is not possible to meet the need to grasp the situation of the entire transmitting site.

The invention according to claim 1 is a communication terminal that transmits data of an omnidirectional image so that another communication terminal receives the omnidirectional image, wherein the other communication terminal displays the omnidirectional image. transmitting means for transmitting a display restriction parameter for restricting an area within a predetermined area of a portion of the omnidirectional image so that the other communication terminal receives the display restriction parameter; The communication terminal transmits display restriction release instruction information indicating an instruction to release the restriction of the display area so that the communication terminal receives the display restriction release instruction information.

As described above, according to the present invention, the image data receiving site may be rough, so it is possible to meet the need for grasping the overall situation of the transmitting site.

(a) is a left side view of the imaging device, (b) is a front view of the imaging device, and (c) is a plan view of the imaging device. It is a use image figure of an imaging device. (a) is a hemispherical image (front) captured by the imaging device, (b) is a hemispherical image (back) captured by the imaging device, and (c) is a diagram showing an image represented by the Mercator projection. (a) A conceptual diagram showing a state in which a sphere is covered with a Mercator image, (b) a diagram showing an omnidirectional panorama image. FIG. 4 is a diagram showing the positions of a virtual camera and a predetermined area when the omnidirectional panorama image is a three-dimensional sphere; (a) is a three-dimensional perspective view of FIG. 5, (b) is a diagram showing a state in which an image of a predetermined area is displayed on the display of the communication terminal, and (c) is an enlarged partial area of FIG. 6(b). FIG. 10 is a diagram showing a new predetermined area image in the case of 3 is a diagram showing a relationship between predetermined area information and an image of a predetermined area T; FIG. Fig. 3 shows a point in three-dimensional Euclidean space with spherical coordinates; 1 is a schematic diagram of an image communication system according to an embodiment of the present invention; FIG. It is a hardware block diagram of an imaging device. 2 is a hardware configuration diagram of a video conference terminal; FIG. 2 is a hardware configuration diagram of a communication management system and a PC; FIG. It is a hardware block diagram of a smart phone. 1 is a functional block diagram of an image communication system; FIG. 4 is a conceptual diagram showing an image type management table; FIG. FIG. 3 is a conceptual diagram showing an imaging device management table; FIG. 4 is a conceptual diagram showing a display restriction management table; FIG. 10 is a conceptual diagram showing a display restricted management table; 4 is a conceptual diagram showing a session management table; FIG. 4 is a conceptual diagram showing an image type management table; FIG. FIG. 11 is a conceptual diagram showing an all-display restriction management table; FIG. 10 is a sequence diagram showing a process of joining a specific communication session; FIG. 11 is a diagram showing a screen for selecting a communication session (virtual conference room); FIG. 11 is a sequence diagram showing management processing of image type information; FIG. 10 is a sequence diagram showing communication processing of image data in a video call; 4 is a flow chart showing processing for creating a predetermined area image. (a) shows the case where the photographing device 1a is not used, and (b) shows the case where the photographing device 1a is used. Shown are screen examples of a video call at base B. (a) shows image data sent from imaging devices 1a and 1b without creating an omnidirectional panorama image and a predetermined area image as they are. FIG. 28(b) shows a case where an omnidirectional panorama image and a predetermined area image are created from the image data sent from the imaging devices 1a and 1b. FIG. 10 is a sequence diagram showing processing for setting display restriction parameters; FIG. 12 is a sequence diagram showing processing for canceling display restriction parameter settings in response to a request from a restricted party; FIG. 10 is a sequence diagram showing processing for canceling display restriction parameter settings in response to a restriction-side request; It is an example of a video call screen including a notification screen. It is an example of a video call screen including an approval request screen. FIG. 11 is an example screen of a video call including a “Release display restriction” button; FIG. FIG. 10 is a sequence diagram showing a process of changing display restriction parameter settings according to a restriction-side instruction; It is an example of a video call screen including a change screen.

An embodiment of the present invention will be described below with reference to the drawings.

<<Outline of Embodiment>>
<Method for Generating Spherical Panorama Image>
A method of generating an omnidirectional panorama image will be described with reference to FIGS. 1 to 8. FIG.

First, the appearance of the imaging device 1 will be described with reference to FIG. The photographing device 1 is a digital camera for obtaining a photographed image that is the basis of an omnidirectional (360°) panoramic image. 1(a) is a left side view of the photographing device, FIG. 1(b) is a front view of the photographing device, and FIG. 1(c) is a plan view of the photographing device.

As shown in FIG. 1A, the photographing device 1 is of a size that can be held by a person with one hand. 1(a), 1(b), and 1(c), an image sensor 103a is provided on the front side (front side) and an image sensor 103a on the rear side (rear side). ) is provided with an image sensor 103b. These imaging devices (image sensors) 103a and 103b are used together with optical members (for example, fisheye lenses 102a and 102b to be described later) capable of capturing hemispherical images (angle of view of 180° or more). Further, as shown in FIG. 1B, an operation unit 115 such as a shutter button is provided on the side opposite to the front side of the photographing apparatus 1 .

Next, with reference to FIG. 2, the use condition of the photographing device 1 will be described. In addition, FIG. 2 is a use image figure of an imaging device. As shown in FIG. 2, the photographing device 1 is used by a user, for example, to photograph a subject around the user. In this case, the imaging elements 103a and 103b shown in FIG. 1 capture images of subjects around the user, respectively, so that two hemispherical images can be obtained.

Next, with reference to FIGS. 3 and 4, the outline of the process until the omnidirectional panorama image is created from the image captured by the imaging device 1 will be described. In addition, FIG. 3(a) is a hemispherical image (front side) photographed by the photographing device, FIG. 3(b) is a hemispherical image (rear side) photographed by the photographing device, and FIG. 3(c) is represented by the Mercator projection. 1 is a diagram showing an image (hereinafter referred to as a "Mercator image"). FIG. 4(a) is a conceptual diagram showing a state in which a sphere is covered with a Mercator image, and FIG. 4(b) is a diagram showing a omnidirectional panorama image.

As shown in FIG. 3A, the image obtained by the imaging device 103a is a hemispherical image (front side) curved by a fisheye lens 102a, which will be described later. Also, as shown in FIG. 3B, the image obtained by the imaging device 103b becomes a curved hemispherical image (rear side) by a fisheye lens 102b, which will be described later. Then, the hemispherical image (front side) and the 180-degree inverted hemispherical image (rear side) are combined by the photographing device 1 to create a Mercator image as shown in FIG. 3(c).

Then, by using OpenGL ES (Open Graphics Library for Embedded Systems), the Mercator image is pasted so as to cover the spherical surface as shown in FIG. A spherical panoramic image as shown is created. Thus, the omnidirectional panorama image is represented as an image in which the Mercator image faces the center of the sphere. Note that OpenGL ES is a graphics library used to visualize 2D (2-Dimensions) and 3D (3-Dimensions) data. Note that the omnidirectional panorama image may be a still image or a moving image.

As described above, since the omnidirectional panoramic image is an image pasted so as to cover the spherical surface, it gives a sense of incongruity when viewed by humans. Therefore, by displaying a predetermined area (hereinafter referred to as a "predetermined area image") of a part of the omnidirectional panorama image as a flat image with little curvature, it is possible to display the image without causing discomfort to the human. This will be described with reference to FIGS. 5 and 6. FIG.

FIG. 5 is a diagram showing the positions of the virtual camera and the predetermined area when the omnidirectional panorama image is a three-dimensional sphere. The virtual camera IC corresponds to the position of the viewpoint of the user viewing the omnidirectional panorama image displayed as a three-dimensional sphere. 6(a) is a three-dimensional perspective view of FIG. 5, FIG. 6(b) is a diagram showing a predetermined area image displayed on the display, and FIG. 6(c) is a partial area of FIG. 6(b). is a diagram showing a new predetermined area image when the is enlarged.

In FIG. 6A, the omnidirectional panorama image shown in FIG. 4 is represented by a three-dimensional solid sphere CS. Assuming that the omnidirectional panorama image generated in this way is the stereosphere CS, the virtual camera IC is positioned outside the omnidirectional panorama image as shown in FIG. The predetermined area T in the omnidirectional panoramic image is the imaging area of the virtual camera IC, and is specified by predetermined area information indicating the imaging direction and angle of view of the virtual camera IC in the three-dimensional virtual space containing the omnidirectional panoramic image. be.

Then, the predetermined area image, which is the image of the predetermined area T shown in FIG. 6(a), is displayed on a predetermined display as an image of the imaging area of the virtual camera IC as shown in FIG. 6(b). displayed as The image shown in FIG. 6B is a predetermined area image represented by default predetermined area information (display parameters). In the following description, the imaging direction (ea, aa) and the angle of view (α) of the virtual camera IC will be used.

The relationship between the predetermined area information and the image of the predetermined area T will be described with reference to FIG. FIG. 7 is a diagram showing the relationship between the predetermined area information and the image of the predetermined area T. As shown in FIG. As shown in FIG. 7, "ea" indicates an elevation angle, "aa" indicates an azimuth angle, and "α" indicates an angle of view. That is, the posture of the virtual camera IC is changed so that the point of gaze of the virtual camera IC indicated by the photographing direction (ea, aa) becomes the center point CP of the predetermined area T which is the photographing area of the virtual camera IC. Become. The predetermined area image Q is an image of the predetermined area T in the omnidirectional image CE. f is the distance from the virtual camera IC to the center point CP. L is the distance between an arbitrary vertex of the predetermined area T and the center point CP (2L is a diagonal line). In FIG. 7, a trigonometric function generally represented by the following (Equation 1) holds.

Lf=tan(α/2) (Formula 1)
Further, when a partial area of the predetermined area image shown in FIG. 6(b) is enlarged, a new predetermined area image is displayed as shown in FIG. 6(c).

FIG. 8 is a diagram showing points in three-dimensional Euclidean space in spherical coordinates. Let (r, θ, φ) be the position coordinates when the center point CP is expressed in a spherical polar coordinate system. (r, θ, φ) are the radial, polar and azimuthal angles, respectively. The radius r is equal to f because it is the distance from the origin of the three-dimensional virtual space containing the omnidirectional panoramic image to the center point CP. FIG. 8 is a diagram showing these relationships. Hereinafter, the position coordinates (r, θ, φ) of the virtual camera IC will be used.

<Overview of image communication system>
Next, with reference to FIG. 9, an outline of the configuration of the image communication system according to this embodiment will be described. FIG. 9 is a schematic diagram of the configuration of the image communication system of this embodiment.

As shown in FIG. 9, the image communication system of the present embodiment includes photographing devices 1a and 1b, a video conference terminal 3, a communication management system 5, a PC (Personal Computer) 7, a photographing device 8, and a smartphone 9. configured and capable of communicating over a communication network 100, such as the Internet. The connection form of the communication network 100 may be either wireless or wired.

Of these, the photographing devices 1a and 1b are special digital cameras for photographing subjects, landscapes, etc., and obtaining two hemispherical images that are the basis of an omnidirectional panorama image, as described above. On the other hand, the photographing device 8 is a general digital camera for photographing subjects, landscapes, etc. to obtain general planar images.

The video conference terminal 3 is a terminal dedicated to video conference, and displays an image of a video call (video call) on the display 4 via a wired cable such as a USB (Universal Serial Bus) cable. The videoconference terminal 3 normally captures images with a camera 312, which will be described later. One hemispherical image can be obtained. When a wired cable is used, the cradle 2a not only performs communication between the photographing device 1a and the video conference terminal 3, but also serves to supply power to the photographing device 1a and support the photographing device 1a. Here, the imaging device 1a, the cradle 2a, the video conference terminal 3, and the display 4 are located at the same base A. At base A, four users A1, A2, A3, and A4 participate in a video call.

The communication management system 5 manages communication between the video conference terminal 3, the PC 7, and the smartphone 9, and manages the types of image data transmitted and received (general image and special image type). Here, the special image is a spherical panorama image. The communication management system 5 is installed in a service company or the like that provides video communication services. Further, the communication management system 5 may be constructed by a single computer, or may be constructed by a plurality of computers in which each part (function, means, or storage part) is divided and arbitrarily assigned.

The PC 7 becomes capable of video communication by attaching the photographing device 8 . It should be noted that the PC 7 and the imaging device 8 are located at the base C, which is the same base. In addition, at base C, one user C participates in a video call.

The smartphone 9 displays an image of the video call on a display 917 (described later) provided on the smartphone 9 . The smartphone 9 normally captures images using a CMOS (Complementary Metal Oxide Semiconductor) sensor 905 or the like, which is provided in the device itself. , it is possible to obtain two hemispherical image data that are the basis of the omnidirectional panorama image obtained by the photographing device 1b. When wireless communication technology is used, the cradle 2b serves only to supply power to the photographing device 1b and to support the photographing device 1b. Here, the imaging device 1b, the cradle 2b, and the smart phone 9 are located at the same base B, which is the same base. At base B, two users B1 and B2 are participating in a video call.

Also, the video conference terminal 3, the PC 7, and the smart phone 9 are examples of communication terminals. OpenGL ES is installed on each communication terminal, and it can create specified area information indicating a part of the spherical panorama image, and can display the spherical panorama image sent from other communication terminals. A predetermined area image can be created.

Note that the arrangement of each terminal, device, and user shown in FIG. 9 is an example, and may be another example. For example, at the base C, instead of the imaging device 8, the user may use an imaging device capable of capturing an omnidirectional panorama image. Communication terminals also include digital televisions, smart watches, car navigation devices, and the like. Further, hereinafter, when an arbitrary image capturing device out of the image capturing devices 1a and 1b is indicated, it will be referred to as "image capturing device 1".

<<Hardware Configuration of Embodiment>>
Next, the hardware configurations of the photographing device 1, the video conference terminal 3, the communication management system 5, the PC 7, and the smart phone 9 of this embodiment will be described in detail with reference to FIGS. 10 to 13. FIG. In addition, since the imaging device 8 is a general camera, detailed description thereof will be omitted.

<Hardware configuration of imaging device 1>
First, the hardware configuration of the photographing device 1 will be described with reference to FIG. FIG. 10 is a hardware configuration diagram of the imaging device 1. As shown in FIG. In the following description, the photographing apparatus 1 is assumed to be an omnidirectional (omnidirectional) photographing apparatus using two imaging elements, but the number of imaging elements may be two or more. Further, the device does not necessarily need to be dedicated to omnidirectional photography, and may have substantially the same functions as the imaging device 1 by attaching an omnidirectional photography unit to an ordinary digital camera, smartphone, or the like. .

As shown in FIG. 10, the imaging apparatus 1 includes an imaging unit 101, an image processing unit 104, an imaging control unit 105, a microphone 108, a sound processing unit 109, a CPU (Central Processing Unit) 111, a ROM (Read Only Memory). ) 112, an SRAM (Static Random Access Memory) 113, a DRAM (Dynamic Random Access Memory) 114, an operation unit 115, a network I/F 116, a communication unit 117, and an antenna 117a.

Among them, the imaging unit 101 includes wide-angle lenses (so-called fish-eye lenses) 102a and 102b each having an angle of view of 180° or more for forming a hemispherical image, and two imaging units provided corresponding to each wide-angle lens. It has elements 103a and 103b. The imaging elements 103a and 103b are image sensors such as CMOS (Complementary Metal Oxide Semiconductor) sensors and CCD (Charge Coupled Device) sensors that convert optical images formed by the fisheye lenses 102a and 102b into electrical signal image data and output them. and a timing generation circuit for generating horizontal or vertical synchronizing signals, pixel clocks, etc., and a group of registers for setting various commands and parameters necessary for the operation of this imaging device.

The imaging elements 103a and 103b of the imaging unit 101 are each connected to the image processing unit 104 via a parallel I/F bus. On the other hand, the imaging elements 103a and 103b of the imaging unit 101 are connected separately from the imaging control unit 105 via a serial I/F bus (such as an I2C bus). The image processing unit 104 and imaging control unit 105 are connected to the CPU 111 via the bus 110 . Furthermore, ROM 112, SRAM 113, DRAM 114, operation unit 115, network I/F 116, communication unit 117, electronic compass 118, and the like are also connected to bus 110. FIG.

The image processing unit 104 fetches the image data output from the imaging elements 103a and 103b through the parallel I/F bus, performs predetermined processing on each image data, and synthesizes these image data. , to create Mercator image data as shown in FIG. 3(c).

The imaging control unit 105 generally uses the I2C bus with the imaging control unit 105 as a master device and the imaging elements 103a and 103b as slave devices to set commands and the like in registers of the imaging elements 103a and 103b. Necessary commands and the like are received from the CPU 111 . The imaging control unit 105 also uses the I2C bus to capture status data and the like of the registers of the imaging elements 103 a and 103 b and sends them to the CPU 111 .

Further, the imaging control unit 105 instructs the imaging devices 103a and 103b to output image data at the timing when the shutter button of the operation unit 115 is pressed. Depending on the photographing device 1, there are cases where it has a preview display function or a function corresponding to moving image display on a display (for example, the display of the video conference terminal 3). In this case, the image data is output continuously from the imaging devices 103a and 103b at a predetermined frame rate (frames/minute).

In addition, as will be described later, the imaging control unit 105 also functions as synchronization control means for synchronizing the output timing of the image data of the imaging devices 103a and 103b in cooperation with the CPU 111. FIG. Note that in the present embodiment, the imaging device 1 is not provided with a display unit (display), but a display unit may be provided.

The microphone 108 converts sound into sound (signal) data. The sound processing unit 109 takes in sound data output from the microphone 108 through the I/F bus and performs predetermined processing on the sound data.

The CPU 111 controls the overall operation of the photographing device 1 and executes necessary processing. ROM 112 stores various programs for CPU 111 . The SRAM 113 and DRAM 114 are work memories, and store programs to be executed by the CPU 111, data during processing, and the like. In particular, the DRAM 114 stores image data being processed in the image processing unit 104 and processed Mercator image data.

The operation unit 115 is a general term for various operation buttons, a power switch, a shutter button, a touch panel having both display and operation functions, and the like. A user inputs various shooting modes, shooting conditions, and the like by operating operation buttons.

The network I/F 116 is a general term for interface circuits (such as USB I/F) with external media such as SD cards and personal computers. Also, the network I/F 116 may be wireless or wired. Mercator image data stored in the DRAM 114 is recorded on an external medium via the network I/F 116, or transmitted to an external device such as the video conference terminal 3 via the network I/F 116 as necessary. or

The communication unit 117 communicates with an external device such as the video conference terminal 3 via an antenna 117a provided in the imaging device 1, using short-range wireless technology such as Wi-Fi or NFC (Near Field Communication). The data of the Mercator image can also be transmitted to the external device of the video conference terminal 3 by this communication unit 117 .

The electronic compass 118 calculates the azimuth and tilt (Roll rotation angle) of the imaging device 1 from the magnetism of the earth, and outputs the azimuth/tilt information. This azimuth/tilt information is an example of related information (metadata) according to Exif, and is used for image processing such as image correction of a captured image. The related information also includes data such as the date and time when the image was taken and the data volume of the image data.

<Hardware configuration of the video conference terminal>
Next, the hardware configuration of the video conference terminal 3 will be described with reference to FIG. FIG. 11 is a hardware configuration diagram of a video conference terminal. As shown in FIG. 11, the video conference terminal 3 includes a CPU 301, a ROM 302, a RAM 303, a flash memory 304, an SSD 305, a media I/F 307, an operation button 308, a power switch 309, a bus line 310, a network I/F 311, Equipped with a camera 312, an image sensor I/F 313, a microphone 314, a speaker 315, a sound input/output I/F 316, a display I/F 317, an external device connection I/F 318, a short-range communication circuit 319, and an antenna 319a of the short-range communication circuit 319. ing.

Among these, the CPU 301 controls the operation of the video conference terminal 3 as a whole. A ROM 302 stores a program such as an IPL (Initial Program Loader) that is used to drive the CPU 301 . A RAM 303 is used as a work area for the CPU 301 . The flash memory 304 stores various data such as communication programs, image data, and sound data. An SSD (Solid State Drive) 305 controls reading or writing of various data to/from a flash memory 304 under the control of the CPU 301 . Note that an HDD may be used in place of the SSD. A media I/F 307 controls reading or writing (storage) of data to a recording medium 306 such as a flash memory. The operation button 308 is a button that is operated when selecting a destination of the video conference terminal 3 or the like. A power switch 309 is a switch for switching ON/OFF of the power of the video conference terminal 3 .

A network I/F 311 is an interface for data communication using the communication network 100 such as the Internet. The camera 312 is a type of built-in image capturing means that captures an image of a subject under the control of the CPU 301 and obtains image data. An imaging device I/F 313 is a circuit that controls driving of the camera 312 . The microphone 314 is a type of built-in sound collecting means for inputting sound. A sound input/output I/F 316 is a circuit that processes input/output of sound signals between the microphone 314 and the speaker 315 under the control of the CPU 301 . A display I/F 317 is a circuit that transmits image data to an external display 320 under the control of the CPU 301 . The external device connection I/F 318 is an interface for connecting various external devices. The short-range communication circuit 319 is a communication circuit such as NFC (registered trademark) and Bluetooth (registered trademark).

A bus line 310 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 301 shown in FIG.

The display 4 is a kind of display means configured by liquid crystal or organic EL for displaying an image of a subject, icons for operation, and the like. Also, the display 4 is connected to the display I/F 317 by a cable 4c. This cable 4c may be a cable for analog RGB (VGA) signals, a cable for component video, HDMI (High-Definition Multimedia Interface) (registered trademark) or DVI (Digital Video Interface). Interactive) signal cable.

Note that the camera 312 includes a lens and a solid-state imaging device that converts light into electric charge and digitizes an image (video) of a subject. As the solid-state imaging device, a CMOS sensor, a CCD sensor, or the like is used. External devices such as an external camera, an external microphone, and an external speaker can be connected to the external device connection I/F 318 via a USB (Universal Serial Bus) cable or the like. When an external camera is connected, the external camera is driven according to the control of the CPU 301 prior to the built-in camera 312 . Similarly, when an external microphone is connected or when an external speaker is connected, according to the control of the CPU 301, the external microphone and speaker are prioritized over the built-in microphone 314 and built-in speaker 315, respectively. Powered by an external speaker.

Also, the recording medium 306 is detachably attached to the video conference terminal 3 . In addition, as long as it is a non-volatile memory from which data is read or written under the control of the CPU 301, not only the flash memory 304 but also an EEPROM (Electrically Erasable and Programmable ROM) or the like may be used.

<Communication management system, PC hardware configuration>
Next, hardware configurations of the communication management system 5 and the PC 7 will be described with reference to FIG. FIG. 12 is a hardware configuration diagram of the communication management system and PC. Since the communication management system 5 and the PC 7 are both computers and have the same configuration, the configuration of the communication management system 5 will be described below, and the description of the configuration of the PC 7 will be omitted.

The communication management system 5 includes a CPU 501 that controls the overall operation of the communication management system 5, a ROM 502 that stores programs used to drive the CPU 501 such as IPL, a RAM 503 that is used as a work area for the CPU 501, and programs for the communication management system 5. HD 504 for storing various data such as; HDD (Hard Disk Drive) 505 for controlling reading or writing of various data to the HD 504 under the control of the CPU 501; A media drive 507 to be controlled, a display 508 for displaying various information such as cursors, menus, windows, characters or images, a network I/F 509 for data communication using the communication network 100, characters, numerical values, various instructions, etc. A keyboard 511 having a plurality of keys for inputting , a mouse 512 for selecting and executing various instructions, selecting a processing target, moving a cursor, etc., and a CD-RW (Compact Disc) as an example of a removable recording medium. -ReWritable) 513, a CD-RW drive 514 that controls reading or writing of various data, and an address bus, data bus, etc. for electrically connecting the above components as shown in FIG. A bus line 510 is provided.

<Hardware configuration of smartphone>
Next, the hardware of the smart phone will be described with reference to FIG. 13 . FIG. 13 is a hardware configuration diagram of a smartphone. As shown in FIG. 13 , the smartphone 9 includes a CPU 901 , ROM 902 , RAM 903 , EEPROM 904 , CMOS sensor 905 , acceleration/direction sensor 906 , media I/F 908 and GPS receiver 909 .

Among these, the CPU 901 controls the operation of the smartphone 9 as a whole. The ROM 902 stores programs such as IPL, which are used to drive the CPU 901 . A RAM 903 is used as a work area for the CPU 901 . The EEPROM 904 reads or writes various data such as smartphone programs under the control of the CPU 901 . A CMOS (Complementary Metal Oxide Semiconductor) sensor 905 captures a subject (mainly self-portrait) and obtains image data under the control of the CPU 901 . The acceleration/direction sensor 906 is various sensors such as an electronic magnetic compass, a gyro compass, and an acceleration sensor for detecting geomagnetism. A media I/F 908 controls reading or writing (storage) of data to a recording medium 907 such as a flash memory. A GPS receiver 909 receives GPS signals from GPS satellites.

In addition, the smartphone 9 includes a long-distance communication circuit 911, a camera 912, an image sensor I/F 913, a microphone 914, a speaker 915, a sound input/output I/F 916, a display 917, an external device connection I/F 918, a short-distance communication circuit 919, An antenna 919 a of a short-range communication circuit 919 and a touch panel 921 are provided.

Among these, the telecommunications circuit 911 is a circuit that communicates with other devices via the communication network 100 . The camera 912 is a type of built-in imaging means for capturing an image of a subject and obtaining image data under the control of the CPU 901 . An imaging device I/F 913 is a circuit that controls driving of the camera 912 . The microphone 914 is a type of built-in sound collecting means for inputting sound. A sound input/output I/F 916 is a circuit that processes input/output of sound signals between the microphone 914 and the speaker 915 under the control of the CPU 901 . A display 917 is a kind of display means such as a liquid crystal or an organic EL that displays an image of a subject, various icons, and the like. The external device connection I/F 918 is an interface for connecting various external devices. A short-range communication circuit 919 is a communication circuit such as NFC or fBluetooth. The touch panel 921 is a type of input means for operating the smartphone 9 by the user pressing the display 917 .

The smartphone 9 also includes a bus line 910 . A bus line 910 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 901 .

Recording media such as CD-ROMs storing the above programs and HDs storing these programs can both be provided domestically or internationally as program products.

<<Function Configuration of Embodiment>>
Next, the functional configuration of this embodiment will be described with reference to FIGS. 14 to 20. FIG. FIG. 14 is a functional block diagram of the imaging devices 1a and 1b, the video conference terminal 3, the communication management system 5, the PC 7, and the smart phone 9, which constitute a part of the image communication system of this embodiment.

<Functional configuration of imaging device 1a>
As shown in FIG. 14, the photographing device 1a has a reception section 12a, an imaging section 13a, a sound collection section 14a, a communication section 18a, and a storage/readout section 19a. Each of these units is a function or means implemented by any one of the components shown in FIG. is.

The photographing apparatus 1 also has a storage unit 1000a constructed by the ROM 112, SRAM 113, and DRAM 114 shown in FIG. The storage unit 1000a stores a GUID (Globally Unique Identifier) of its own device.

(Each functional configuration of the imaging device 1a)
Next, with reference to FIGS. 10 and 14, each functional configuration of the photographing device 1a will be described in further detail.

The reception unit 12a of the photographing device 1a is realized mainly by processing of the operation unit 115 and the CPU 111 shown in FIG. 10, and receives operation input from the user.

The imaging unit 13a is realized mainly by the processing of the imaging unit 101, the image processing unit 104, the imaging control unit 105, and the CPU 111 shown in FIG.

The sound collecting unit 14a is implemented by the processing of the microphone 108, the sound processing unit 109, and the CPU 111 shown in FIG.

The communication unit 18a is mainly implemented by the processing of the CPU 111, and can communicate with the communication unit 38 of the video conference terminal 3 by short-range wireless communication technology such as the NFC standard, BlueTooth, and Wi-Fi.

The storage/readout unit 19a is mainly realized by the processing of the CPU 111 shown in FIG. read out.

<Each functional configuration of the imaging device 1b>
The photographing device 1b has a reception section 12b, an imaging section 13b, a sound collection section 14b, a communication section 18b, and a storage/readout section 19b. Since these have functions similar to those of the receiving unit 12a, the imaging unit 13a, the sound collecting unit 14a, the communication unit 18a, and the storage/reading unit 19a in the photographing device 1a, description thereof will be omitted. The imaging apparatus 1 also has a storage unit 1000b constructed by the ROM 112, SRAM 113, and DRAM 114 shown in FIG. The storage unit 1000b stores the GUID of its own device.

<Functional Configuration of Video Conference Terminal 3>
As shown in FIG. 14, the videoconference terminal 3 includes a transmission/reception unit 31, a reception unit 32, an image/sound processing unit 33, a display control unit 34, a determination unit 35, a creation unit 36, a communication unit 38, and a storage unit.・It has a reading unit 39 . Each of these units is realized by any one of the components shown in FIG. It is a function or a means.

The video conference terminal 3 also has a storage unit 3000 constructed by the ROM 302, RAM 303, and flash memory 304 shown in FIG. In this storage unit 3000, an image type management DB 3001, an imaging device management DB 3002, a display restriction management DB 3003, and a display restriction management DB 3004 are constructed. Of these, the image type management DB 3001 is composed of the image type management table shown in FIG. The imaging device management DB 3002 is composed of the imaging device management table shown in FIG. The display restriction management DB 3003 is composed of a display restriction management table shown in FIG. The restricted display management DB 3004 is composed of a restricted display management table shown in FIG.

(Image type management table)
FIG. 15 is a conceptual diagram showing an image type management table. In this image type management table, an image data ID, an IP address as an example of a destination of a transmission source terminal, and a source name are stored and managed in association with each other. Among these, the image data ID is an example of image data identification information for identifying image data when performing video communication. The same image data ID is added to the image data transmitted from the same source terminal. As a result, the destination terminal (communication terminal on the receiving side) can identify the source terminal of the received image data. The IP address of the transmission source terminal indicates the IP address of the communication terminal that transmits the image data indicated by the associated image data ID. A source name is a name for specifying an imaging device that outputs image data indicated by an associated image data ID, and is an example of image type information. This source name is a name created by each communication terminal such as the video conference terminal 3 according to a predetermined naming rule.

For example, three communication terminals with IP addresses "1.2.1.3", "1.2.2.3", and "1.3.1.3" respectively have image data ID "RS001". , "RS002" and "RS003" are transmitted. Furthermore, the types of images indicated by the source names of the respective communication terminals are "Video_Theta", "Video", and "Video_Theta", and the image types are "special image", "general image", and "special image" in order. It indicates that Note that the special image is a spherical panorama image here.

Data other than image data may also be managed in association with the image data ID. Data other than image data are, for example, sound data and document data for screen sharing.

(Imaging device management table)
FIG. 16 is a conceptual diagram showing an imaging device management table. In this photographing device management table, vendor IDs and product IDs of the GUIDs of photographing devices that can obtain two hemispherical images that are the basis of the omnidirectional panorama image are stored and managed. As the GUID, for example, the vendor ID (VID) and product ID (PID) used in the USB device can be used. The vendor ID and product ID are stored when the communication terminal such as the video conference terminal is shipped from the factory, but they may be additionally stored after the shipment from the factory.

(Display restriction management table)
FIG. 17 is a conceptual diagram showing a display restriction management table. In this display restriction management table, display restriction of the predetermined area image is given for each IP address of the communication terminal (destination terminal) to which the data of the two hemispherical images that are the basis of the omnidirectional panorama image are transmitted. It manages display restriction parameters that indicate content. Specifically, the display restriction parameter is set to Restrictions are indicated. For example, for a communication terminal with an IP address of "1.3.1.2", the content set so that the viewing angle α must exceed 120° (cannot be less than 120°) is managed. there is

(Display restricted management table)
FIG. 18 is a conceptual diagram showing a display restriction management table. In this display-restricted management table, the own terminal (video conference terminal 3) is specified for each IP address of the communication terminal (source terminal) that transmits the data of the two hemispherical images that are the basis of the omnidirectional panorama image. It manages display restriction parameters that indicate what restrictions are given (subjected) to the display of area images. This display restriction parameter is the same as the display restriction parameter managed by the display restriction management table.

(Each functional configuration of the video conference terminal)
Next, each functional configuration of the video conference terminal 3 will be described in more detail with reference to FIGS. 11 and 14. FIG.

The transmission/reception unit 31 of the video conference terminal 3 is mainly realized by processing of the network I/F 311 and CPU 301 shown in FIG. send and receive

The reception unit 32 is realized mainly by processing by the operation buttons 308 and the CPU 301, and receives various selections or inputs from the user. In addition to the operation buttons 308, a touch panel or the like may be used as other input means.

The image/sound processing unit 33 is implemented by a command from the CPU 301 shown in FIG. 11, and performs image processing on image data obtained by imaging a subject with the camera 312 . Further, after the user's voice is converted into an audio signal by the microphone 314, the image/sound processing unit 33 performs audio processing on the audio data related to this audio signal.

Further, the image/sound processing unit 33 performs image processing on image data received from another communication terminal based on the image type information such as the source name so that the display control unit 34 causes the display 4 to display an image. do Specifically, when the image type information indicates that the image is a special image, the image/sound processing unit 33 processes the image data (for example, each image shown in FIGS. 3(a) and 3(b)). Hemispherical image data) is converted into omnidirectional panorama image data as shown in FIG. Create a given area image as shown. Also, the image/sound processing unit 33 outputs to the speaker 315 an audio signal related to the sound data received from another communication terminal via the communication management system 5, and causes the speaker 315 to output sound.

The display control unit 34 is mainly realized by processing of the display I/F 317 and the CPU 301 , and performs control for displaying various images, characters, etc. on the display 4 .

The determination unit 35 is mainly implemented by the processing of the CPU 301, and determines, for example, the image type related to the image data received from the imaging device 1a.

The creating unit 36 is mainly realized by the processing of the CPU 301, and based on the result of the determination by the determination unit 35 that the image is a general image or a special image (here, a spherical panorama image), determines the image type according to the naming rule described above. Create a source name, which is an example of information. For example, when the determination unit 35 determines that the image is a general image, the creation unit 36 creates a source name “Video” indicating that it is a general image. On the other hand, when the determination unit 35 determines that the image is a special image, the creation unit 36 creates a source name “Video_Theta” indicating that it is a special image.

The communication unit 38 is mainly implemented by the short-range communication circuit 319, the antenna 318a, and the processing of the CPU 301, and communicates with the communication unit 18a of the imaging device 1a by short-range wireless technology such as NFC, BlueTooth, and Wi-Fi. be able to. Although the communication unit 38 and the transmission/reception unit 31 have separate communication units, they may be shared.

The storage/readout unit 39 is mainly realized by the processing of the CPU 301 shown in FIG. read out.

<Functional configuration of communication management system>
Next, each functional configuration of the communication management system 5 will be described in detail with reference to FIGS. 12 and 14. FIG. The communication management system 5 has a transmission/reception section 51 , a judgment section 55 , a generation section 56 and a storage/read section 59 . 12 operates according to commands from the CPU 501 in accordance with the program for the communication management system 5 developed on the RAM 503 from the HD 504. It is a means.

The communication management system 5 also has a storage unit 5000 constructed by the RAM 503 and the HD 504 shown in FIG. A session management DB 5001 , an image type management DB 5002 , and a total display restriction management DB 5003 are constructed in the storage unit 5000 . Of these, the session management DB 5001 is composed of the session management table shown in FIG. The image type management DB 5002 is composed of an image type management table shown in FIG. The all-display restriction management DB 5003 is composed of the all-display restriction management table shown in FIG.

(session management table)
FIG. 19 is a conceptual diagram showing a session management table. In this session management table, session IDs and IP addresses of participating communication terminals are stored and managed in association with each other. Among these, the session ID is an example of session identification information for identifying a communication session that implements a video call, and is generated for each virtual conference room. The session ID is also managed by each communication terminal such as the video conference terminal 3, and is used when selecting a communication session in each communication terminal. The IP address of the participating communication terminal indicates the IP address of the communication terminal that has participated in the virtual conference room indicated by the associated session ID.

(Image type management table)
FIG. 20 is a conceptual diagram showing an image type management table. In the image type management table shown in FIG. 20, in addition to each information managed in the image type management table shown in FIG. 15, the same session IDs as those managed in the session management table are associated. managed by Here, the virtual conference rooms indicated by the same session ID "se101" have IP addresses "1.2.1.3", "1.2.2.3", and "1.3.1. 3" are shown participating. In the communication management system 5, the image data ID, the IP address of the transmission source terminal, and the image type information managed by the communication terminal such as the video conference terminal 3 are managed by the new communication terminal. This is for transmitting image type information and the like to a communication terminal that is already engaged in a video call and a communication terminal that newly participates, such as when entering a virtual conference room. As a result, there is no need to transmit and receive image type information and the like between a communication terminal that is already engaged in a video call and a communication terminal that has newly joined.

(All display restriction management table)
FIG. 21 is a conceptual diagram showing a total display restriction management table. This all-display restriction management table is a table that integrates the display restriction table (see FIG. 17) and the display restricted table (see FIG. 18). That is, in the all display restriction management table, the IP address of the communication terminal (source terminal) that transmits the data of the two hemispherical images that are the basis of the omnidirectional panorama image and the IP address of the communication terminal (destination terminal) that is the destination of the transmission. For each IP address, a display restriction parameter is managed that indicates the contents of restriction on display of the predetermined area image. Accordingly, the communication management system 5 can manage what kind of restrictions the image data transmission source imposes on each transmission destination.

(Each functional configuration of the communication management system)
Next, each functional configuration of the communication management system 5 will be described in detail with reference to FIGS. 12 and 14. FIG.

The transmission/reception unit 51 of the communication management system 5 is mainly realized by processing of the network I/F 509 and CPU 501 shown in FIG. information).

The determination unit 55 is mainly implemented by the processing of the CPU 501 and makes various determinations.

The generation unit 56 is mainly realized by processing of the CPU 501 and generates an image data ID.

The storage/readout unit 59 is mainly realized by processing of the HDD 505 and the CPU 501 shown in FIG. information).

<PC functional configuration>
Next, the functional configuration of the PC 7 will be described in detail with reference to FIGS. 12 and 14. FIG. The PC 7 basically has the same functions as the videoconference terminal 3 . That is, as shown in FIG. 14, the PC 7 includes a transmission/reception section 71, a reception section 72, an image/sound processing section 73, a display control section 74, a determination section 75, a creation section 76, a communication section 78, and a storage/ It has a reading section 79 . Each of these units is a function or means realized by any one of the components shown in FIG. .

The PC 7 also has a storage unit 7000 constructed by the ROM 502, RAM 503, and HD 504 shown in FIG. An image type management DB 7001 and an imaging device management DB 7002 are constructed in the storage unit 7000 . Note that the image type management DB 7001 and the photographing device management DB 7002 have the same configurations as the image type management DB 3001 and the photographing device management DB 3002, respectively, and thus description thereof will be omitted.

(Each functional configuration of the PC)
The transmitting/receiving section 71 of the PC 7 is mainly implemented by the processing of the network I/F 509 and the CPU 501 shown in FIG.

The reception unit 72 is realized mainly by the keyboard 511, the mouse 512, and the processing of the CPU 501, and realizes the same functions as the reception unit 32. FIG. The image/sound processing unit 73 is mainly implemented by commands from the CPU 501 and implements the same functions as the image/sound processing unit 33 . The display control unit 74 is mainly implemented by processing of the CPU 501 and implements the same functions as the display control unit 34 . The determination unit 75 is mainly implemented by the processing of the CPU 501 and implements the same function as the determination unit 35 . The creating unit 76 is mainly implemented by the processing of the CPU 501 and implements the same function as the creating unit 36 . The communication unit 78 is mainly implemented by the processing of the CPU 501 and implements the same functions as the communication unit 38 . The storage/readout unit 79 is implemented by the processing of the CPU 501 , stores various data (or information) in the storage unit 7000 and reads out various data (or information) from the storage unit 7000 .

<Smartphone functional configuration>
Next, the functional configuration of the smartphone 9 will be described in detail with reference to FIGS. 13 and 14. FIG. The smart phone 9 basically has the same functions as the video conference terminal 3 . That is, as shown in FIG. 14, the smartphone 9 includes a transmission/reception unit 91, a reception unit 92, an image/sound processing unit 93, a display control unit 94, a determination unit 95, a creation unit 96, a communication unit 98, and a storage unit.・It has a reading unit 99 . Each of these units is a function or means implemented by any one of the components shown in FIG. be.

The smartphone 9 also has a storage unit 9000 constructed by the ROM 902, RAM 903, and EEPROM 904 shown in FIG. An image type management DB 9001 and an imaging device management DB 9002 are constructed in this storage unit 9000 . Note that the image type management DB 9001 and the photographing device management DB 9002 have the same configurations as the image type management DB 3001 and the photographing device management DB 3002, respectively, and therefore description thereof will be omitted.

(Each function configuration of the smartphone)
The transmitting/receiving unit 91 of the smartphone 9 is mainly implemented by the processing of the long-distance communication circuit 911 and the CPU 901 shown in FIG.

The reception unit 92 is mainly realized by processing of the touch panel 921 and the CPU 901 and realizes the same function as the reception unit 32 .

The image/sound processing unit 93 is mainly implemented by commands from the CPU 901 and implements the same functions as the image/sound processing unit 33 .

The display control unit 94 is mainly implemented by the processing of the CPU 901 and implements the same functions as the display control unit 34 .

The determination unit 95 is mainly implemented by the processing of the CPU 901 and implements the same function as the determination unit 35 .

The creating unit 96 is mainly implemented by the processing of the CPU 901 and implements the same functions as the creating unit 36 .

The communication unit 98 is mainly implemented by the processing of the CPU 901 and implements the same functions as the communication unit 38 .

A storage/readout unit 99 is realized by processing of the CPU 901 , stores various data (or information) in the storage unit 9000 , and reads out various data (or information) from the storage unit 9000 .

<<Processing or operation of the embodiment>>
<Participation process>
Next, the processing or operation of this embodiment will be described with reference to FIGS. 22 to 27. FIG. First, processing for joining a specific communication session will be described with reference to FIGS. 22 and 23. FIG. FIG. 22 is a sequence diagram showing the process of joining a specific communication session. FIG. 23 is a diagram showing a communication session (virtual conference room) selection screen.

First, when a user of site A (for example, user A1) performs an operation to display a communication session (virtual conference room) selection screen on video conference terminal 3, reception unit 32 displays the selection screen. Upon receiving the operation, the display control unit 34 displays a selection screen as shown in FIG. 23 on the display 4 (step S21). On this selection screen, selection buttons b1, b2, b3, etc., indicating virtual conference rooms R1, R2, R3, etc., to be selected are displayed. Each session ID is associated with each selection button b1 and the like.

Here, when the user A1 selects a desired virtual conference room selection button (here, the selection button b1), the reception unit 32 receives selection of a communication session (step S22). Then, the transmitter/receiver 31 transmits a request to participate in the virtual conference room to the communication management system 5 (step S23). This participation request includes the session ID indicating the communication session whose selection was accepted in step S22, and the IP address of the videoconference terminal 3, which is the requesting terminal. Thereby, the transmission/reception unit 51 of the communication management system 5 receives the participation request.

Next, in the session management DB 5001 (see FIG. 19), the storage/reading unit 99 stores the IP address received in step S23 in the participating terminal IP address field of the record with the same session ID as the session ID received in step S23. By adding the address, processing for joining the communication session is performed (step S24). Then, the transmission/reception unit 51 transmits a participation request response to the video conference terminal 3 (step S25). This participation request response includes the session ID received in step S23 and the result of the participation processing. As a result, the transmission/reception unit 31 of the video conference terminal 3 receives the participation request response. Hereinafter, a case where the participation processing is successful will be described.

<Management processing of image type information>
Next, the image type information management process will be described with reference to FIG. FIG. 24 is a sequence diagram showing management processing of image type information.

First, when a user at site A (for example, user A1) connects the USB cable of cradle 2a with photographing device 1a attached to videoconference terminal 3, storage/reading unit 19a of photographing device 1a The GUID of its own device (photographing device 1a) stored in the storage unit 1000a is read, and the communication unit 18a transmits the GUID of its own device to the communication unit 38 of the video conference terminal 3 (step S51). As a result, the communication unit 38 of the video conference terminal 3 receives the GUID of the imaging device 1a.

Next, the determination unit 35 of the video conference terminal 3 manages the same vendor ID and product ID as the vendor ID and product ID in the GUID received in step S51 in the imaging device management DB 3002 (see FIG. 16). The type of image is determined by determining whether or not it exists (step S52). Specifically, when the same vendor ID and product ID are managed in the imaging device management DB 3002, the determination unit 35 determines that the imaging device 1a captures a special image (here, a spherical panorama image). It is determined that it is a photographing device. On the other hand, if the same vendor ID and product ID are not managed in the image capturing apparatus management DB 3002, the determination unit 35 determines that the image capturing apparatus 1a is an image capturing apparatus that captures general images.

Next, the storage/readout unit 39 stores the IP address of its own terminal (video conference terminal 3), which is the transmission source terminal, and the determination result determined in step S52 in the image type management DB 3001 (see FIG. 15). It is stored in association with certain image type information (step S53). In this state, no image data ID is associated. The image type information is, for example, a source name determined according to a predetermined naming rule and an image type (general image, special image).

Next, the transmission/reception unit 31 transmits a request for adding image type information to the communication management system 5 (step S54). This image type information addition request includes the IP address of the terminal itself, which is the transmission source terminal, stored in step S53, and the image type information. As a result, the transmission/reception unit 51 of the communication management system 5 receives the request for adding the image type information.

Next, the storage/readout unit 59 of the communication management system 5 retrieves the corresponding session ID by searching the session management DB 5001 (see FIG. 19) using the IP address of the source terminal received in step S54 as a search key. is read out (step S55).

Next, the generator 56 generates a unique image data ID (step S56). Then, the storage/readout unit 59 stores the session ID read out in step S55, the image data ID generated in step S56, and the image data ID generated in step S54 as new records in the image type management DB 5002 (see FIG. 20). The received IP address of the transmission source terminal and the image type information are associated and stored (step S57). Then, the transmission/reception unit 51 transmits the image data ID generated in step S56 to the video conference terminal 3. FIG. As a result, the transmission/reception unit 31 of the video conference terminal 3 receives the image data ID (step S58).

Next, the storage/readout unit 39 of the video conference terminal 3 stores the IP of the own terminal (video conference terminal 3) which is the source terminal stored in step S53 in the image type management DB 3001 (see FIG. 15). The image data ID received in step S58 is stored in association with the address and image type information (step S59).

On the other hand, the transmission/reception unit 51 of the communication management system 5 transmits an additional notification of image type information to the smartphone 9, which is another communication terminal (step S60). This additional notification of image type information includes the image data ID generated in step S56, and the IP address and image type information of the own terminal (video conference terminal 3) that is the transmission source terminal stored in step S53. is Thereby, the transmitting/receiving unit 91 of the smartphone 9 receives the additional notification of the image type information. The transmission destination of the transmission/reception unit 51 is another IP address associated with the same session ID as the IP address of the video conference terminal 3 in the session management DB 5001 (see FIG. 19). That is, the destination is another communication terminal in the same virtual conference room as the video conference terminal 3 .

Next, the storage/readout unit 99 of the smartphone 9 stores the image data ID, the IP address of the transmission source terminal, and the image type received in step S60 as a new record in the image type management DB 9001 (see FIG. 15). The information is associated and stored (step S61). Similarly, the additional communication of the image type information is transmitted to the PC 7, which is another communication terminal, and the PC 7 also stores the image type information in the image type management DB 7001 (see FIG. 15). As described above, the image type management DBs 3001, 7001, and 9001 can share the same information in each communication terminal.

<Communication processing of image data>
Communication processing of image data in a video call will be described with reference to FIGS. 25 to 28. FIG. FIG. 25 is a sequence diagram showing communication processing of image data in a video call.

First, the communication unit 18a of the photographing device 1a transmits image data obtained by photographing a subject or scenery to the communication unit 38 of the video conference terminal 3 (step S71). In this case, since the photographing device 1a is a device capable of obtaining two hemispherical images that are the basis of the omnidirectional panorama image, the image data is as shown in FIGS. , consists of data of two hemispherical images. Thereby, the communication unit 38 of the video conference terminal 3 receives the image data.

Next, the transmission/reception unit 31 of the video conference terminal 3 transmits the image data sent from the imaging device 1a to the communication management system 5 (step S72). This transmission includes an image data ID for identifying the image data to be transmitted. Also, the IP address of the video conference terminal 3 that is the transmission source is also transmitted. Thereby, the transmission/reception unit 51 of the communication management system 5 receives the image data, the image data ID and the IP address.

Next, the transmission/reception unit 51 of the communication management system 5 transmits the image data sent from the video conference terminal 3 to the smart phone 9 (step S73). This transmission includes an image data ID for identifying the image data to be transmitted. The IP address of the videoconference terminal 3, which is the transmission source, is also transmitted. Thereby, the transmission/reception unit 51 of the smartphone 9 receives the image data, the image data ID, and the IP address.

Next, in the smartphone 9, the storage/readout unit 99 searches the image type management DB 9001 (see FIG. 15) using the image data ID received in step S73 as a search key to obtain the corresponding image type information (source name) is read out (step S74). Then, the smartphone 9 performs processing for creating a predetermined area image (step S75). Here, the processing of step S75 will be described in detail using FIG. FIG. 26 is a flow chart showing processing for creating a predetermined area image.

First, based on the image type information read out in step S74, the determination unit 95 of the smartphone 9 determines that the image type of the image data received in step S73 is a special image (here, spherical panorama image). (step S71-1). If the image is a special image (YES), the storage/readout unit 99 uses the IP address of the transmission source communication terminal (here, the video conference terminal 3) as a search key to search for the corresponding display restriction parameter ( step S71-2).

Next, the determination unit 95 determines whether there is a display restriction parameter, that is, whether the display is restricted (step S71-3). If the display is restricted (YES), the determining unit 95 further determines whether the display restriction parameter limits the display size more than the default display parameter (step S71-4). If the judgment unit 95 judges that it is restricted (YES), the image/sound processing unit 93 creates an omnidirectional panorama image from the image data received in step S73. Based on this, a predetermined area image is created (step S71-5). In this case, the image/sound processing unit 93 synthesizes an icon 191 indicating an omnidirectional panorama image, which will be described later, with the predetermined area image based on image type information indicating a special image (Video_Theta).

On the other hand, in step S71-3, if the determination unit 95 determines that the display is not restricted (NO), and in step S71-4, if the determination unit 95 determines that the display is not restricted (NO ), the image/sound processing unit 93 creates an omnidirectional panorama image from the image data received in step S73, and further creates a predetermined area image based on the default display parameters (step S71-6). In this case, the image/sound processing unit 93 synthesizes an icon 191 indicating an omnidirectional panorama image, which will be described later, with the predetermined area image based on image type information indicating a special image (Video_Theta).

Further, in step S71-1, when the determination unit 95 determines that the image type of the image data is not a special image (NO), the process of step S71 ends.

Next, returning to FIG. 25, the display control unit 94 displays a predetermined area image including the icon 191 on the display 917 of the smartphone 9 (step S76). Note that when the image type information indicates that the image is a general image (Video), the image/sound processing unit 93 does not create a omnidirectional panorama image from the image data received in step S73, and the display control unit 94 displays a generic image without icons 191 included.

Next, with reference to FIG. 27, the state of the video call will be explained. FIG. 27 is an image diagram showing the state of a video call. Among them, FIG. 27(a) shows the case where the photographing device 1a is not used, and FIG. 27(b) shows the case where the photographing device 1a is used.

First, as shown in FIG. 27(a), when the camera 312 (see FIG. 11) provided in advance in the video conference terminal 3 is used without using the photographing device 1a, the angle of view is 125 degrees horizontally. , the videoconference terminal 3 must be placed at the end of the desk where each user A1 and the like can be seen. Therefore, each user A1 or the like must face the videoconference terminal 3 when speaking. In addition, since each user A1 etc. faces the video conference terminal 3, the display 4 is also placed on the video conference terminal 3 side. As a result, the users A2 and A4 who are away from the videoconference terminal 3 have to speak in a relatively loud voice because they are away from the microphone 314 (see FIG. 11), and the content displayed on the display 4 is difficult to see.

On the other hand, as shown in FIG. 27(b), when using the imaging device 1a, it is possible to obtain two hemispherical images that form the basis of the omnidirectional panorama image. The terminal 3 and the display 4 can be relatively placed in the center of the desk. As a result, since each user A1 or the like is close to the microphone 314, they can speak in a relatively low voice, and the contents displayed on the display 4 can be easily seen.

Next, a display example of the display 917 at the site B will be described with reference to FIG. 28 . FIG. 28 shows an example of a video call screen at base B. FIG. Among them, FIG. 28(a) shows the image data sent from the videoconference terminal 3 (photographing device 1a) and the photographing device 1b, without creating the omnidirectional panorama image and the predetermined area image. It shows a case of display and a case of displaying the image data sent from the PC 7 (photographing device 8) as it is. FIG. 28(b) shows a case where an omnidirectional panorama image and a predetermined area image are created from the image data sent from the video conference terminal 3 (image capturing device 1a) and the image capturing device 1b, and the PC 7 ( This shows the case where the image data sent from the photographing device 8) is displayed as it is. An image of base A is displayed in the upper left display area of the display 917, an image of base B (own base) is displayed in the upper right display area, and an image of base C is displayed in the lower left display area. is displayed. Also, in this example, since a simultaneous video call is being made with three sites, nothing is displayed in the display area on the lower right side.

If the image data sent from the photographing devices 1a and 1b capable of photographing an omnidirectional panorama image is displayed as it is, it is displayed as shown in FIG. 28(a). In this case, site A on the upper left side and site B (own site) on the upper right side are displayed as a front hemispherical image and a rear hemispherical image, respectively, as shown in FIGS. It is

On the other hand, the image/sound processing unit 93 converts the omnidirectional panoramic image from the image data sent from the imaging devices 1a and 1b capable of obtaining two hemispherical images that are the basis of the omnidirectional panoramic image. When the image is created and then the predetermined area image is created, the predetermined area image, which is a plane image, is displayed as shown in FIG. 28(b). In addition, since the site C is equipped with a photographing device 8 for obtaining a general image, the general image is displayed in both FIGS. 28(a) and (b).

Furthermore, an icon 191 is displayed at the upper left of each of the images of site A and site B (own site) to indicate that the image is a spherical panorama image. It should be noted that the display position of the icon 191 may be anywhere, such as the upper right, the lower left, and the lower right, instead of the upper left. Further, the type of icon 191 is not limited to that shown in FIG. 28(b). Further, instead of the icon 191, characters such as "omnidirectional image" may be used, or a combination of an icon and characters may be used. As shown in FIG. 28(b), an icon 192 is similarly displayed in the display area of base B as well.

Further, the user can change the predetermined area related to the predetermined area image in the same omnidirectional panorama image. That is, when the users B1 and B2 touch and move their fingers on the touch panel 921 of the smartphone 9, the reception unit 92 receives the movement of the fingers, and the display control unit 94 shifts or rotates the predetermined area image. can be reduced, expanded, or enlarged. As a result, as shown in FIG. 28(b), even if only some users A1 and A2 of site A are displayed in the initial setting (default), the predetermined area image is shifted. Users A4 and A3 can be displayed.

<Display restriction parameter setting processing>
Next, processing for setting the display restriction parameter will be described with reference to FIG. 29 . FIG. 29 is a sequence diagram showing processing for setting display restriction parameters.

First, at the transmission source communication terminal (here, the video conference terminal 3), the user A1 or the like uses the operation button 308 to set the initial value of the display restriction parameter. is received (step S101). For example, as an initial value, α (viewing angle)>120° and a state in which only enlarged state display is permitted can be mentioned. Next, the storage/readout unit 99 stores the display restriction parameter of the initial value in the display restriction management DB 3003 (step S102). After that, the video conference terminal 3 notifies the communication terminals at all sites participating in the conference of the initial value of the display restriction parameter. Display restriction setting information indicating the setting of is transmitted (step S103). This display restriction setting information includes the IP address of the transmission source communication terminal (here, the video conference terminal 3), a list of IP addresses of the transmission destination communication terminals, and the display restriction parameters set in step S101. ing. The list of IP addresses includes the IP addresses of all communication terminals (here, PC 7 and smart phone 9) other than the own terminal (here, video conference terminal) participating in the conference. Thereby, the transmission/reception unit 51 of the communication management system 5 receives the display restriction setting information.

Next, in the communication management system 5, the storage/readout unit 99 stores the IP address of the transmission source communication terminal received in step S103 and the A display restriction parameter is stored for each IP address of the destination communication terminal (step S105). In this case, if the display restriction parameter is already stored for the same pair of the IP address of the communication terminal of the transmission source and the IP address of the communication terminal of the transmission destination, the storage/reading unit 99 overwrites and updates.

Next, the transmission/reception unit 51 of the communication management system 5 transmits display restriction setting information indicating display restriction settings to the smartphone 9 (step S106). This display restriction setting information includes the IP address of the transmission source communication terminal (here, the video conference terminal 3) and the display restriction parameter set in step S101. Thereby, the transmission/reception unit 91 of the smartphone 9 receives the display restriction setting information. Then, in the smartphone 9, the storage/readout unit 99 associates the IP address of the transmission source communication terminal and the display restriction parameter received in step S105, and stores them in the display restriction management DB 9004 (step S106). This completes setting of the display restriction parameter requested by the video conference terminal 3 on the smartphone 9 .

Similarly, the transmission/reception unit 51 of the communication management system 5 transmits display restriction setting information indicating display restriction settings to the PC 7 (step S107). This display restriction setting information includes the IP address of the transmission source communication terminal (here, the video conference terminal 3) and the display restriction parameter set in step S101. Thereby, the transmission/reception unit 71 of the PC 7 receives the display restriction setting information. Then, in the PC 7, the storage/readout unit 79 associates the IP address of the transmission source communication terminal and the display restriction parameter received in step S107, and stores them in the display restriction management DB 7004 (step S108). This completes the setting of the display restriction parameter requested by the video conference terminal 3 on the PC 7 .

With the above, the display restriction parameter setting process is completed.

<Display restriction parameter setting release processing (restricted side)>
Next, processing for canceling the setting of the display restriction parameter at the request of the restricted side will be described with reference to FIG. 30 . FIG. 30 is a sequence diagram showing processing for canceling display restriction parameter settings in response to a request from the restricted party.

For example, when the user of the smartphone 9 on the restricted side wants to release the restricted display restriction parameter from the video conference terminal 3, when the user of the smartphone 9 performs an operation for releasing the display restriction, the smartphone 9 accepts the cancellation of the display restriction (step 121). For example, the user B1 of the smartphone 9 selects the display area of the image for which the display restriction is to be lifted (here, the upper left display where the user A1 etc. is displayed) on the screen shown in FIG. 28(b). area).

Next, the transmitting/receiving unit 91 transmits display restriction release request information indicating a request to release the display restriction to the communication management system 5 (step S122). This display restriction release request information includes the IP address of the request source communication terminal (here, the smartphone 9) and the IP address of the request destination communication terminal (here, the video conference terminal 3). Note that the request source communication terminal is the transmission destination communication terminal that has received the display restriction setting. Also, the communication terminal of the request destination is the communication terminal of the transmission source that has set the display restriction. Thereby, the transmission/reception unit 51 of the communication management system 5 receives the display restriction release request information. Then, the transmission/reception unit 51 transmits the display restriction release request information received in step S122 to the video conference terminal 3 (step S123). This display restriction release request information includes the IP address of the request source communication terminal (here, the smartphone 9) received in step S122. As a result, the transmission/reception unit 31 of the video conference terminal 3 receives the display restriction release request information.

Next, the video conference terminal 3 executes processing for temporarily canceling the display restriction (step S124). For example, the storage/reading unit 39 searches the display restriction management DB 3003 using the IP address of the request source (destination) communication terminal (here, the smart phone 9) received in step S123 as a search key. Temporarily remove the display limit parameter. In this case, the image/sound processing unit 33 outputs a notification sound or notifies as shown in FIG. A screen 351 may be displayed. FIG. 32 is a screen example of a video call including a notification screen. In addition, the video conference terminal 3 may automatically cause the storage/readout unit 39 to temporarily delete the display restriction parameter after a certain period of time (for example, 10 seconds) after outputting the notification sound. Also, the temporarily deleted display restriction parameter is restored after a predetermined time (for example, 10 minutes) from its deletion, or is restored by a restoration instruction from the transmission source communication terminal that instructed the display restriction.

Further, as the process of step S124, the display control unit 34 instructs the user A1 or the like of the requested communication terminal (here, the video conference terminal 3) to cancel the display restriction, as shown in FIG. Approval request screen 352 may be displayed to request approval. FIG. 33 is a screen example of a video call including an approval request screen.

This approval request screen 352 includes a "Yes" button that is pressed for approval and a "No" button that is pressed for non-approval. If approved, the storage/readout unit 39 searches the display restriction management DB 3003 using the IP address of the request source (destination) communication terminal (here, the smartphone 9) received in step S123 as a search key. to temporarily delete the corresponding display restriction parameter. On the other hand, if the request is not approved, the transmitting/receiving unit 31 notifies the requesting communication terminal (here, the smartphone 9) via the communication management system 5 that the request is not approved (rejected).

Next, the transmission/reception unit 31 of the video conference terminal 3 transmits display restriction release instruction information indicating that the display restriction has been released to the communication management system 5 (step S125). This display restriction release instruction information includes the IP address of the request destination (transmission source) communication terminal, which is the own terminal (here, the video conference terminal 3), and the request source (destination) terminal IP address. . As a result, the transmission/reception unit 51 of the communication management system 5 receives the display restriction release instruction information.

Next, in communication management system 5, storage/readout unit 59 searches all display restriction management DB 5003 using the set of both IP addresses received in step S125 as a search key, and temporarily stores the corresponding display restriction parameter. By deleting, the restriction is released (step S126). Then, the transmitting/receiving unit 51 transmits display restriction release instruction information indicating that the display restriction has been released to the smartphone 9 . This display restriction release instruction information includes the IP address of the communication terminal of the request destination (source) received in step S125. Thereby, the transmission/reception unit 91 of the smartphone 9 receives the display restriction release instruction information.

Next, the smartphone 9 performs a process of canceling the display restriction (step S128). For example, the storage/readout unit 99 searches the display restricted management DB 3004 using the IP address of the communication terminal (here, the video conference terminal 3) of the request destination (source) received in step S127 as a search key. , temporarily removes the corresponding visibility limit parameter. As a result, in step 71-3 shown in FIG. 26, it is determined that the smartphone 9 is not subject to display restrictions, and the image/sound processing unit 93 displays the display 917 according to the operations of the users B1 and B2. can display a predetermined area image.

Thus, the display restriction parameter setting cancellation process (restricted side) is completed.

<Display restriction parameter setting release processing (restriction side)>
Next, processing for individually canceling the setting of display restriction parameters according to instructions from the restricting side will be described with reference to FIG. 35 . FIG. 31 is a sequence diagram showing a process of canceling display restriction parameter settings in response to a request from the restriction side.

For example, the display control unit 34 displays "Release display restriction" buttons 361 and 362 as shown in FIG. 34 during the video call. Then, when the user A1 or the like of the video conference terminal 3 on the restricted side wants to release the display restriction only for the site B out of the sites B and C, the user A1 or the like presses the "Release display restriction" button 361. When pressed, the receiving unit 32 of the video conference terminal 3 receives cancellation of the display restriction on the site B (step 201). As a result, the storage/reading unit 39 temporarily deletes the corresponding display restriction parameter by searching the display restriction management DB 3003 using the IP address of the communication terminal of the selected site B as a search key, thereby reducing the display The restriction is released (step S202). Then, the video conference terminal 3 notifies the communication terminal (here, the smart phone 9) of the site B participating in the conference that the display restriction has been lifted. Then, display restriction release information indicating release of display restriction is transmitted (step S203). This display restriction release information includes the IP address of the transmission source communication terminal (here, the video conference terminal 3) and the IP address of the transmission destination communication terminal that is the destination of the display release. Thereby, the transmission/reception unit 51 of the communication management system 5 receives the display restriction release information.

Next, in the communication management system 5, the storage/readout unit 99 stores the combination of the IP address of the communication terminal of the transmission source and the IP address of the communication terminal of the transmission destination received in step S203 in the all display restriction management DB 5003. By temporarily deleting the display restriction parameter corresponding to , restriction release processing is performed (step S204).

Next, the transmission/reception unit 51 of the communication management system 5 transmits display restriction release information indicating release of display restriction to the smartphone 9 (step S205). This display restriction release information includes the IP address of the transmission source communication terminal (here, the video conference terminal 3) from which the display restriction is released. Thereby, the transmission/reception unit 91 of the smartphone 9 receives the display restriction release information. Then, in the smartphone 9, the storage/readout unit 99 searches the display-restricted management DB 9004 using the IP address of the transmission source communication terminal received in step S205 as a search key, thereby temporarily storing the corresponding display restriction parameter. In step S206, the restriction cancellation process is performed. As a result, the smartphone 9 releases the display restriction requested by the video conference terminal 3, the PC 7 does not release the display restriction, and the display restriction parameter setting release processing (restriction side) ends.

<Display restriction parameter setting change processing (restriction side)>
Next, processing for canceling the setting of the display restriction parameter according to an instruction from the restricting side will be described with reference to FIG. 35 . FIG. 35 is a sequence diagram showing a process of changing display restriction parameter settings according to an instruction from the restriction side.

FIG. 36 is a screen example of a video call including a change screen. For example, when user A1 operates the operation button 308 during a video call, the display control unit 34 displays a display restriction change screen 371 as shown in FIG. When the user A1 selects whether or not to restrict the display of each of the bases B and C by operating the operation button 308 and presses the "OK" button, the video conference terminal 3 The reception unit 32 receives the presence or absence of display restrictions for each site (step 221). Here, a case is shown in which the display of the site B is restricted and the display of the site C is not restricted. As a result, the storage/reading unit 39 temporarily deletes the corresponding display restriction parameter by searching the display restriction management DB 3003 using the IP address of the communication terminal of the selected site C as a search key, thereby enabling the display The presence or absence of restrictions is collectively changed (step S222). In addition, since the display restriction is continued for the site B, the storage/reading unit 39 does not temporarily delete the display restriction parameter of the site B but maintains it.

Next, the transmission/reception unit 31 transmits display restriction change information indicating a change in display restriction to the communication management system 5 (step S223). This display restriction change information includes a list of IP addresses including the IP address of the transmission source communication terminal (here, the video conference terminal 3), the IP address of the transmission destination communication terminal that is the destination of the display restriction change, and display Display restriction release instruction information indicating an instruction to release the restriction and display restriction reset instruction information indicating an instruction to reset the display restriction are included. In this case, display restriction release instruction information or display restriction reset instruction information is associated with each IP address of the destination communication terminal. Thereby, the transmission/reception unit 51 of the communication management system 5 receives the display restriction change information.

Next, in the communication management system 5, the storage/reading unit 99 sends the all display restriction management DB 5003 to the communication terminal of the transmission destination associated with the display restriction release instruction information in step S223. By temporarily deleting the display restriction parameter corresponding to the set of the IP address of the communication terminal of the transmission source and the IP address of the communication terminal of the transmission destination, restriction change processing is performed (step S224). Note that the storage/reading unit 99 temporarily maintains the display restriction parameters in the all display restriction management DB 5003 without deleting them for the destination communication terminal associated with the display restriction release instruction information. leave it as is.

Next, the transmission/reception unit 51 of the communication management system 5 transmits display restriction change information including an instruction to reset the display restriction to the smartphone 9 (step S225). This display restriction setting information includes the IP address of the transmission source communication terminal (here, the video conference terminal 3). Thereby, the transmission/reception unit 91 of the smartphone 9 receives the display restriction change information. Then, in the smartphone 9, the storage/readout unit 99 performs a process of resetting the display restriction parameter for the IP address of the transmission source communication terminal received in step S205 in the display restriction management DB 9004 (step S226). In addition, in the case of resetting, the communication management system 5 does not have to transmit the display restriction change information to the communication terminal.

Further, the transmission/reception unit 51 of the communication management system 5 transmits display restriction change information including an instruction to cancel the display restriction to the PC 7 (step S227). This display restriction change information includes the IP address of the transmission source communication terminal (here, the video conference terminal 3) from which the display restriction is released. Thereby, the transmission/reception unit 71 of the PC 7 receives the display restriction change information. Then, in PC 7, storage/reading unit 79 searches display-restricted management DB 7004 using the IP address of the transmission source communication terminal received in step S227 as a search key, thereby temporarily storing the corresponding display restriction parameter. Processing for releasing the restriction to be deleted is performed (step S228). As a result, in the PC 7, the display restriction instructed by the video conference terminal 3 is released, and the display restriction parameter setting change process (restriction side) ends.

<<main effects of the present embodiment>>
As described above, according to the present embodiment, the communication terminal such as the video conference terminal 3 creates an omnidirectional panorama image based on the image data ID sent together with the image data and the corresponding image type information. Furthermore, a predetermined area image can be created. As a result, as shown in FIG. 28A, it is possible to prevent the hemispherical image on the front side and the hemispherical image on the rear side from being displayed.

Further, the communication terminal may release the display restriction upon request from another communication terminal (see steps S121 to S128), or actively release the display restriction for a specific other communication terminal from its own terminal ( Steps S201 to S206), the own terminal actively resets (or maintains) or cancels the display restriction on each of the other communication terminals (steps S221 to S228). As a result, it is possible to flexibly restrict and cancel the display, so that the user of the other communication terminal who has once received the display restriction can cancel the display restriction when he/she wants to roughly grasp the location of the communication terminal. can be dealt with.

1a photographing device 1b photographing device 3 video conference terminal (an example of a communication terminal)
5 Communication management system 7 PC (an example of a communication terminal)
8 Photographing device 9 Smartphone (an example of a communication terminal)
31 transmission/reception unit (an example of transmission means, an example of reception means)
32 reception unit (an example of reception means)
34 display control unit (an example of display control means)
71 Transmitting/receiving unit (an example of transmitting means, an example of receiving means)
72 reception unit (an example of reception means)
74 display control unit (an example of display control means)
91 transmission/reception unit (an example of transmission means, an example of reception means)
92 reception unit (an example of reception means)
94 Display control unit (an example of display control means)

JP 2012-178135 A JP 2011-223076 A JP-A-2002-034009 JP-A-2001-148850

The invention according to claim 1 is a communication terminal that creates a predetermined area image of an omnidirectional panorama image from image data received from another communication terminal, and determining means for determining whether or not display is restricted. and display control means for displaying a predetermined area image of an omnidirectional panorama image from the received image data based on the judgment result of the judgment means.

Claims (10)

A communication terminal that transmits spherical image data to be received by another communication terminal,
so that the other communication terminal receives a display restriction parameter for restricting a display area within a predetermined area of a part of the omnidirectional image when the other communication terminal displays the omnidirectional image. having transmission means for transmitting to
The transmission means is a communication terminal configured to transmit display restriction release instruction information indicating an instruction to release restriction of the display area so that the other communication terminal receives the display restriction release instruction information. A communication terminal according to claim 1, further comprising:
receiving means for receiving display restriction release request information, which is transmitted from the other communication terminal and indicates a request to release the restriction of the display area;
The transmission means is a communication terminal that transmits the display restriction release instruction information so that the other communication terminal receives it. The communication terminal according to claim 2, further comprising:
display control means for displaying on a predetermined display means that the limitation of the display area is to be released;
The transmission means is a communication terminal that transmits the display restriction release instruction information in response to reception of the display restriction release request information. The communication terminal according to claim 2, further comprising:
display control means for displaying, on a predetermined display means, an approval request screen for selecting whether or not to approve a request related to the display restriction cancellation request information;
The communication terminal, wherein the transmission means transmits the display restriction release instruction information when the selection to approve is made. A communication terminal according to claim 1, further comprising:
receiving means for receiving cancellation of the display area restriction for each of a plurality of other communication terminals;
A communication terminal in which the transmission means transmits the display restriction release instruction information so that a specific other communication terminal for which the release has been accepted by the acceptance means receives the display restriction release instruction information. A communication terminal according to claim 1, further comprising:
receiving means for receiving release or non-release of the restriction of the display area for each of a plurality of other communication terminals;
The transmission means transmits the display restriction release instruction information so that a specific other communication terminal that has accepted the release among the plurality of other communication terminals accepted by the reception means receives the display restriction release instruction information. A communication terminal according to any one of claims 1 to 6;
a communication management system that relays communication between the communication terminal and the other communication terminal;
a communication system having The other communication terminal is
transmitting display restriction release request information indicating a request to release the display area restriction to the communication terminal via the communication management system;
8. The communication system according to claim 7, wherein display restriction release instruction information indicating an instruction to release the restriction of the display area is received from the communication terminal via the communication management system. A communication method performed by a communication terminal that transmits spherical image data to be received by another communication terminal,
The communication terminal is
so that the other communication terminal receives a display restriction parameter for restricting a display area within a predetermined area of a part of the omnidirectional image when the other communication terminal displays the omnidirectional image. send to
A communication method for transmitting display restriction release instruction information indicating an instruction to release restriction of the display area so as to be received by the other communication terminal. A program for causing a computer to execute the method according to claim 9.