JP7358938B2 - Communication terminal, image communication system, display method, and program - Google Patents

Description

The present disclosure relates to an invention that displays a predetermined area image that is a predetermined area in an image.

In recent years, the range of photography and display has expanded by displaying equirectangular cylindrical projection images obtained by photographing with a spherical (360°) camera on a communication terminal such as a smartphone. When displaying an equirectangular projection image on a communication terminal, the entire image can be displayed as a spherical image. It is possible to display a predetermined area image showing a predetermined area of . In this case, the user of the communication terminal can display a predetermined region image showing another predetermined region of the same spherical image by performing a swipe operation on the predetermined region image displayed on the communication terminal. (See Patent Document 1). This operation and display will be explained in detail below using FIGS. 26 to 29.

26 to 29 respectively relate to conventional display examples, (a) is a diagram showing a predetermined area image in a plurality of equirectangular projection images (here, the state of equirectangular projection images), (b) ) is a diagram showing a corresponding predetermined area image before operating the smartphone, and (c) is a diagram showing a corresponding predetermined area image after operating the smartphone. Here, a case is shown in which the equirectangular projection image EC1 is first read out, and then the equirectangular projection image EC2 is read out.

First, when the user displays a predetermined region image of a spherical image as an equirectangular projection image on the smartphone 30, as shown in FIG. A predetermined area image pa11 of the predetermined area is displayed. In this predetermined area image pa11, as shown in FIG. It is displayed so as to be located at the starting end BE1 of the equirectangular projection image EC1. In this state, when the user performs a predetermined swipe operation to the left on the smartphone 30, the smartphone 30 displays the predetermined area image on the right side of the predetermined area image p11, as shown in FIG. 26(c). Display pa12. This predetermined area image pa12, as shown in FIG. 26(a), shows a substantially central area of the equirectangular projection image EC1.

Next, as shown in FIG. 27(b), when the user further performs a swipe operation in the same direction (to the left) on the smartphone 30 on which the predetermined area image pa12 is displayed, the smartphone 30 , the predetermined area image pa13 is displayed, as shown in FIG. 27(c). In this case, as shown in FIG. 27(a), the tip a13 in the image movement direction (leftward) due to the swipe operation (rightward) in the predetermined area image pa13 is the terminal end TE1 of the equirectangular projection image EC1. It is displayed as being located at .

Next, as shown in FIG. 28(b), when the user further performs a predetermined swipe operation (leftward) in the same direction on the smartphone 30 on which the predetermined area image ps13 is displayed, the user 30 displays a predetermined area image pa14, as shown in FIG. 28(c). As shown in FIG. 28(a), this predetermined area image pa14 is composed of a left predetermined area image pa14a and a right predetermined area image pa14b after one rotation of the equirectangular projection image EC1. . In this way, when the user continues the swipe operation, the next equirectangular projection image EC2 is not read out, and the next equirectangular projection image EC2 is circulated within the equirectangular projection image EC1.

If the user wishes to read out the next equirectangular projection image EC2 and display the predetermined area image pa21, the user presses the "Next" button n1, as shown in FIG. 29(b). Thereby, the smartphone 30 displays the predetermined area image pa21, as shown in FIG. 29(c). In this predetermined area image pa21, as shown in FIG. It is displayed so as to be located at the starting end BE2 of the equirectangular projection image EC2. Note that since the predetermined area image revolves on the equirectangular projection image, the terminal end TE1 of the equirectangular projection image EC1 is also the starting end BE1 of the equirectangular projection image EC1.

However, in the conventional method, when a user performs a swipe operation on a spherical image, the displayed predetermined area image only revolves within an equirectangular projection image (an example of a wide-angle image). When reading and displaying the equirectangular projection image as the next spherical image, it was necessary to perform another operation such as pressing the "next" button n1. For example, when a user moves a predetermined area image with the momentum of moving a finger, such as in a swipe operation, it may feel strange to stop the finger during the swipe operation and perform another operation. However, there was a problem of poor operability.

The invention according to claim 1 is a communication terminal for displaying a first predetermined area image, which is a predetermined area in an arbitrary wide-angle image, on a display means, wherein the first predetermined area image being displayed on the display means is A reception means for accepting movement, and the reception means accept movement of the first predetermined area image in a predetermined movement direction over a certain area in the arbitrary wide-angle image, and further allow movement in the movement direction. The communication terminal is characterized in that it has a display control means for causing the display means to display a second predetermined area, which is a predetermined area in the next wide-angle image, when the communication terminal is accepted.

As explained above, according to the present invention, the user can move a predetermined area image in a predetermined moving direction and then move it in the same moving direction, so that the user can move the image to the next wide-angle image by repeating the same operation. can be displayed. This has the effect of improving operability.

(a) is a left side view of the photographing device, (b) is a rear view of the photographing device, (c) is a plan view of the photographing device, and (d) is a bottom view of the photographing device. It is a usage image diagram of a photographing device. (a) is a hemispherical image taken by the imaging device (front), (b) is a hemispherical image taken by the imaging device (back), and (c) is an image expressed by equirectangular projection. be. (a) is a conceptual diagram showing a state where a sphere is covered by an equirectangular projection image, and (b) is a diagram showing a celestial sphere image. FIG. 7 is a diagram showing the positions of a virtual camera and a predetermined area when a celestial sphere image is a three-dimensional solid sphere. (a) is a three-dimensional perspective view of FIG. 5, and (b) is a diagram showing a state in which an image of a predetermined area is displayed on a display of a communication terminal. 3 is a diagram showing a relationship between predetermined area information and an image of a predetermined area T. FIG. 1 is a schematic diagram of an image communication system according to a first embodiment. FIG. 2 is a hardware configuration diagram of a photographing device. It is a hardware configuration diagram of a smartphone. FIG. 1 is a functional block diagram of an image communication system according to a first embodiment. FIG. 2 is a sequence diagram showing communication processing of equirectangular projection image data according to the first embodiment. 5 is a flowchart showing a process for displaying a predetermined area image according to the first embodiment. According to the first embodiment, (a) is a diagram showing a predetermined area image in a plurality of equirectangular projection images, (b) is a diagram showing a corresponding predetermined area image before operating the smartphone, (c) is a diagram showing a corresponding predetermined area image after the smartphone is operated. According to the first embodiment, (a) is a diagram showing a predetermined area image in a plurality of equirectangular projection images, (b) is a diagram showing a corresponding predetermined area image before operating the smartphone, (c) is a diagram showing a corresponding predetermined area image after the smartphone is operated. According to the first embodiment, (a) is a diagram showing a predetermined area image in a plurality of equirectangular projection images, (b) is a diagram showing a corresponding predetermined area image before operating the smartphone, (c) is a diagram showing a corresponding predetermined area image after the smartphone is operated. According to the first embodiment, (a) is a diagram showing a predetermined area image in a plurality of equirectangular projection images, (b) is a diagram showing a corresponding predetermined area image before operating the smartphone, (c) is a diagram showing a corresponding predetermined area image after the smartphone is operated. 7 is a flowchart illustrating display processing of a predetermined area image according to the second embodiment. According to the second embodiment, (a) is a diagram showing a predetermined area image in a plurality of equirectangular projection images, (b) is a diagram showing a corresponding predetermined area image before operating the smartphone, (c) is a diagram showing a corresponding predetermined area image after the smartphone is operated. According to the second embodiment, (a) is a diagram showing a predetermined area image in a plurality of equirectangular projection images, (b) is a diagram showing a corresponding predetermined area image before operating the smartphone, (c) is a diagram showing a corresponding predetermined area image after the smartphone is operated. FIG. 7 is a diagram illustrating another example of the second embodiment, showing a passing region and a predetermined region in an equirectangular projection image. FIG. 2 is a schematic diagram of an image communication system according to a third embodiment.

FIG. 2 is a hardware configuration diagram of an image management server. FIG. 3 is a functional block diagram of an image communication system according to a third embodiment. FIG. 7 is a sequence diagram showing communication processing of equirectangular projection image data according to the third embodiment. Regarding conventional display examples, (a) is a diagram showing a predetermined area image in a plurality of equirectangular projection images, (b) is a diagram showing a corresponding predetermined area image before operating the smartphone, and (c) is a diagram showing a corresponding predetermined area image before operating the smartphone. It is a figure showing a corresponding predetermined area image after operation of a smartphone. FIG. 7 is a conceptual diagram of a wide-angle image when a narrow-angle image is cut out using only the importance level, ignoring the composition. Regarding conventional display examples, (a) is a diagram showing a predetermined area image in a plurality of equirectangular projection images, (b) is a diagram showing a corresponding predetermined area image before operating the smartphone, and (c) is a diagram showing a corresponding predetermined area image before operating the smartphone. It is a figure showing a corresponding predetermined area image after operation of a smartphone. FIG. 7 is a conceptual diagram of a wide-angle image when a narrow-angle image is cut out using only the importance level, ignoring the composition. Regarding conventional display examples, (a) is a diagram showing a predetermined area image in a plurality of equirectangular projection images, (b) is a diagram showing a corresponding predetermined area image before operating the smartphone, and (c) is a diagram showing a corresponding predetermined area image before operating the smartphone. It is a figure showing a corresponding predetermined area image after operation of a smartphone. FIG. 7 is a conceptual diagram of a wide-angle image when a narrow-angle image is cut out using only the importance level, ignoring the composition. Regarding conventional display examples, (a) is a diagram showing a predetermined area image in a plurality of equirectangular projection images, (b) is a diagram showing a corresponding predetermined area image before operating the smartphone, and (c) is a diagram showing a corresponding predetermined area image before operating the smartphone. It is a figure showing a corresponding predetermined area image after operation of a smartphone. FIG. 7 is a conceptual diagram of a wide-angle image when a narrow-angle image is cut out using only the importance level, ignoring the composition.

Embodiments of the present invention will be described below with reference to the drawings.

[Overview of embodiment]
An overview of this embodiment will be described below.

First, a method for generating a spherical image will be explained using FIGS. 1 to 7.

First, the appearance of the photographing device 1 will be explained using FIG. 1. The photographing device 1 is a digital camera for obtaining a photographed image that is the basis of a spherical (360°) panoramic image. Note that FIG. 1(a) is a left side view of the imaging device, FIG. 1(b) is a rear view of the imaging device, FIG. 1(c) is a plan view of the imaging device, and FIG. 1(d) is a top view of the imaging device. is a bottom view of the photographing device.

As shown in FIG. 1(a), FIG. 1(b), FIG. 1(c), and FIG. A fisheye lens 102b is provided on the back side (rear side). Inside the photographing device 1, image sensors (image sensors) 103a and 103b, which will be described later, are provided, and by photographing a subject or scenery through lenses 102a and 102b, respectively, a hemispherical image (an angle of view of 180° or more) is provided. ) can be obtained. A shutter button 115a is provided on the surface of the photographing device 1 opposite to the front side. Further, on the side surface of the photographing device 1, a power button 115b, a Wi-Fi (Wireless Fidelity) button 115c, and a photographing mode switching button 115d are provided. The power button 115b and the Wi-Fi button 115c are both turned on and off each time they are pressed. Furthermore, each time the shooting mode switching button 115d is pressed, the shooting mode for still images and the shooting mode for moving images are switched. Note that the shutter button 115a, the power button 115b, the Wi-Fi button 115c, and the shooting mode switching button 115d are part of the operation unit 115, and the operation unit 115 is not limited to these buttons.

Further, a tripod screw hole 151 for attaching the photographing device 1 to a camera tripod is provided at the center of the bottom portion 150 of the photographing device 1. Further, a Micro USB (Universal Serial Bus) terminal 152 is provided on the left end side of the bottom portion 150. On the right end side of the bottom portion 150, an HDMI (High-Definition Multimedia Interface) terminal 153 is provided. Please note that HDMI is a registered trademark.

Next, using FIG. 2, the usage status of the photographing device 1 will be explained. Note that FIG. 2 is an image diagram of how the photographing device is used. As shown in FIG. 2, the photographing device 1 is, for example, held by a user and used to photograph objects around the user. In this case, two hemispherical images can be obtained by capturing images of objects around the user using the imaging device 103a and the imaging device 103b shown in FIG. 1, respectively.

Next, using FIGS. 3 and 4, an outline of the processing from the image photographed by the photographing device 1 to the creation of the equirectangular projection image EC and the celestial sphere image CE will be described. In addition, FIG. 3(a) is a hemispherical image taken with the imaging device 1 (front side), FIG. 3(b) is a hemispherical image taken with the imaging device (rear side), and FIG. 3(c) is an equirectangular projection. FIG. 2 is a diagram showing an image represented by (hereinafter referred to as "equirectangular projection image"). FIG. 4(a) is a conceptual diagram showing a state where a sphere is covered by an equirectangular projection image, and FIG. 4(b) is a diagram showing a spherical image. Note that the equirectangular projection image is an example of a wide-angle image.

As shown in FIG. 3A, the image obtained by the image sensor 103a becomes a hemispherical image (front side) curved by a fisheye lens 102a, which will be described later. Further, as shown in FIG. 3(b), the image obtained by the image sensor 103b becomes a hemispherical image (rear side) curved by a fisheye lens 102b, which will be described later. The hemispherical image (front side) and the hemispherical image (rear side) inverted by 180 degrees are combined by the imaging device 1, and as shown in FIG. 3(c), an equirectangular projection image EC is obtained. Created.

Then, by using OpenGL ES (Open Graphics Library for Embedded Systems), the equirectangular projection image is pasted to cover the spherical surface, as shown in Figure 4(a). A spherical image CE as shown in b) is created. In this way, the spherical image CE is represented as an image in which the equirectangular projection image EC faces the center of the sphere. Note that OpenGL ES is a graphics library used to visualize 2D (2-Dimensions) and 3D (3-Dimensions) data. Note that the omnidirectional image CE may be a still image or a moving image. Further, although an example has been shown in which an equirectangular projection image is pasted onto the three-dimensional sphere CS, it may be not only the three-dimensional sphere CS but also a cube or a prism.

As described above, since the spherical image CE is an image pasted so as to cover a spherical surface, when viewed by a human being, it is distorted and feels strange. Therefore, by displaying a predetermined area (hereinafter referred to as a "predetermined area image") of a part of the spherical image CE as a flat image with less curvature, it is possible to perform a display that does not give a sense of discomfort to humans. This will be explained using FIGS. 5 and 6.

Note that FIG. 5 is a diagram showing the positions of the virtual camera and the predetermined area when the omnidirectional image is a three-dimensional solid sphere. The virtual camera IC corresponds to the position of the viewpoint of the user who views the spherical image CE displayed as a three-dimensional solid sphere. Further, FIG. 6(a) is a three-dimensional perspective view of FIG. 5, and FIG. 6(b) is a diagram showing a predetermined area image when displayed on a display. Furthermore, in FIG. 6(a), the omnidirectional spherical image CE shown in FIG. 4 is pasted onto a three-dimensional solid sphere CS. When the spherical image CE generated in this way is pasted on the three-dimensional sphere CS, the virtual camera IC is located inside the spherical image CE, as shown in FIG. There is. The predetermined area T in the spherical image CE is a shooting area of the virtual camera IC, and is specified by the predetermined area information indicating the shooting direction and angle of view of the virtual camera IC in the three-dimensional virtual space including the spherical image CE. Ru.

The predetermined area image Q shown in FIG. 6(a) is displayed on a predetermined display as an image of the imaging area of the virtual camera IC, as shown in FIG. 6(b). The image shown in FIG. 6(b) is a predetermined area image represented by initially set (default) predetermined area information. In the following, description will be given using the photographing direction (θ, φ) and angle of view (α) of the virtual camera IC. Note that the shooting direction is also referred to as the line-of-sight direction.

The relationship between the predetermined area information and the image of the predetermined area T will be explained using FIG. 7. Note that FIG. 7 is a diagram showing the relationship between the predetermined area information and the image of the predetermined area T. As shown in FIG. 7, "θ" represents the pan angle of the virtual camera IC in the photographing direction, "φ" represents the tilt angle of the virtual camera IC in the photographing direction, and "α" represents the angle of view. Further, "W" indicates the width of the predetermined area T, and "H" indicates the height of the predetermined area H. That is, the attitude of the virtual camera IC is changed so that the point of view of the virtual camera IC indicated by the shooting direction (θ, φ) becomes the center point CP of the predetermined area T that is the shooting area of the virtual camera IC. Become. The predetermined area image Q is an image of a predetermined area T in the omnidirectional image CE. f is the distance from the virtual camera IC to the center point CP. L is the distance between an arbitrary vertex of the predetermined region T and the center point CP (2L is a diagonal line). In FIG. 7, the trigonometric function expressed by the following (Equation 1) generally holds true.

L/f=tan(α/2)...(Formula 1)
●First Embodiment Next, the first embodiment of the present invention will be described using FIGS. 8 to 17.

[Outline of the configuration of the image communication system]
First, the outline of the configuration of the image communication system according to the first embodiment will be described using FIG. 8. FIG. 8 is a schematic diagram of an image communication system according to the first embodiment.

As shown in FIG. 8, the image communication system of this embodiment is constructed by a photographing device 1 and a smartphone 3. The photographing device 1 and the smartphone 3 can wirelessly communicate using short-range wireless communication technology such as Wi-Fi, Bluetooth (registered trademark), and NFC (Near Field Communication).

Among these, the photographing device 1 is a special digital camera for photographing a subject, scenery, etc. to obtain two hemispherical images that are the basis of an equirectangular projection image, as described above. Note that a photographing device (such as a single-lens reflex camera, a compact digital camera, a smartphone, a tablet PC, etc.) for obtaining a general wide-angle planar image instead of an equirectangular projection image may be used, but in this embodiment, the main The case of a digital camera for obtaining an equirectangular projection image will be explained below. Note that here, the wide-angle image is generally an image photographed using a wide-angle lens, and is one that is photographed with a lens that can photograph a wider range than that perceived by the human eye. Also, generally refers to images taken with a lens with a focal length of 35mm or less in 35mm film equivalent.

The smartphone 3 is a PC (Personal Computer) used by the user A. The smartphone 3 is an example of a communication terminal. Note that the photographing device 1 and the smartphone 3 may perform wired communication using a USB (Universal Serial Bus) cable or the like instead of performing wireless communication.

The smartphone 3 is an example of a communication terminal that receives spherical image data created by the photographing device 1 after photographing a subject or the like through wireless communication or the like, and displays an image of a predetermined area of the spherical image. In addition to the smartphone 3, the communication terminal includes a PC, a smart watch, a display with a display application installed, a game console, a car navigation device, a wearable terminal, and the like.

[Hardware configuration of embodiment]
Next, the hardware configurations of the photographing device 1 and the smartphone 3 of this embodiment will be described in detail using FIGS. 9 and 10.

<Hardware configuration of imaging device>
First, the hardware configuration of the imaging device 1 will be explained using FIG. 9. FIG. 9 is a hardware configuration diagram of the imaging device 1. In the following, the photographing device 1 is assumed to be a spherical (omnidirectional) photographing device using two image sensors, but the number of image sensors may be any number greater than or equal to two. Further, the device does not necessarily have to be dedicated to omnidirectional photography, and may have substantially the same functions as the photographing device 1 by attaching an aftermarket omnidirectional imaging unit to a normal digital camera or the like.

As shown in FIG. 9, the imaging device 1 includes an imaging unit 101, an image processing unit 104, an imaging control unit 105, a microphone 108, a sound processing unit 109, a CPU (Central Processing Unit) 111, and a ROM (Read Only Memory). ) 112, SRAM (Static Random Access Memory) 113, DRAM (Dynamic Random Access Memory) 114, operation unit 115, network I/F 116, communication unit 117, antenna 117a, electronic compass 118, gyro sensor 119, acceleration sensor 120, It is configured by a terminal 121.

Among these, the imaging unit 101 includes wide-angle lenses (so-called fisheye lenses) 102a and 102b each having an angle of view of 180° or more for forming a hemispherical image, and two imaging units provided corresponding to each wide-angle lens. It includes elements 103a and 103b. The image sensors 103a and 103b are image sensors such as CMOS (Complementary Metal Oxide Semiconductor) sensors and CCD (Charge Coupled Device) sensors that convert optical images formed by fisheye lenses 102a and 102b into electrical signal image data and output the image data. It has a timing generation circuit that generates horizontal or vertical synchronization signals and pixel clocks, and a group of registers in which various commands and parameters necessary for the operation of this image sensor are set.

The imaging elements 103a and 103b of the imaging unit 101 are each connected to the image processing unit 104 via a parallel I/F bus. On the other hand, the imaging elements 103a and 103b of the imaging unit 101 are connected to the imaging control unit 105 via a serial I/F bus (such as an I2C bus). The image processing unit 104, the imaging control unit 105, and the sound processing unit 109 are connected to the CPU 111 via a bus 110. Furthermore, a ROM 112, an SRAM 113, a DRAM 114, an operation unit 115, a network I/F 116, a communication unit 117, an electronic compass 118, and the like are also connected to the bus 110.

The image processing unit 104 takes in image data output from the image sensors 103a and 103b through the parallel I/F bus, performs predetermined processing on each image data, and then synthesizes these image data. , create equirectangular cylindrical projection image data as shown in FIG. 3(c).

Generally, the imaging control unit 105 uses the I2C bus to set commands and the like in register groups of the imaging devices 103a and 103b, using the imaging control unit 105 as a master device and the imaging devices 103a and 103b as slave devices. Necessary commands and the like are received from the CPU 111. The imaging control unit 105 also uses the I2C bus to take in status data and the like of the register groups of the imaging elements 103a and 103b, and sends it to the CPU 111.

Further, the imaging control unit 105 instructs the imaging elements 103a and 103b to output image data at the timing when the shutter button of the operation unit 115 is pressed. Depending on the photographing device 1, it may have a preview display function or a function corresponding to video display on a display (for example, the display 318 of the smartphone 3). In this case, image data is output continuously from the image sensors 103a and 103b at a predetermined frame rate (frames/minute).

The imaging control unit 105 also functions as a synchronization control unit that synchronizes the output timing of image data of the imaging elements 103a and 103b in cooperation with the CPU 111, as will be described later. Note that in this embodiment, the photographing device 1 is not provided with a display, but may be provided with a display section.

The microphone 108 converts sound into sound (signal) data. The sound processing unit 109 takes in sound data output from the microphone 108 through the I/F bus, and performs predetermined processing on the sound data.

The CPU 111 controls the overall operation of the photographing device 1 and executes necessary processing. ROM112 stores various programs for CPU111. The SRAM 113 and DRAM 114 are work memories that store programs executed by the CPU 111, data being processed, and the like. In particular, the DRAM 114 stores image data that is being processed by the image processing unit 104 and data of processed equirectangular projection images.

The operation unit 115 is a general term for operation buttons such as the shutter button 115a. By operating the operation unit 115, the user inputs various shooting modes, shooting conditions, and the like.

The network I/F 116 is a general term for interface circuits (USB I/F, etc.) with external media such as an SD card, a personal computer, and the like. Further, the network I/F 116 may be wireless or wired. The data of the equirectangular projection image stored in the DRAM 114 may be recorded on an external media via this network I/F 116, or may be transferred to an external terminal (device) such as a smartphone 3 via the network I/F 116 as necessary. ).

The communication unit 117 communicates with an external terminal (device) such as the smartphone 3 via an antenna 117a provided in the photographing device 1 using a short-range wireless communication technology such as Wi-Fi, NFC, or Bluetooth. This communication unit 117 can also transmit the data of the equirectangular projection image to an external terminal (device) such as the smartphone 3.

The electronic compass 118 calculates the direction of the imaging device 1 from the earth's magnetism and outputs direction information. This orientation information is an example of related information (metadata) in accordance with Exif, and is used for image processing such as image correction of captured images. Note that the related information also includes data such as the date and time when the image was taken and the data capacity of the image data.

The gyro sensor 119 is a sensor that detects changes in angle (roll angle, pitch angle, yaw angle) accompanying movement of the omnidirectional camera 20. A change in angle is an example of related information (metadata) according to Exif, and is used for image processing such as image correction of a captured image.

The acceleration sensor 120 is a sensor that detects acceleration in three axial directions. The imaging device 1 calculates the attitude (angle with respect to the direction of gravity) of its own device (imaging device 1) based on the acceleration detected by the acceleration sensor 120. By providing both the gyro sensor 119 and the acceleration sensor 120 in the photographing device 1, the accuracy of correcting the tilt of the image is improved.

The terminal 121 is a concave terminal for Micro USB.

<Hardware configuration of smartphone>
Next, the hardware configuration of the smartphone will be described in detail using FIG. 10. FIG. 10 is a hardware configuration diagram of the smartphone.

As shown in FIG. 10, the smartphone 3 includes a CPU 301, a ROM 302, a RAM 303, an EEPROM 304, a CMOS sensor 305, an image sensor I/F 306, an acceleration/direction sensor 307, a media I/F 309, and a GPS receiver 311. There is.

Among these, the CPU 301 controls the entire operation of the smartphone 3. The ROM 302 stores the CPU 301 and programs used to drive the CPU 301 such as IPL. RAM 303 is used as a work area for CPU 301. The EEPROM 304 reads or writes various data such as smartphone programs under the control of the CPU 301. A CMOS (Complementary Metal Oxide Semiconductor) sensor 305 is a type of built-in imaging means that images a subject (mainly a self-portrait) and obtains image data under the control of the CPU 301. Note that instead of a CMOS sensor, an imaging means such as a CCD (Charge Coupled Device) sensor may be used. The image sensor I/F 306 is a circuit that controls driving of the CMOS sensor 305. The acceleration/direction sensor 307 is a variety of sensors such as an electronic magnetic compass, a gyro compass, and an acceleration sensor that detect geomagnetism. A media I/F 309 controls reading or writing (storage) of data to a recording medium 308 such as a flash memory. The GPS receiving unit 311 receives GPS signals from GPS satellites.

The smartphone 3 also includes a long-distance communication circuit 312, a CMOS sensor 313, an image sensor I/F 314, a microphone 315, a speaker 316, a sound input/output I/F 317, a display 318, an external device connection I/F (Interface) 319, and a nearby It includes a long-distance communication circuit 320, an antenna 320a of a short-distance communication circuit 320, and a touch panel 321.

Among these, the long distance communication circuit 312 is a circuit that communicates with other devices via the communication network 100. The CMOS sensor 313 is a type of built-in imaging means that images a subject and obtains image data under the control of the CPU 301. The image sensor I/F 314 is a circuit that controls driving of the CMOS sensor 313. Microphone 315 is a built-in circuit that converts sound into electrical signals. The speaker 316 is a built-in circuit that converts electrical signals into physical vibrations to produce sounds such as music and voice. The sound input/output I/F 317 is a circuit that processes input/output of sound signals between the microphone 315 and the speaker 316 under the control of the CPU 301 . The display 318 is a type of display means such as a liquid crystal or organic EL (Electro Luminescence) that displays an image of a subject, various icons, and the like. The external device connection I/F 319 is an interface for connecting various external devices. The near field communication circuit 320 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The touch panel 321 is a type of input means by which the user operates the smartphone 3 by pressing the display 318.

The smartphone 3 also includes a bus line 310. The bus line 310 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 301 shown in FIG. 10.

[Functional configuration of embodiment]
Next, the functional configuration of this embodiment will be described using FIGS. 9 to 11. FIG. 11 is a functional block diagram of the image communication system according to the first embodiment.

<Functional configuration of imaging device>
As shown in FIG. 11, the photographing device 1 includes a receiving section 12, an imaging section 13, a sound collecting section 14, a determining section 15, an image/sound processing section 16, a communication section 18, and a storage/reading processing section 19. have. Each of these units is a function or means that is realized when any of the components shown in FIG. It is.

Further, the photographing device 1 has a storage unit 1000 constructed by a ROM 112, an SRAM 113, and a DRAM 114 shown in FIG.

(Functional configuration of photographing device 1)
Next, each functional configuration of the photographing device 1 will be described in more detail using FIGS. 9 and 11.

The reception unit 12 of the photographing device 1 is mainly realized by the processing of the CPU 111 on the operation unit 115 shown in FIG. 9, and receives operation input from the user.

The imaging unit 13 is mainly realized by the processing of the CPU 111 for the imaging unit 101, the image processing unit 104, and the imaging control unit 105 shown in FIG. 9, and images a landscape or the like to obtain captured image data.

The sound collecting unit 14 is mainly realized by the processing of the CPU 111 for the sound processing unit 108 and the sound processing unit 109 shown in FIG. 9, and collects sounds around the photographing device 1.

The determination unit 15 is realized by the processing of the CPU 111 and performs various determinations.

The image/sound processing section 16 is realized by the processing of the CPU 111 for the image processing unit 104, and is capable of processing the image shown in FIG. 3(c) from the two hemispherical images shown in FIGS. 3(a) and 3(b). An equirectangular cylindrical projection image EC as a spherical image is created. Further, the image/sound processing unit 16 is realized by the processing of the CPU 111 for the sound processing unit 109, and creates sound data from the sound signal obtained by collecting sound.

The communication unit 18 is mainly realized by the communication unit 117, the antenna 117a, and the processing of the CPU 111, and performs data communication with the communication unit 38 of the smartphone 3, which will be described later, by short-range wireless communication. In the case of wired communication, the communication unit 18 is realized by electrical contacts and processing by the CPU 111, receives power supply from the smartphone 3, and performs data communication.

The storage/read processing unit 19 is mainly realized by the processing of the CPU 111 shown in FIG. read out.

<Functional configuration of smartphone>
Next, the functional configuration of the smartphone 3 will be described in detail using FIGS. 10 and 11. As shown in FIG. 11, the smartphone 3 includes a transmitting/receiving section 31, a receiving section 32, a display control section 34, a determining section 35, a communication section 38, and a storage/reading processing section 39. Each of these parts is a function or means that is realized when any of the components shown in FIG. be.

Furthermore, the smartphone 3 has a storage unit 3000 constructed from a RAM 303 and an EEPROM 304 shown in FIG.

(Each function configuration of smartphone)
The transmitting/receiving unit 31 of the smartphone 3 is mainly realized by the processing of the CPU 301 for the long-distance communication circuit 312, and exchanges various data with other devices (for example, the photographing device 1) via the communication network 100 such as the Internet. (or information).

The reception unit 32 is mainly realized by the processing of the CPU 301 on the touch panel 321, and receives various selections or inputs from the user.

The display control unit 34 is mainly realized by the processing of the CPU 301, and causes the display 318 to display various images.

The determining unit 35 is mainly realized by the processing of the CPU 301, and performs various determinations. The details of the judgment will be described later.

The communication unit 38 is mainly realized by the processing of the CPU 301 for the short-range communication circuit 320, and performs data communication with the communication unit 18 of the photographing device 1. In the case of wired communication, image data and the like are communicated by connecting the photographing device 1 with a communication cable.

The storage/read processing unit 39 is mainly realized by the processing of the CPU 301, and stores various data (or information) in the storage unit 3000 and reads various data (or information) from the storage unit 3000.

[Processing or operation of embodiment]
<Management of wide-angle images>
First, with reference to FIG. 12, a process for managing the data of the equirectangular projection image acquired by the smartphone 3 from the photographing device 1 will be described. FIG. 12 is a sequence diagram showing communication processing of equirectangular projection image data according to the first embodiment.

First, user A uses the photographing device 1 to photograph a subject, scenery, etc. (see FIG. 2) (S11).

Next, the image/sound processing unit 16 of the photographing device 1 generates an equirectangular projection image based on the two hemispherical images obtained by photographing in step S11 (see FIGS. 3(a) and 3(b)). (See FIG. 3(c)) is created (S12). Then, the communication unit 18 of the photographing device 1 transmits the data of the equirectangular projection image created in step S12 to the communication unit 38 of the smartphone 3 (S13). Then, the storage/read processing unit 39 stores the equirectangular projection image data received in step S13 in the storage unit 3000 (S14).

Next, the smartphone 3 performs display processing on the display 318 (S15). Here, the process of step S15 will be explained in detail using FIGS. 13 to 17. FIG. 13 is a flowchart showing a process for displaying a predetermined area image according to the first embodiment. 14 to 17 relate to the first embodiment, in which (a) is a diagram showing a predetermined area image in a plurality of equirectangular projection images, and (b) is a diagram showing a corresponding predetermined area image before operating the smartphone. The illustrated figure (c) is a diagram showing a corresponding predetermined area image after the smartphone is operated.

First, as shown in FIG. 13, the storage/readout processing unit 39 first retrieves data of an equirectangular projection image from the storage unit 3000 to be displayed as a spherical image. is read out (S31). For example, the storage/read processing unit 39 reads data of the equirectangular projection image EC1 shown in FIG. 14.

Next, the display control unit 34 displays the initial display area (starting end contact area) in the equirectangular projection image predetermined by the above (Formula 1) as a predetermined area image on the display 318 (S32). For example, the display control unit 34 converts the initial display area (contact area of the starting edge BE1) in the equirectangular projection image EC1 as shown in FIG. 14(a) into the predetermined area image pa11 in FIG. 14(b). Display on display 318 as shown.

Then, when the user A performs a swipe operation on the displayed predetermined area image, the reception unit 32 accepts the swipe (S33). For example, as shown in FIG. 14(b), user A performs a swipe operation in the direction of the arrow (leftward) with a finger on the predetermined area image pa11.

Next, the determination unit 35 determines whether the "tip" in the image movement direction (rightward) due to the swipe operation (leftward) in the predetermined area image has moved (reached) the "edge" in the display of the equirectangular projection image. (S34-1). For example, the determination unit 35 determines whether the tip a11 in the predetermined area image pa11 has moved to the end (starting end BE1 or ending end TE1) of the equirectangular projection image due to movement by a swipe operation.

Next, if the determination unit 35 determines that the image has not moved (S34-1; NO), the display control unit 34 displays the changed predetermined area image on the display 318 (S35). For example, as shown in FIG. 14(a), if the tip a12 of the predetermined area image pa12 has not moved to the end (starting end BE1 or ending end TE1) of the equirectangular projection image EC1, as shown in FIG. 14(c). As shown in , the display control unit 34 displays the changed predetermined area image pa12 within the same equirectangular projection image EC1 on the display 318.

Next, when the accepting unit 32 accepts the request to end the display of the predetermined area image (S36; YES), the display of the predetermined area image ends. On the other hand, if the reception unit 32 has not accepted the end of the display of the predetermined area image (S36; NO), the display of the predetermined area image is continued, and the process returns to step S33.

Further, when the determining unit 35 determines that the movement has occurred in step S34-1 (S34-1; YES), the determining unit 35 further determines whether the “end” is the “starting end” of the equirectangular projection image or the “terminating end” of the equirectangular projection image. ” (S37). If the determining unit 35 determines that it is the “terminus” (S37; terminating), the determining unit 35 further determines whether data of the next equirectangular projection image is stored in the storage unit 3000 (S38 ). If the determining unit 35 determines that the next equirectangular projected image is stored (S38; YES), the storage/reading processing unit 39 stores the next equirectangular projected image from the storage unit 3000. The data is read out (S39). Then, the process returns to step S32.

That is, the receiving unit 32 accepts movement of the first predetermined area image in the equirectangular projection image EC1 in a predetermined movement direction over a certain area in the equirectangular projection image EC1, and the movement in the same movement direction is accepted. If it is accepted, the display control unit 34 causes the display 318 to display a second predetermined area that is a predetermined area in the next equirectangular projection image EC2. To explain in more detail, in this embodiment, the receiving unit 32 accepts movement of the first predetermined area image in a predetermined moving direction over a certain area in the equirectangular projection image EC1. 32, the first predetermined area image can be moved in a predetermined movement direction over the entire area of the equirectangular projection image EC1.

For example, as shown in FIG. 15(b), the user A performs a swipe operation on the predetermined area image pa12 in the arrow direction (left direction) with a finger. As a result, as shown in FIG. 15(a), by moving the predetermined area image from the predetermined area image pa12 to the predetermined area image pa13, the display control unit 34 displays the same equirectangular cylinder on the display 318. The changed predetermined area image pa13 in the projected image EC1 is displayed. Furthermore, as shown in FIG. 16(b), by continuing the swipe operation in the direction of the arrow (to the left), the tip 13a of the predetermined area image pa13 moves (reaches) the terminal end TE1 of the equirectangular projection image EC1. In this case, as shown in FIG. 16A, the predetermined area image moves from the predetermined area image pa13 to the predetermined area image pa21 in the next equirectangular projection image EC2. Thereby, the display control unit 34 displays the predetermined area image pa21, which is the initial display area (the contact area of the starting end EB2) in the next equirectangular projection image EC2, on the display 318. In this case, the rear end r21 of the predetermined area image pa21 is in contact with the starting end BE2 of the equirectangular projection image EC2.

On the other hand, if the determining unit 35 determines in step S38 that the next equirectangular projection image is not stored (S38; NO), the process returns to step S33.

Further, in step 37 above, when the determining unit 35 determines that it is the “starting end” (S37; starting end), the determining unit 35 further determines whether the data of the previous equirectangular projection image is stored in the storage unit 3000. (S40). If the determining unit 35 determines that the data of the previous equirectangular projection image is stored (S40; YES), the storage/reading processing unit 39 stores the data of the previous equirectangular projection image from the storage unit 3000. Data of the projected image is read (S41). Then, the display control unit 34 displays the initial display area (terminal contact area) in the previous equirectangular projection image as a predetermined area image (S42). Thereafter, the process returns to step S33.

That is, the reception unit 32 accepts movement of the second predetermined area image in the equirectangular projection image EC2 over a certain area in the opposite direction to the predetermined movement direction in the equirectangular projection image EC2, and furthermore, If the movement in the direction is accepted, the display control unit 34 displays the first predetermined area in the equirectangular projection image EC1 on the display 318. To explain in more detail, in this embodiment, the reception unit 32 causes the predetermined area image of the equirectangular projection image EC2 to move over a certain area in the opposite direction to the predetermined movement direction in the equirectangular projection image EC2. What is accepted means that the reception unit 32 accepts the movement of the second predetermined area image to the area in which the second predetermined area image is first displayed in the equirectangular projection image EC2.

For example, as shown in FIG. 17(b), when user A performs a swipe operation in the direction of the arrow opposite to the previous direction (to the right), the screen shown in FIG. 17(a) As shown, the predetermined area image returns from the predetermined area image pa21 to the predetermined area image pa13, which is the initial display position (the contact area of the terminal end BE1) in the previous equirectangular projection image EC1. In this case, the rear end r'13 in the moving direction (left direction) of the predetermined area image pa13 after returning is in contact with the terminal end BE1 of the equirectangular projection image EC1. In this way, it is possible to return to displaying the previous equirectangular projection image simply by a swipe operation.

On the other hand, if the determining unit 35 determines in step S40 that the data of the previous equirectangular projection image is not stored (S40; NO), the process returns to step S33.

This concludes the description of the process shown in FIG. 13.

[Main effects of this embodiment]
As described above, according to the present embodiment, the predetermined area image (an example of the first predetermined area image) is moved in a predetermined movement direction by a swipe operation by the user A, and is further moved in the same movement direction by a swipe operation. By doing so, the user can display the predetermined area image (an example of the second predetermined area image) of the next equirectangular projection image by repeating the same operation. Specifically, when the user moves a predetermined area image with the momentum of moving the user's finger, as in the case of a swipe operation, as shown in FIG. There is no need to stop the finger during operation and press the "next" button n1, which is another operation. Therefore, by switching from a swipe operation to a button press, it is possible to solve the problem that the user feels uncomfortable with the operability.

●Second Embodiment Next, a second embodiment of the present invention will be described using FIGS. 18 to 20. Note that the second embodiment and the first embodiment have basically the same hardware configuration and functional configuration, so similar configurations, functions, and processing will be described with the same reference numerals. Omitted. In the second embodiment, step S34-1 in FIG. 13 in the first embodiment is simply replaced with S34-2 as shown in FIG. 18. Therefore, only the processes or operations that are different from the first embodiment will be described below.

[Processing or operation of embodiment]
As shown in FIG. 18, after the receiving unit 32 accepts the swipe operation (S33), the determining unit 35 determines whether the “back end” of the image in the predetermined area image in the moving direction due to the swipe operation is correct. It is determined whether it has moved to the "end" in the display of the cyclocylindrical projection image (S34-2). If the determining unit 35 determines that the object has not moved (S34-1; NO), the process proceeds to step S35. For example, as shown in FIG. 19(a), even if the leading end a13 in the image movement direction due to the swipe operation in the predetermined area image pa13 moves to the rear end TE1 of the equirectangular projection image EC1, the first Unlike the embodiment described above, the process does not proceed to step S35. Then, as shown in FIG. 19(b), the user A swipes the predetermined area image pa13 in the direction of the arrow (leftward), and as shown in FIG. 19(c), The display control unit 34 displays the left area image pa14a of the predetermined area image pa14 in FIG. 19(a) on the left side of the display 318 as shown in FIG. 19(c). Further, the display control unit 34 displays the right area image pa14b of the predetermined area image pa14 in FIG. 19(a) on the right side of the display 318 as shown in FIG. 19(c). In this manner, when the display control unit 34 displays the predetermined area image pa14, the image is displayed while going around the equirectangular projection image as the equirectangular projection image EC1.

On the other hand, if the determining unit 35 determines in step S34-2 that the object has moved (S34-2; YES), the process proceeds to step S37. For example, if the swipe operation is further performed from FIG. As shown in FIG. 20(a), the predetermined area image pa21, which is the initial display area in the equirectangular projection image EC2 as the next equirectangular projection image, is converted into a predetermined area image pa21 as shown in FIG. 20(b). It is displayed on the display 318 as if it were.

Then, as shown in FIG. 20(b), when the user A swipes in the opposite direction (to the right) from the previous swipe direction, the swipe direction shown in FIG. 20(a) The display control unit 34 displays the left area image pa22a of the predetermined area image pa22 in FIG. 20(a) on the left side of the display 318 as shown in FIG. 20(c). Further, the display control unit 34 displays the right area image pa22b of the predetermined area image pa22 in FIG. 20(a) on the right side of the display 318 as shown in FIG. 20(c). In this way, when the display control unit 34 displays the predetermined area image pa22, it does not immediately return to the equirectangular projection image EC1 as the previous equirectangular projection image, but displays the entire sky as the equirectangular projection image EC2. The spherical image will be displayed while orbiting within the image.

That is, the receiving unit 32 accepts movement of the first predetermined area image in the equirectangular projection image EC1 in a predetermined movement direction over a certain area in the equirectangular projection image EC1, and the movement in the same movement direction is accepted. If it is accepted, the display control unit 34 causes the display 318 to display a second predetermined area that is a predetermined area in the next equirectangular projection image EC2. To explain in more detail, in this embodiment, the receiving unit 32 accepts movement of the first predetermined area image in a predetermined moving direction over a certain area in the equirectangular projection image EC1. 32, the "rear end" of the first predetermined area image in the moving direction can accept movement over the entire area of the equirectangular projection image EC1.

Then, from the predetermined area image shown in FIG. 20(c), the user A further swipes in the direction of the arrow (rightward) in FIG. When r'22 moves (reaches) the starting edge BE2 of the equirectangular projection image EC2 (see S37; starting edge, S40; YES in FIG. 18), the display control unit 34 operates as shown in FIG. 19(a). A predetermined area image pa13 in such an initial display area (contact area of the terminal end TE1) is displayed as shown in FIG. 19(b) (see S41 and S42). In FIG. 20(a), the right end of the predetermined area image pa22 is the leading edge in the initial movement direction of the image by the swipe operation, but when the swipe direction is reversed, the right end of the predetermined area image pa21 is the rear end r' It becomes 22.

That is, the reception unit 32 accepts movement of the second predetermined area image in the equirectangular projection image EC2 over a certain area in the opposite direction to the predetermined movement direction in the equirectangular projection image EC2, and furthermore, If the movement in the direction is accepted, the display control unit 34 displays the first predetermined area in the equirectangular projection image EC1 on the display 318. To explain in more detail, in this embodiment, the receiving unit 32 accepts movement of the second predetermined area image over a certain area in a direction opposite to the predetermined movement direction of the equirectangular projection image EC2. The reception unit 32 determines that the "rear end" in the movement direction of the second predetermined region image is the same as that of the second predetermined region image when the second predetermined region image is first displayed in the equirectangular projection image EC2. It is possible to accept movement toward the "tip" in the direction of movement.

This concludes the explanation of the process shown in FIG. 18.

<Other examples>
Other examples of the second embodiment will be described below using FIG. 21. In the second embodiment, as shown in step S34-2 of FIG. 18, the determination unit 35 determines that the rear end of the image in the movement direction due to the swipe operation in the predetermined area image is in the display of the equirectangular projection image. I was deciding whether to move to the edge. That is, the determining unit 35 determines whether the line that is the rear end has moved (reached) the line that is the end in the display of the equirectangular projection image. As shown in FIG. 21, the determining unit 35 is not limited to determining whether such a line has moved to a line, and as shown in FIG. It may be determined whether the passing area PA1 between the two areas is the same as all areas of the equirectangular projection image EC1. If they are the same, the storage/reading processing section 39 reads out the data of the next equirectangular projection image from the storage section 3000, and the display control section 34 outputs the data of the next equirectangular projection image as the next equirectangular projection image. The initial display area (starting end contact area) in the cylindrical projection image EC2 is displayed as a predetermined area image.

[Main effects of this embodiment]
As explained above, according to this embodiment, the effects of the first embodiment are achieved, and the following effects are also achieved. That is, as shown in FIG. 19(c), the area near the end TE1 of the equirectangular projection image EC1 is displayed as the left end of the predetermined area image. Area image pa21) is not displayed. As a result, since the user A has completely gone around the equirectangular projection image to display the predetermined area image, the user A has the effect of making it easier to view the spherical image.

●Third Embodiment Next, a third embodiment of the present invention will be described using FIGS. 22 to 25. Note that in the first and second embodiments, the data of the equirectangular projection image is temporarily stored in the storage unit 3000 of the smartphone 3, and read out as needed. On the other hand, in the third embodiment, data of an equirectangular projection image is newly stored in the image management server 5 instead of the smartphone 3. The third embodiment will be specifically described below.

[Outline of the configuration of the image communication system]
First, the outline of the configuration of the image communication system according to the third embodiment will be described using FIG. 22. FIG. 22 is a schematic diagram of an image communication system according to the third embodiment. In the third embodiment, an image management server 5 is further added to the system configuration shown in FIG. The image management server 5 can perform data communication with the smartphone 3 via a communication network 100 such as the Internet. The image management server 5 is configured by a computer. Note that the image management server 5 may be composed of a single computer or a plurality of computers.

[Hardware configuration of embodiment]
Next, the hardware configuration of the image management server 5 of this embodiment will be described in detail using FIG. 23. Note that the hardware configurations of the photographing device 1 and the smartphone 3 are the same as those in the first embodiment, so their descriptions will be omitted.

<Hardware configuration of imaging device>
FIG. 23 is a hardware configuration diagram of the image management server 5. As shown in FIG. The image management server 5 is constructed by a computer, and as shown in FIG. interface) 508, a network I/F 509, a data bus 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I/F 516.

Among these, the CPU 501 controls the overall operation of the image management server 5. The ROM 502 stores programs used to drive the CPU 501 such as IPL. RAM 503 is used as a work area for CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls reading and writing of various data to the HD 504 under the control of the CPU 501. The display 506 displays various information such as a cursor, menu, window, characters, or images. External device connection I/F 508 is an interface for connecting various external devices. The external device in this case is, for example, a USB (Universal Serial Bus) memory, a printer, or the like. The network I/F 509 is an interface for data communication using the communication network 100. The bus line 510 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 501 shown in FIG. 5.

Further, the keyboard 511 is a type of input means that includes a plurality of keys for inputting characters, numerical values, various instructions, and the like. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. The DVD-RW drive 514 controls reading and writing of various data on a DVD-RW 513, which is an example of a removable recording medium. Note that the disc is not limited to DVD-RW, but may be DVD-R, Blu-ray Disc, or the like. The media I/F 516 controls reading or writing (storage) of data to a recording medium 515 such as a flash memory.

[Functional configuration of embodiment]
Next, the functional configuration of this embodiment will be described using FIG. 24. FIG. 24 is a functional block diagram of an image communication system according to the third embodiment. Note that the functional configurations of the photographing device 1 and the smartphone 3 are the same as those in the first embodiment, so similar functional configurations are denoted by the same reference numerals and description thereof will be omitted.

<Functional configuration of image management server>
Next, the functional configuration of the image management server 5 will be described in detail using FIG. 24. FIG. 24 is a functional block diagram of an image communication system according to the third embodiment.

As shown in FIG. 24, the image management server 5 includes a transmitting/receiving section 51 and a storage/reading processing section 59. Each of these units is a function or means that is realized when any of the components shown in FIG. It is.

Further, the image management server 5 has a storage unit 5000 constructed by a ROM 502 and an HD 504 shown in FIG.

(Functional configuration of image management system)
Next, each functional configuration of the photographing device 1 will be explained in more detail using FIGS. 23 and 24.

The transmitting/receiving unit 51 of the image management system 5 is mainly realized by the processing of the CPU 501 on the network I/F 510, and exchanges various data with other devices (for example, the smartphone 3) via the communication network 100 such as the Internet. (or information).

The storage/read processing unit 59 is mainly realized by the processing of the CPU 501, and stores various data (or information) in the storage unit 5000 and reads various data (or information) from the storage unit 5000.

[Processing or operation of embodiment]
The processing or operation of the third embodiment will be described using FIG. 25. FIG. 25 is a sequence diagram showing communication processing of equirectangular projection image data according to the third embodiment.

First, user A uses the photographing device 1 to photograph a subject, scenery, etc. (see FIG. 2) (S111).

Next, the image/sound processing unit 16 of the photographing device 1 generates an equirectangular projection image based on the two hemispherical images obtained by photographing in step S111 (see FIGS. 3(a) and 3(b)). (See FIG. 3(c)) is created (S112). Then, the communication unit 18 of the photographing device 1 transmits the data of the equirectangular projection image created in step S112 to the communication unit 38 of the smartphone 3 (S113). Then, the transmitting/receiving unit 31 transmits the equirectangular projection image data received by the communication unit 18 to the image management server 5 via the communication network 100 (S114). Thereby, the transmitting/receiving unit 51 of the image management server 5 receives the equirectangular projection image data.

Next, in the image management server 5, the storage/read processing unit 59 stores the equirectangular projection image data in the storage unit 5000 (S115). Thereby, the equirectangular projection image data is managed on the image management server 5 side.

Next, the smartphone 3 performs display processing (S116). The process in step S116 is basically the same as the process shown in FIG. 13 or 18, but steps S31, S39, and S41 are different. That is, in steps S31, S39, and S41, the storage/read processing unit 39 of the smartphone 3 reads equirectangular projection image data from the storage unit 3000 of the smartphone 3, whereas in the third embodiment, the process shown in FIG. As shown, the two differ in that the smartphone 3 acquires equirectangular projection image data from the image management server 5 via the communication network 100.

Specifically, as shown in FIG. 25, the transmitting/receiving unit 31 of the smartphone 3 transmits request information indicating a request for equirectangular projection image data to the image management server 5. Thereby, the transmitting/receiving unit 51 of the image management server 5 receives the request information. Then, the storage/read processing unit 59 of the image management server 5 reads the requested equirectangular projection image data from the storage unit 5000 (S117). Then, the transmitting/receiving unit 51 transmits the requested equirectangular projection image data to the smartphone 3 (S117). Thereby, the transmitting/receiving unit 31 of the smartphone 3 receives the equirectangular projection image data. After that, the smartphone 3 executes the process of step S32 or step S42.

[Main effects of this embodiment]
As explained above, according to the present embodiment, the effects of the first embodiment are achieved, and even if the equirectangular projection image data has a large volume or a large number of data, the smartphone 3 can This has the effect that the free capacity of the storage unit 3000 is not compressed.

〔supplement〕
Each function in the embodiments described above can be realized by one or more processing circuits. Here, the "processing circuit" in this embodiment refers to a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, or an ASIC (ASIC) designed to execute each function described above. Application Specific Integrated
circuit), DSP (digital signal processor), FPGA (field programmable gate array), SOC (system on a chip), GPU, and conventional circuit modules.

Further, the omnidirectional image is a three-dimensional image and is an example of a wide-angle image. The wide-angle image has a size (number of pixels) larger than the predetermined area image, and includes a panoramic image as a two-dimensional image and a general planar image.

Furthermore, the communication network 100 also includes a network constructed using blockchain.

1 Photographing device 3 Smartphone (an example of a communication terminal)
5 Image management server 12 Reception section 13 Photography section 14 Sound collection section 15 Judgment section 16 Image processing section 19 Storage/reading processing section 31 Transmission/reception section 32 Reception section (an example of reception means)
34 Display control unit (an example of display control means)
35 Judgment unit 38 Communication unit 39 Storage/reading processing unit 51 Transmission/reception unit 59 Storage/reading processing unit 318 Display (an example of display means)
1000 Storage unit 3000 Storage unit 5000 Storage unit

Unexamined Japanese Patent Publication No. 2014-10611

Claims (12)

A communication terminal that causes a display means to display a first predetermined area image that is a predetermined area in an arbitrary wide-angle image,
reception means for accepting movement of the first predetermined area image being displayed on the display means;
When the acceptance means accepts movement of the first predetermined area image in a predetermined movement direction over a certain area in the arbitrary wide-angle image, and further accepts movement in the movement direction, Display control means for causing the display means to display a second predetermined area that is a predetermined area in the next wide-angle image;
A communication terminal comprising: The accepting means accepts a movement of the second predetermined area image across a certain area in a direction opposite to the predetermined movement direction in the next wide-angle image, and further accepts movement of the second predetermined area image in the opposite direction. 2. The communication terminal according to claim 1, wherein the display control means displays the first predetermined area in the arbitrary wide-angle image on the display means. The receiving means accepts movement of the first predetermined area image in a predetermined moving direction over a certain area in the arbitrary wide-angle image. 2. The communication terminal according to claim 1, wherein the communication terminal is capable of accepting movement in a predetermined movement direction over the entire area of the arbitrary wide-angle image. The receiving means accepts movement of the second predetermined area image over a certain area in the next wide-angle image in a direction opposite to the predetermined moving direction. 3. The communication terminal according to claim 2, wherein the predetermined area image accepts movement to an area in which the second predetermined area image is first displayed in the next wide-angle image. The reception means accepts movement of the first predetermined area image in a predetermined movement direction over a certain area in the arbitrary wide-angle image. 2. The communication terminal according to claim 1, wherein the rear end in the movement direction is capable of accepting movement over the entire area of the arbitrary wide-angle image. The receiving means accepts movement of the second predetermined area image over a certain area in the next wide-angle image in a direction opposite to the predetermined moving direction. The rear end of the predetermined area image in the moving direction may move to the leading end of the second predetermined area image in the moving direction when the second predetermined area image is first displayed in the next wide-angle image. 3. The communication terminal according to claim 2, wherein the communication terminal is accepted. 7. The communication terminal according to claim 1, wherein the wide-angle image is a spherical image. A communication terminal according to any one of claims 1 to 7,
a photographing device that transmits data of the wide-angle image obtained by photographing a subject to the communication terminal;
An image communication system comprising: The image communication system according to claim 8,
an image management server that manages data of the wide-angle image;
An image communication system characterized in that the communication terminal has a receiving means for receiving data of the wide-angle image from the image management server via a communication network. The image communication system according to claim 9, wherein the communication network also includes a network constructed by blockchain. A display method for displaying a first predetermined region image, which is a predetermined region in an arbitrary wide-angle image, on a display means,
a reception step of accepting a movement of the first predetermined area image being displayed on the display means;
In the accepting step, when the first predetermined area image is accepted for movement in a predetermined movement direction over a certain area in the arbitrary wide-angle image, and further movement in the movement direction is accepted, a display control step of causing the display means to display a second predetermined area that is a predetermined area in the next wide-angle image;
A display method characterized by performing. A program that causes a computer to execute the method according to claim 11.