JP2022129799A - Display control program, display control method, and display apparatus - Google Patents

Abstract

To provide an intuitive user interface for continuously and comfortably viewing spherical images.SOLUTION: A display control program for displaying spherical images, causes a computer to execute: a display step of displaying a first image which is an image generated by cutting out at least a part of a target image selected from among a plurality of selectable spherical images, in a first display area of a display device and displaying a second image which is an image generated by cutting out a part of the target image along a horizontal direction, in a second display area of the display device; and an update step of updating any one of spherical images ordered before and after the target image among the plurality of spherical images to the newly selected target image if a user's touch operation along the horizontal direction is received in the second display area via a touch sensor superimposed on the display device.SELECTED DRAWING: Figure 6

Description

The present invention relates to a display control program, a display control method, and a display device.

2. Description of the Related Art A camera capable of capturing an omnidirectional image that integrates a 360° horizontal image and a zenith image is known (see, for example, Patent Document 1).

JP 2013-218278 A

A omnidirectional image is an image that captures a range that is significantly different from the human field of view. Therefore, unlike an image with a normal angle of view, if the entire image is displayed as it is, it will be greatly distorted. Such a display image that displays the whole image is suitable for thumbnail display of a list of a plurality of captured images, but is not suitable for one-image display for viewing. In one-image display, a part of the omnidirectional image is cut out and displayed in a form close to the human visual field. That is, it is displayed so that the user can easily grasp the image space. By operating the display screen displayed in this way, the user can move the displayed partial image in a desired viewing direction, or enlarge or reduce the image.

If such thumbnail display and single-image display are adopted, each time a specific omnidirectional image is to be viewed, it is necessary to select the specific image from the thumbnail display and switch to the single-image display. That is, when viewing omnidirectional images consecutively, it is necessary to perform an operation to temporarily return to the thumbnail display and an operation to select the next image in the thumbnail display.

SUMMARY OF THE INVENTION The present invention has been made to solve such problems, and provides an intuitive user interface that allows continuous and comfortable viewing of omnidirectional images.

A display control program according to a first aspect of the present invention displays a first image, which is an image generated by cutting out at least a part of a target image selected from among a plurality of selectable omnidirectional images, on a display device. a display step of displaying a second image, which is an image generated by cutting out a portion of the target image along the horizontal direction, in the display area, and displaying the second image in the second display area of the display; one of the omnidirectional images ordered before or after the target image among the plurality of omnidirectional images when the user's contact operation along the horizontal direction is received in the second display area by the touch sensor. to the newly selected target image.

A display control method according to a second aspect of the present invention provides a first image generated by cutting out at least a part of a target image selected from among a plurality of selectable omnidirectional images. a display step of displaying a second image, which is an image generated by cutting out a portion of the target image along the horizontal direction, in the display area, and displaying the second image in the second display area of the display; one of the omnidirectional images ordered before or after the target image among the plurality of omnidirectional images when the user's contact operation along the horizontal direction is received in the second display area by the touch sensor. to the newly selected target image.

A display device according to a third aspect of the present invention displays a first image, which is an image generated by cutting out at least part of a target image selected from among a plurality of selectable omnidirectional images, on a display device. a display control unit for displaying a second image, which is an image generated by cutting out a part of the target image along the horizontal direction, in the second display area of the display, and superimposed on the display. and an omnidirectional image that is ordered before and after the target image among the plurality of omnidirectional images when the touch sensor receives a user's contact operation along the horizontal direction in the second display area. an updating unit for updating any of the images to the newly selected target image.

According to the present invention, it is possible to provide an intuitive user interface that allows continuous and comfortable viewing of omnidirectional images.

FIG. 3 is a diagram for explaining how an omnidirectional camera captures an omnidirectional image; FIG. 4 is a diagram illustrating a route through which image data of an omnidirectional image is transmitted and received; 1 is a system configuration diagram of a smartphone regarding display of an omnidirectional image; FIG. FIG. 10 is a diagram for explaining a switching procedure between thumbnail display and single image display; It is a figure explaining the screen structure of a one image display. FIG. 10 is a diagram illustrating a state of direct transition from one-image display to one-image display of the next image; FIG. 10 is a diagram for explaining a state of direct transition from one-image display to one-image display of a previous image; FIG. 10 is a diagram for explaining generation of sub-images; FIG. 10 is a diagram illustrating another technique for directly transitioning from one-image display to one-image display of the next image; FIG. 10 is a diagram for explaining changes in an image in response to a contact operation on the main area; FIG. 10 is a diagram for explaining changes in an image in response to a contact operation on the main area; FIG. 10 is a diagram illustrating another layout of the main area and sub-areas; FIG. 10 is a diagram illustrating generation of a sub-image according to a subject image by another method; FIG. 10 is a diagram illustrating generation of a sub-image according to a subject image by another method; FIG. 10 is a flowchart showing a processing procedure of an arithmetic processing unit regarding transition of target images in one-image display;

Hereinafter, the present invention will be described with reference to the accompanying drawings, but the invention according to the scope of claims is not limited to the following embodiments. Moreover, not all the configurations described in the embodiments are essential as means for solving the problems. In addition, in each figure, when there are multiple elements having the same or similar configuration, in order to avoid complication, there are cases where the same reference numerals are attached to some and the same reference numerals are omitted. be.

FIG. 1 is a diagram illustrating how an omnidirectional camera 500 captures an omnidirectional image. The omnidirectional camera 500 includes a camera section having a plurality of built-in imaging units above a grip held by a user. A plurality of imaging units arranged in the horizontal direction, which is the equatorial direction when the user grips the grip in the vertical direction, captures 360° images along the equatorial direction, and images are taken in the vertical direction, which is the zenith direction. The imaging unit arranged facing the sky captures an image in the zenith direction. The omnidirectional camera 500 generates omnidirectional image data by connecting the images captured by the respective imaging units. Since the omnidirectional image depends on the configuration of the imaging unit of the omnidirectional camera, it may be an image in which part of the zenith direction and the ground direction is missing. The example in the figure shows a state in which part of the ground direction is missing.

FIG. 2 is a diagram illustrating a route through which image data of an omnidirectional image is transmitted and received. The omnidirectional image data generated by the omnidirectional camera 500 is transferred to the image server 600 via the network 700 such as the Internet, and stored in the storage device of the image server 600 . Smartphone 100 as a user terminal acquires omnidirectional image data from image server 600 via network 700 . Alternatively, it is obtained directly from the omnidirectional camera 500 by proximity communication.

The user can use a display application (display control program) installed in the smartphone 100 to develop the acquired omnidirectional image data into a visible image and display the image on the display panel 120 . A touch panel 130 is superimposed on the display panel 120, and the user can view omnidirectional images in various modes by touching the surface of the display panel with a finger or a touch pen.

In this embodiment, the smartphone 100 is used as an example of a display device that displays an omnidirectional image, but the display device is not limited to this, and may be another device such as a tablet terminal. A stationary PC or the like may be used instead of a mobile terminal as long as it has a display in which a touch device is superimposed on a display panel.

FIG. 3 is a system configuration diagram of the smartphone 100 regarding display of an omnidirectional image. The smartphone 100 mainly includes an arithmetic processing unit 110, a display panel 120, a touch panel 130, an input/output IF 140, and a storage unit 150 as a system configuration for displaying an omnidirectional image. The arithmetic processing unit 110 is a processor (CPU: Central Processing Unit) that executes control and arithmetic processing of the smartphone 100 . The arithmetic processing unit 110 may be configured to include arithmetic processing chips such as GPUs (Graphic Processing Units) and ASICs (Application Specific Integrated Circuits), and processing circuits that process various electrical signals. The arithmetic processing unit 110 executes a display control program read from the storage unit 150 or given from an external device via the input/output IF 140, and executes various processes related to displaying the omnidirectional image.

The display panel 120 is an example of a display device, such as an organic EL panel. The display panel 120 receives the image signal adjusted by the display control unit 111 and visibly displays an image obtained by expanding the image signal. The touch panel 130 is a transparent touch device that overlaps substantially the entire display area of the display panel 120 . The touch panel 130 is an example of a touch sensor, receives various user touch operations, converts the operations into electrical signals, and transmits the electrical signals to the arithmetic processing unit 110 . The touch panel 130 distinguishes and recognizes touch operations such as a swipe operation of sliding in one direction while touching, a tap operation of touching and immediately releasing, and a pinch operation of opening and closing two fingers in contact.

The input/output IF 140 is an interface for exchanging information with an external device, such as a wireless LAN unit or a Bluetooth (registered trademark) unit. The input/output IF may include a wired connection unit such as USB (registered trademark). The storage unit 150 is a non-volatile storage medium, and is configured by flash memory, for example. The storage unit 150 can store various parameter values, functions, lookup tables, etc. related to display, in addition to the display control program. It also stores the acquired omnidirectional image data.

The computation processing unit 110 also serves as a functional computation unit that executes various computations according to the processing instructed by the display control program. The arithmetic processing unit 110 can function as a display control unit 111 and an image update unit 112 . Although the details will be described later, the display control unit 111 mainly displays a main image, which is an image generated by cutting out at least part of a target image selected from among a plurality of selectable omnidirectional images, on the display panel. 120 is displayed in the main area (first display area), and a sub-image, which is an image generated by cutting out a part of the target image along the horizontal direction, is displayed in the sub-area (second display area) of the display panel 120. do. The image update unit 112 mainly updates any of the omnidirectional images ordered before and after the target image being displayed when the touch panel 130 receives the user's contact operation along the horizontal direction in the sub-region. to the newly selected target image.

Next, the display mode of the omnidirectional image realized by the display control program will be described. In any of the display modes described below, the display control unit 111 controls the display by adjusting the display signal according to the display mode instructed by the display control program from the display target image displayed on the display panel 120, and updates the image. A unit 112 updates the display target image at the timing instructed by the display control program.

FIG. 4 is a diagram for explaining a switching procedure between thumbnail display and single image display. The left figure is a thumbnail display in which a plurality of omnidirectional images are arranged in order, and the right figure is a one-image display in which a part of one selected omnidirectional image is arranged in the entire display area.

In the thumbnail display, one omnidirectional image is arranged and displayed in each thumbnail area 210 in which the image display area excluding the display area for icons and the like is subdivided. Each omnidirectional image displayed in the thumbnail area 210 may be displayed in a manner in which the entire image is displayed in a circle and the margins are filled. A rectangular image cut out from a part of the omnidirectional image is displayed. Such thumbnail display is convenient for checking what kind of omnidirectional images are stored in the storage unit 150 . Note that the display mode of the thumbnail display is not limited to the lattice arrangement shown in the figure. Horizontally long panoramic images may be arranged in the vertical direction, or may have different sizes.

By tapping a specific image displayed as a thumbnail, the user can switch the omnidirectional image to a single image display. In the one-image display, at least part of the target image, which is the selected omnidirectional image, is cut out and generated in the main area 310 (first area) set over the entire display area of the display panel 120. A main image (first image), which is an image, is displayed. At the same time, a sub-image (second image) generated by cutting out a part of the target image along the horizontal direction is displayed in a sub-area 320 (second area) which is a small area set and surrounded by the main area 310 . be done. A close button 330 is arranged on the upper left of the main area 310 , and the user can switch to thumbnail display by tapping the close button 330 .

FIG. 5 is a diagram for explaining the screen configuration of one-image display. Since the main area 310 is set over the entire display panel 120 as described above, it can be said that the main image is displayed in a large size, and therefore has high visibility and is suitable for viewing.

The sub-region 320 is arranged below the main region 310 so as to be entirely surrounded by the main region 310 and has a horizontally long oval shape. A sub-image is a small image having a wider viewing angle than the main image, which is generated by cutting out a portion of the target image along the horizontal direction. Therefore, the sub-image can be said to be an image for grasping the space for the user to recognize which part of the whole image the main image displayed in the main area 310 is extracted from.

If the entire omnidirectional image is displayed at once, the subject image will be greatly distorted. Therefore, in order to achieve both appreciation and spatial awareness, it is preferable to display the main image and sub-image side by side at the same time. In the present embodiment, the sub-region 320 has a horizontally elongated oval shape, but other shapes such as a horizontally elongated rectangle may be used as long as the sub-image, which is a horizontal image, can be easily displayed.

FIG. 6 is a diagram for explaining how the one-image display is directly transitioned to the one-image display of the next image without going through the thumbnail display. At time t=T ₀ , the image of the omnidirectional image data stored as the n-th among the omnidirectional image data stored in the storage unit 150 (hereinafter referred to as the “nth image”). ) is displayed as one image (left diagram in FIG. 6). From this state, when the user starts swiping from the right side of the sub-region 320 to the left, the sub-image scrolls leftward within the sub-region 320 and the main image also scrolls leftward in the main region 310. do. In other words, with respect to the sub-image, the sub-image obtained by splicing the image portion that continues to the right end of the original sub-image by the amount of the image that disappears from the left end of the sub-region 320 in conjunction with the movement of the swipe is added from the omnidirectional image. It cuts out and successively updates to new sub-images. As for the main image, as much as the image that disappears from the left end of the main area 310 in conjunction with the movement of the swipe, the main image that is added to the right end of the original main image is cut out from the omnidirectional image, It updates to new main image one by one.

The diagram in FIG. 6 shows the state of display at time t=T ₁ , and shows the manner in which both the main image and the sub-image, which are n-th images, are being scrolled.

In this embodiment, "the amount of transition of sub-images reaches 360 degrees" is set as the update condition. That is, when the user continues to swipe the sub-region 320 and completes one round of scrolling, the target image to be displayed is switched to the (n+1)th omnidirectional image. The right diagram of FIG. 6 shows a state in which display is switched to the n+1-th omnidirectional image at time t=T ₂ when the transition amount of the sub-image reaches 360°. That is, at the time t=T ₂ , the omnidirectional image that is ordered one after the omnidirectional image that has been displayed until then is displayed as a new target image. Here, the omnidirectional image that is ordered after one is set as a new target image. may be used as a target image. In addition, depending on the date set in advance, for example, the first omnidirectional image after a set number of days after the date when the n-th image was taken, or the first omnidirectional image taken after the next day The image may be the new target image.

FIG. 7 is a diagram for explaining how the one-image display is directly transitioned to the one-image display of the previous image without going through the thumbnail display. As in the left diagram of FIG. 6, it is assumed that the n-th image is displayed as one image at time t= _T0 (left diagram of FIG. 7). From this state, when the user starts swiping from the left side of the sub-region 320 toward the right side, the sub-image scrolls toward the right side within the sub-region 320 and the main image also scrolls toward the right side within the main region 310. do. In other words, with respect to the sub-image, the sub-image obtained by splicing the image portion that continues from the left end of the original sub-image by the amount of the image that disappears from the right end of the sub-region 320 in association with the movement of the swipe is added from the omnidirectional image. It is cut out and successively updated to new sub-images. As for the main image, as much as the image that disappears from the right end of the main area 310 in conjunction with the movement of the swipe, the main image that is spliced from the left end of the original main image is cut out from the omnidirectional image, It updates to new main image one by one.

The diagram in FIG. 7 shows the state of display at time t=T ₁ , and shows how both the main image and the sub-image, which are n-th images, are being scrolled.

The right diagram in FIG. 7 shows a state in which display is switched to the n-1th omnidirectional image at time t=T ₂ when the transition amount of the sub-image reaches 360°. That is, at the time t=T ₂ , the omnidirectional image that is ordered one before the omnidirectional image that has been displayed until then is displayed as a new target image. Here, the omnidirectional image that is ordered one before is used as a new target image. may be used as a new target image.

FIG. 8 is a diagram for explaining generation of sub-images. A sub-image is generated by cutting out a portion of the omnidirectional image along the horizontal direction, as described above. Specifically, the omnidirectional image is generated by cutting out along the horizontal direction so that the equator line in the omnidirectional image becomes the center line in the vertical direction. The equator line is a line of intersection that intersects a plane passing through the center point in the phantom sphere of the omnidirectional image.

As shown in the figure, a 360° horizontal partial image is cut out from the omnidirectional image, and the left end region (a) of the 360° image is copied and connected to the right side of the right end region (b), and the right end region ( b) is copied and connected to the left side of the left end region (a) to form a horizontal image of 360° or more. The lower diagram of FIG. 8 is an example of a horizontal image formed in this way.

Sub-images are generated from such horizontal images. For example, when the sub-image is set to have an angle of view A less than 360°, the sub-image is generated by cutting out the portion surrounded by the illustrated sub-region frame 320a. When such a sub-image is displayed in the sub-region 320, the subject image is displayed relatively large compared to other angles of view, so it is possible to check which subject image is shown in the sub-region as well. Cheap.

When the sub-image is set to the angle of view B of 360°, the sub-image is generated by cutting out the portion surrounded by the illustrated sub-region frame 320b. When such a sub-image is displayed in the sub-region 320, the angle of view for one round can be visually recognized at once, so that spatial comprehension is excellent. Note that when the angle of view A and the angle of view B are set, it is not necessary to form a horizontal image of 360° or more as the original image to be cut out.

When the sub-image is set to an angle of view C larger than 360°, the sub-image is generated by cutting out the portion surrounded by the illustrated sub-region frame 320c. When such a sub-image is displayed in the sub-area 320, the subject area in the vertical direction is enlarged, so it is easy to grasp the relationship in the vertical direction with the main image. Which angle of view to generate the sub-image is determined by which effect is prioritized. Alternatively, the configuration may be such that the user can specify and select the angle of view.

FIG. 9 is a diagram for explaining another technique for directly transitioning from one-image display to one-image display of the next image. In the examples of FIGS. 6 and 7, the swipe action causes a transition to another image, but in the example of FIG. 9, a tap action causes a transition to another image.

As in the left diagram of FIG. 6, it is assumed that the n-th image is displayed as one image at time t= _T0 (left diagram of FIG. 9). From this state, when the user taps the sub-region 320 twice, the displayed target image switches to the (n+1)th omnidirectional image. The right diagram in FIG. 9 shows a state in which the display is switched to the n+ ₁ -th omnidirectional image at time t=T1 when two tap operations are completed. That is, at the time t=T1, the omnidirectional image that is ordered _one after the omnidirectional image that has been displayed until then is displayed as a new target image. In this case, "the sub-region 320 is tapped twice" is set as the update condition.

Here, the omnidirectional image that is ordered after one is set as a new target image. may be used as a target image. When the transition is made by a swipe operation, the omnidirectional image that is ordered after one is set as a new target image. A celestial image may be used as a new target image.

Also, when the user taps the sub-region 320 three times from the state of time t=T ₀ , the displayed target image switches to the (n−1)th omnidirectional image. That is, the omnidirectional image that is ordered one before the omnidirectional image that has been displayed so far is displayed as a new target image. In this case, "the sub-region 320 is tapped three times" is set as the update condition.

Here, the omnidirectional image that is ordered one before is used as a new target image. may be used as a new target image. Also, when the transition is made by swiping, the previously ordered omnidirectional image is set as the new target image. The omnidirectional image may be used as the new target image.

The update conditions for transitioning the target image to the omnidirectional images ordered forward and backward are not limited to these. For example, in the examples of FIGS. 6 and 7, the update condition is the amount of transition of the sub-image accompanying the swipe motion. Update conditions may be set. For example, an update condition can be set such that if the sub-region 320 is swiped by a horizontal distance of L mm in less than X msec, a transition is made to another image. In this case, if a swipe motion is detected at a speed that transitions to another image, the main image does not have to be updated until the transition to another image is made. For a swipe operation slower than this, scrolling of both the main image and the sub-image should be continued without transitioning to another image. That is, it is preferable to change the update of the main image according to the mode of the contact operation on the sub-region 320 .

10A and 10B are diagrams illustrating changes in an image in response to a contact operation on the main area 310. FIG. As in the left diagram of FIG. 6, it is assumed that the n-th image is displayed as one image at time t= _T0 (left diagram of FIG. 10). From this state, when the user starts to swipe any of the main areas 310 from left to right, the main image scrolls rightward in the main area 310 . However, unlike the case of swiping the sub-region 320, the sub-image does not move with the transition of the main image and does not scroll.

The diagram in FIG. 10 shows the display at time t=T ₁ when the scrolling of the main image is completed, and shows how the user further performs a pinch-out operation on part of the main area 310 . When a pinch-out action is performed on a portion of the main area 310, the main image is updated to an enlarged image obtained by enlarging the area. Also at this time, the sub-image is not linked to the transition of the main image and is not enlarged. The right diagram of FIG. 10 shows the display state at time t=T ₂ when the enlargement of the main image is completed.

In this way, when a user's touch operation is received in the main area 310, the main image is updated and displayed according to the touch operation, but the sub-image is not updated. By controlling the display in this way, the user can freely appreciate the main image without affecting the display of the sub-image intended for understanding the space. The sub-image may be updated in conjunction with the update of the main screen. In this case, the sub-image may be updated only in the horizontal direction according to the amount of transition of the horizontal component of the main screen. In such a configuration, even if the main image does not show a characteristic subject, the user can recognize which direction the user is viewing by checking the sub-image.

11A and 11B are diagrams illustrating changes in an image in response to a contact operation on the main area 310. FIG. Specifically, it shows how the user has tapped one of the main areas 310 . When the user taps the main area 310 once while the main image is displayed in the main area 310 and the sub-image is displayed in the sub-area 320 as shown in the left diagram of FIG. The display of 320 is canceled. The area where sub-region 320 was present is incorporated into main region 310, revealing the portion of the image that was hidden by sub-region 320. FIG. In the illustrated example, the removal of subregion 320 allows the entire upper body of the person to be visible.

When the user taps the main area 310 once in the state shown in the right diagram of FIG. 11, the state returns to the state shown in the left diagram of FIG. 11 in which the sub-area 320 is displayed. When the sub-region 320 is set to be surrounded by the main region 310, providing a function for removing such a sub-region 320 can improve the viewing quality of the main image.

FIG. 12 is a diagram illustrating another layout of the main area 310 and the sub-areas 320. As shown in FIG. In the example described so far, the sub-region 320 was set surrounded by the main region 310, but as shown in the figure, they may be set as separate regions. Even with such a layout, any of the display modes described above can be implemented except for the functions described with reference to FIG.

Although generation of sub-images has been described with reference to FIG. 8, various other methods can be employed. FIG. 13 is a diagram illustrating generation of sub-images according to a subject image by another method. Specifically, the face detection function is used to detect the face region 340 in the horizontal image. Then, a sub-region frame 320d is applied to the horizontal direction that includes as many face regions 340 as possible, and a portion surrounded by this is cut out to generate a sub-image. By generating sub-images in this way, it is possible to enhance the spatial comprehension of an omnidirectional image in which a person is captured.

14A and 14B are diagrams for explaining generation of sub-images according to a subject image by still another method. Specifically, a sky area and ground area detection function is used to extract a sky boundary line 341 and a ground boundary line 342 from a horizontal image. Then, the sub-region frame 320d is applied to the horizontal direction that does not include the sky region and the ground region as much as possible, and the portion surrounded by this is cut out to generate a sub-image. For example, the sub-region frame 320d may be arranged such that the middle line between the sky boundary line 341 and the ground boundary line 342 coincides with the center line of the sub-region frame 320d. Generating sub-images in this way eliminates monotonous sky regions and ground regions as much as possible, thereby enhancing the spatial comprehension.

Alternatively, sub-images may be generated using a trained model obtained by learning the area of the main subject existing in the image area. Specifically, a horizontal image is input to a trained model, an area that is most likely to be the main subject is output, and a sub-area frame is applied so as to include that area to generate a sub-image. Alternatively, a sub-image is generated by outputting a plurality of areas whose probability of being the main subject is equal to or greater than a threshold, and applying a sub-area frame so as to include as many of them as possible. By generating sub-images in this way, it is possible to include not only people but also various subjects in the sub-images.

Next, the processing procedure of the arithmetic processing unit 110 regarding the transition of the target image in the one-image display described especially with reference to FIGS. 6 and 7 will be described. FIG. 15 is a flowchart showing the processing procedure. Here, the processing procedure of the processing explained except for FIG. 6 and FIG. 7 is omitted. The illustrated flow starts when the n-th omnidirectional image, which is the n-th omnidirectional image, is selected in the thumbnail display and the display is switched to single-image display.

The display control unit 111 displays a main image generated by cutting out part of the nth image in the main area 310 of the display panel 120, and displays a sub image generated by cutting out part of the nth image along the horizontal direction. Displayed in the sub-region 320 of the display panel 120 . With the n-th image displayed in this way, the image updating unit 112 checks whether the touch panel 130 has detected a swipe operation in the sub-region 320 in step S101. If the swipe motion is not detected, the process proceeds to step S102, and if it is detected, the process proceeds to step S103.

Upon proceeding to step S<b>102 , image updating unit 112 checks whether touch panel 130 has detected a tap operation on close button 330 . If no tapping action is detected, the process returns to step S101 to continue the one-image display of the n-th image. When the tap operation is detected, the display is switched to the thumbnail display, and the one-image display ends.

Proceeding to step S103, the image update unit 112 confirms whether the swipe operation on the sub-region 320 is leftward or rightward. If it is the left direction, the process proceeds to step S104, and if it is the right direction, the process proceeds to step S109.

Proceeding to step S104, the image updating unit 112 updates the entire image by splicing the image portion that continues to the right end of the original sub-image by the amount of the image that disappears from the left end of the sub-region 320 according to the leftward swipe amount. Cut out from the celestial image and update the sub image. Similarly, an image obtained by splicing an image portion that continues to the right end of the original main image by the amount of the image to disappear from the left end of the main area 310 is cut out from the omnidirectional image, and the main image is updated. The display control unit 111 displays the sub-image and the main image updated by the image updating unit 112 in the sub-region 320 and the main region 310, respectively, thereby scrolling both images to the left.

The image update unit 112 advances to step S105 and confirms whether or not the swipe motion to the left continues. If it continues, it progresses to step S106, and if it does not continue, it progresses to step S102. After proceeding to step S106, the image update unit 112 checks whether or not the transition amount of the sub-image has reached 360°. If not reached, the process returns to step S104 to continue scrolling to the left, and if reached, the process proceeds to step S107.

When proceeding to step S107, the image update unit 112 reads out the (n+1)-th omnidirectional image data from the storage unit 150, and updates the n-th image to the (n+1)-th image. In response to the update to the n+1th image, the display control unit 111 displays the main image generated by cutting out a part of the n+1th image in the main area 310 of the display panel 120, and horizontally moves a part of the n+1th image. A sub-image generated by cutting out along the line is displayed in the sub-region 320 of the display panel 120 . When the display processing of the n+1th image is completed, the arithmetic processing unit 110 updates the variable n to the value of n+1 in step S108. When updating of variables is completed, the process proceeds to step S102.

Proceeding to step S109, the image updating unit 112 updates the entire image by splicing the image portion continuing to the left end of the original sub-image by the amount of the image to disappear from the right end of the sub-region 320 according to the amount of swiping in the right direction. Cut out from the celestial image and update the sub image. Also, an image obtained by splicing an image portion continuing to the left end of the original main image by the amount of the image to be erased from the right end of the main area 310 is clipped from the omnidirectional image, and the main image is updated. The display control unit 111 displays the sub-image and the main image updated by the image updating unit 112 in the sub-region 320 and the main region 310, respectively, thereby scrolling both images to the right.

The image update unit 112 advances to step S110 and checks whether or not the swipe motion to the right continues. If it continues, it progresses to step S111, and if it does not continue, it progresses to step S102. When proceeding to step S111, the image update unit 112 confirms whether or not the transition amount of the sub-image has reached 360°. If not reached, the process returns to step S109 to continue scrolling to the right, and if reached, the process proceeds to step S112.

Proceeding to step S112, the image update unit 112 reads out the (n−1)th omnidirectional image data from the storage unit 150, and updates the nth image to the (n−1)th image. The display control unit 111 receives the update to the n−1th image, displays the main image generated by cutting out a part of the n−1th image in the main area 310 of the display panel 120, and displays the n−1th image. A sub-image generated by cutting out a portion along the horizontal direction is displayed in the sub-region 320 of the display panel 120 . When the display processing of the n−1th image is completed, the arithmetic processing unit 110 updates the variable n to the value of n−1 in step S113. When updating of variables is completed, the process proceeds to step S102. The transition of the target image in the one-image display is realized by the above processing procedure.

In the present embodiment described above, a still image was used as an example, but the omnidirectional image may be a moving image. When the omnidirectional images are moving images, still images designated as thumbnail images of the respective omnidirectional images may be used in the thumbnail display, and moving images may be reproduced and displayed in the one-image display. Alternatively, a still image or a moving image may be superimposed with another still image or a 3D object.

In the present embodiment described above, a layout example in which the sub-region 320 is arranged below the display region has been described, but a layout in which the sub-region 320 is arranged above the display region may be employed, and various display layouts may be adopted. can. Also, the specific numerical values described as the update conditions are examples, and other numerical values may be adopted. The sub-image transition amount may not be 360°, but may be smaller than 360°, such as 200°, or larger than 360°, such as 400°.

DESCRIPTION OF SYMBOLS 100... Smartphone, 110... Arithmetic processing part, 111... Display control part, 112... Image update part, 120... Display panel, 130... Touch panel, 140... Input/output IF, 150... Storage part, 210... Thumbnail area, 310... Main Area 320 Sub-area 330 Close button 340 Face area 341 Sky boundary 342 Ground boundary 500 Spherical camera 600 Image server 700 Network

Claims (20)

displaying a first image, which is an image generated by cutting out at least part of a target image selected from among a plurality of selectable omnidirectional images, in a first display area of a display device, and displaying a part of the target image; a display step of displaying a second image, which is an image generated by cutting out along the horizontal direction, in a second display area of the display;
When a user's contact operation along the horizontal direction is received in the second display area by a touch sensor provided superimposed on the display, the target image of the plurality of omnidirectional images is moved forward and backward. a display control program for causing a computer to execute an updating step of updating any of the ordered omnidirectional images to the newly selected target image. The updating step newly selects an image ordered after the target image from among the plurality of omnidirectional images when a contact operation of sliding in one of the horizontal directions is received. and when receiving a contact operation of sliding in a direction opposite to the one direction out of the horizontal directions, the target image is ordered before the target image among the plurality of omnidirectional images. 2. The display control program according to claim 1, wherein the current image is the newly selected target image. In the updating step, the second image is sequentially updated in conjunction with the contact operation and displayed in the second display area until the contact operation satisfies a preset update condition, and the update condition is satisfied. 3. The method according to claim 1 or 2, wherein, if received, one of the plurality of omnidirectional images ordered before or after the target image is updated to the newly selected target image. Display control program. 4. The display control program according to claim 3, wherein the update condition is set based on a transition amount of the second image transitioned by updating the second image. 5. The display control program according to claim 3, wherein the update condition is set based on the number of times the contact operation is repeatedly received. The display control program according to any one of claims 3 to 5, wherein the update condition is set based on the speed of the contact operation. 7. The display control program according to any one of claims 3 to 6, wherein said updating step successively updates said first image in conjunction with updating said second image and displays said first image in said first display area. 8. The display control program according to claim 7, wherein said updating step varies updating of said first image according to a mode of said contact operation. In the displaying step, when the touch sensor receives a user's contact operation in the first display area, the first image is updated and displayed according to the contact operation, and the second image is updated. The display control program according to any one of claims 1 to 8. 10. The display control program according to any one of claims 1 to 9, wherein in said display, said second display area is set to be surrounded by said first display area. 11. The display control program according to claim 10, wherein said displaying step stops displaying said second image in said second display area when said touch sensor receives a user's tap operation in said first display area. . 12. The display control program according to any one of claims 1 to 11, wherein the displaying step generates the second image from the target image so that an equator line in the omnidirectional image becomes a center line in the vertical direction. 12. The display control program according to any one of claims 1 to 11, wherein said displaying step generates said second image from said target image based on a region of a person's face appearing in said target image. 12. The display control program according to any one of claims 1 to 11, wherein the displaying step generates the second image from the target image based on respective regions of the sky and the ground reflected in the target image. 12. The display control according to any one of claims 1 to 11, wherein the displaying step generates the second image from the target image based on a region of a main subject recognized by a trained model for the target image. program. 16. The display control program according to any one of claims 1 to 15, wherein the displaying step generates the second image by cutting out a horizontal 360° range in the omnidirectional image from the target image. 16. The display control program according to any one of claims 1 to 15, wherein the display step generates the second image by cutting out a horizontal range of less than 360° in the omnidirectional image from the target image. In the displaying step, a horizontal 360° range in the omnidirectional image is cut out from the target image, and a continuous range is further connected to an end portion thereof to generate the second image having a range larger than 360°. 16. The display control program according to any one of items 1 to 15. displaying a first image, which is an image generated by cutting out at least part of a target image selected from among a plurality of selectable omnidirectional images, in a first display area of a display device, and displaying a part of the target image; a display step of displaying a second image, which is an image generated by cutting out along the horizontal direction, in a second display area of the display;
When a user's contact operation along the horizontal direction is received in the second display area by a touch sensor provided superimposed on the display, the target image of the plurality of omnidirectional images is moved forward and backward. and updating any of the ordered omnidirectional images to the newly selected target image. displaying a first image, which is an image generated by cutting out at least part of a target image selected from among a plurality of selectable omnidirectional images, in a first display area of a display device, and displaying a part of the target image; a display control unit that displays a second image, which is an image generated by cutting out along the horizontal direction, in a second display area of the display;
a touch sensor superimposed on the display;
The omnidirectional images are ordered one after another from the target image among the plurality of omnidirectional images when the touch sensor receives a user's contact operation along the horizontal direction in the second display area. an updating unit for updating any of the images to the newly selected target image.

Images