JP6443505B2 - Program, display control apparatus, and display control method - Google Patents

Description

  The present invention relates to a display control device, a program, and a recording medium.

  A technique is known in which the omnidirectional image of a shooting point is photographed and the omnidirectional image of the shooting point is converted into a single image. For example, Patent Document 1 discloses an invention of an observation apparatus that converts an image obtained by photographing all directions of a photographing point into a display image and displays the display image on a display unit of a cylindrical casing. . In addition, there is known a technique for displaying a part of an image in which all directions of a shooting point are taken as one image on a personal computer display or the like in a format in which a predetermined direction is actually viewed from the shooting point. In addition, a technique for freely changing the display range (direction) of an image according to a user operation is known.

  However, with the conventional technology, it has been difficult for the user to grasp the relationship between the display range of the image displayed on the screen and information indicating a list of viewpoints on the image.

  The present invention has been made in view of the above, and a display control device, a program, and a recording for clarifying a relationship between a display range of an image displayed on a screen and information indicating a list of viewpoints on the image The purpose is to provide a medium.

In order to solve the above-described problems and achieve the object, the present invention provides a computer with an image of a partial region of the omnidirectional image from an omnidirectional image, and an arbitrary image on the omnidirectional image. A generation step of generating a first image including a plurality of viewpoints; a display step of displaying a display image including a second image that is a part of the spherical image and a plurality of the first images on a display device; Receiving an input indicating change of the display image in response to an operation on the plurality of first images or the second image displayed in the display step, and the display step includes The second image displayed in the display step is changed in response to an input indicating a change to two images, and the first image corresponding to the changed region of the second image is displayed at a predetermined position. As order Change, the modified and the first image containing the viewpoint included in the second image, the changed size of different display formats of a region and the first image including a viewpoint that is not included in the second image Is displayed.

  According to the present invention, there is an effect that the relationship between the display range of the image displayed on the screen and the information indicating the list of viewpoints on the image can be clarified.

FIG. 1 is a diagram illustrating an example of the configuration of the display control apparatus according to the first embodiment. FIG. 2 is a diagram illustrating an example of an omnidirectional image according to the first embodiment. FIG. 3 is a diagram illustrating an example of a display image displayed on the display device of the first embodiment. FIG. 4 is a diagram illustrating a change example of the display image displayed on the display device of the first embodiment. FIG. 5 is a diagram for explaining the position of the viewpoint on the omnidirectional image of the first embodiment. FIG. 6 is a diagram for explaining the relationship between the angle of view, the camera angle, and the camera position. FIG. 7 is a diagram for explaining the relationship between the angle of view, the camera angle, and the camera position. FIG. 8 is a diagram illustrating an example of viewpoint list information of the display control apparatus according to the first embodiment. FIG. 9 is a flowchart illustrating an example of the display control method according to the first embodiment. FIG. 10 is a diagram illustrating a modification example of the display image displayed on the display device according to the first embodiment. FIG. 11 is a diagram illustrating an example of a display image displayed on the display device of the second embodiment. FIG. 12 is a diagram illustrating a modification example of the display image displayed on the display device according to the second embodiment. FIG. 13 is a diagram illustrating an example of viewpoint list information of the display control apparatus according to the third embodiment. FIG. 14 is a diagram illustrating an example of a display image displayed on the display device of the third embodiment. FIG. 15 is a diagram illustrating an example of a hardware configuration of a main part of the display control device according to the first to third embodiments.

  Hereinafter, embodiments of a display control device, a program, and a recording medium will be described in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a diagram illustrating an example of the configuration of the display control apparatus 10 according to the first embodiment. The display control apparatus 10 of this embodiment includes an acquisition unit 11, a storage unit 12, a generation unit 13, a display control unit 14, and a reception unit 15. The display control device 10 is connected to the display device 20. The display control device 10 controls an image displayed on the display device 20 (hereinafter referred to as “display image”).

  The acquisition unit 11 acquires an omnidirectional image that is an image in which the omnidirectional image of the imaging point is captured from another device such as an imaging device that can generate the omnidirectional image. An imaging device includes, for example, an imaging unit that captures all directions of an imaging point using two fisheye lenses facing in opposite directions, and a creation unit that combines two images captured by the two fisheye lenses into one image The omnidirectional image can be created. In addition, the acquisition part 11 does not need to acquire an omnidirectional image from another apparatus by providing the function of the said imaging device. Further, in the present embodiment, the omnidirectional image is illustrated, but not limited to the omnidirectional image, it may be a normal image, a panoramic image extending horizontally or vertically, and the like. Therefore, in the following embodiments, the omnidirectional image is simply referred to as an image, and it is needless to say that an omnidirectional partial image described later can be simply referred to as a partial image.

  The storage unit 12 stores the omnidirectional image. FIG. 2 is a diagram illustrating an example of an omnidirectional image according to the first embodiment. As shown in FIG. 2, the omnidirectional image is an image in which the omnidirectional image of the shooting point is taken, and thus becomes an image like a panoramic image. In FIG. 2, the horizontal portion is shown in all directions, but the upper or lower portion is not displayed. When displaying the upper part or the lower part, the display is made by the user virtually moving in the image. There are a plurality of viewpoints (A to G) on the omnidirectional image. Here, the viewpoint is information that indicates an area that a user who has created an omnidirectional image or who uses the omnidirectional image pays attention to. An image obtained by cutting out the region including the viewpoint (corresponding to a later-described omnidirectional partial image) is an image shown in viewpoint list information described later. Therefore, the image shown in this viewpoint list information is associated with the position on the omnidirectional image including this image. FIG. 2 is an example when the viewpoints A to G exist on the omnidirectional image. The storage unit 12 stores information indicating the position of the viewpoint on the omnidirectional image (hereinafter referred to as “viewpoint list information”). Details of the omnidirectional image and viewpoint list information will be described later.

  The generation unit 13 generates an omnidirectional partial image that is an image of a partial region of the omnidirectional image from the omnidirectional image. The omnidirectional partial image includes an arbitrary viewpoint on the omnidirectional image at the center of the image. The omnidirectional image is an image including all azimuths of the shooting point, and the user can view the omnidirectional image in an arbitrary direction. Here, when the user wants to confirm the image in the direction he / she wants to see later or to show it to other users, the image is represented as an omnidirectional partial image including a predetermined viewpoint at the center. The generation unit 13 may generate an omnidirectional partial image corresponding to a predetermined viewpoint in advance. Specifically, the display control device 10 stores, in the storage unit 12, omnidirectional partial images that are frequently used, omnidirectional partial images corresponding to viewpoints registered in viewpoint list information described later, and the like. It may be left.

The display control unit 14 displays on the display device 20 a display image that is an image including an omnidirectional image and a viewpoint list image that is a list of thumbnails of a plurality of omnidirectional partial images. The display control device 10 displays the viewpoint list image together with the omnidirectional image on the display device 20, thereby improving the operability and convenience when the user selects the viewpoint. Here, omnidirectional image displayed on the display device 20 may have been displayed by the like expanding only a part of the area of the celestial sphere image. Hereinafter, the omnidirectional image may mean that all images are shown, or may mean an image in which only a part of the region of all images is displayed. In this way, even when only a part of the entire image area is displayed, the image outside the displayed area can be scrolled or touch panel operated to be moved to the display area. Can do.

  FIG. 3 is a diagram illustrating an example of a display image displayed on the display device 20 according to the first embodiment. The display image includes an omnidirectional image 30 showing a part of the omnidirectional image and a viewpoint list image 40. The omnidirectional image 30 in FIG. 3 is an example in the case where the viewpoint D on the omnidirectional image in FIG. 2 is included in the center of the image. That is, this is an example in which the display device 20 displays the omnidirectional image 30 corresponding to the viewpoint D. Note that the omnidirectional image may include a plurality of viewpoints on the omnidirectional image. For example, the omnidirectional image 30 includes the viewpoint C on the left side of the image and the viewpoint E on the right side of the image in addition to the viewpoint D.

  The region of the viewpoint list image 40 in FIG. 3 is an example in the case of being displayed at the lower part of the screen of the display device 20. In the region of the viewpoint list image 40, thumbnails of omnidirectional partial images corresponding to the viewpoints B to F are displayed. That is, five thumbnails are displayed, and other thumbnails are omitted by “...”. However, the display control unit 14 does not omit the display of other thumbnails, and the omnidirectional spheres of all viewpoints specified by reducing the size of the thumbnails according to the size of the area of the viewpoint list image 40. A thumbnail of the partial image may be displayed.

  The display control unit 14 controls to display a thumbnail of the viewpoint included in the center of the omnidirectional partial image at a predetermined position in the viewpoint list image. In the example of FIG. 3, the display control unit 14 displays an area corresponding to the omnidirectional partial image corresponding to the viewpoint D in the omnidirectional image 30 in the center, and displays a thumbnail of the omnidirectional partial image. Then, control is performed so that the viewpoint list image 40 is displayed at a predetermined position. In the display control apparatus 10 according to the present embodiment, the predetermined position is set to the center position of the viewpoint list image, whereby the relationship between the omnidirectional image displayed on the display apparatus 20 and the thumbnail of the viewpoint list image. It becomes possible to clarify.

  Further, the display control unit 14 sets the size of the thumbnail of the omnidirectional partial image including the viewpoint included in the omnidirectional image 30 of the display image to all the sizes including the viewpoints not included in the omnidirectional partial image of the display image. Make it larger than the thumbnail size of the celestial sphere partial image. In the example of FIG. 3, since the viewpoints C to E are included in the omnidirectional image 30 included in the display image, the display control unit 14 determines the size of the thumbnail of the omnidirectional partial image corresponding to the viewpoints C to E. To expand. Thereby, the display control apparatus 10 of this embodiment can clarify the relationship between the omnidirectional image currently displayed on the display apparatus 20, and the thumbnail of the omnidirectional partial image shown by a viewpoint list image. .

  Returning to FIG. 1, the accepting unit 15 accepts an input indicating a change in the omnidirectional image of the display image in response to an operation on the omnidirectional image or the viewpoint list image. The operation on the omnidirectional image is, for example, an operation of instructing (selecting) the viewpoint displayed on the omnidirectional image. In conjunction with this operation, for example, the omnidirectional image 30 in FIG. 3 moves. For example, when the user designates the viewpoint C on the omnidirectional image, the omnidirectional image 30 is changed to the omnidirectional image 30 with the viewpoint C at the center. Along with this, the viewpoint list image is also changed with the thumbnail of the omnidirectional partial image of the viewpoint C being centered.

  The operation on the viewpoint list image includes an operation of selecting a thumbnail of the omnidirectional partial image on the viewpoint list image and an omnidirectional sphere displayed at a predetermined position on the viewpoint list image by scrolling on the viewpoint list image. For example, an operation for changing a thumbnail of a partial image. In conjunction with this operation, for example, the omnidirectional image 30 in FIG. 3 moves. For example, when the user selects a thumbnail of the omnidirectional partial image of the viewpoint F on the viewpoint list image, the omnidirectional image 30 is changed to the omnidirectional image 30 having the viewpoint F as the center. Accordingly, the viewpoint list image is also changed with the thumbnail of the omnidirectional partial image of the viewpoint F being centered. In the display control apparatus 10 of the present embodiment, the predetermined position is the center position of the viewpoint list image.

  Note that the input device used by the user for the operation may be arbitrary. For example, the mouse, the keyboard, or the display device 20 may be a touch panel that serves as both a display unit and an input unit.

  The display control unit 14 changes the omnidirectional image of the display image according to the input indicating the change of the omnidirectional image, and newly displays the omnidirectional image according to the input indicating the change of the omnidirectional image. The order of the thumbnails of the viewpoint list image is changed so that the thumbnails of the omnidirectional partial images corresponding to the region are displayed in the center of the viewpoint list image. Note that there is no difference between the omnidirectional image before the change and the omnidirectional image after the change, and the display control unit 14 changes the area of the omnidirectional image to be displayed. Will be changed. With reference to FIGS. 4A to 4C, a display image change example will be specifically described.

  FIGS. 4A to 4C are diagrams showing examples of changing the display image displayed on the display device 20 of the first embodiment. The omnidirectional image 30 in FIG. 4B is a case where the omnidirectional partial image that is an image of the region including the viewpoint D is included in the center, and shows a display image before the change.

  FIG. 4C illustrates a case where the receiving unit 15 receives an instruction to change to the viewpoint E from the user. Specifically, the accepting unit 15 accepts an input indicating an instruction to change the display of the omnidirectional image included in the display image to the viewpoint E in accordance with a user operation. The user's operation is an operation of instructing the viewpoint E on the omnidirectional image 30, a thumbnail 401 of the omnidirectional partial image on the viewpoint list image 40 (an omnidirectional partial image that is an image of an area including the viewpoint E). The operation of selecting the thumbnail) or the operation of making the thumbnail 401 of the omnidirectional partial image the center of the viewpoint list image by scrolling on the viewpoint list image 40 (from the state of the viewpoint list image 40 to the state of the viewpoint list image 41) Operation).

  When the receiving unit 15 receives an input to change to the viewpoint E from the user, the display control unit 14 displays a display image including the omnidirectional image 31 and the viewpoint list image 41. That is, the display control unit 14 changes the display of the omnidirectional image included in the display image from the omnidirectional image 30 having the viewpoint D as the center to the omnidirectional image 31 having the viewpoint E as the center. At the same time, the display control unit 14 changes the viewpoint list image included in the display image from the viewpoint list image 40 to the viewpoint list image 41. Thus, the omnidirectional image 31 is an omnidirectional image that includes the omnidirectional partial image that is the region including the viewpoint E in the center. In addition, the viewpoint list image 41 displays a thumbnail corresponding to the viewpoint included in the omnidirectional image 31 at the center of the viewpoint list image 41, and the thumbnail size of the omnidirectional partial images corresponding to the viewpoints D, E, and F is large. The viewpoint list image is enlarged.

  FIG. 4A shows a case where the receiving unit 15 receives an instruction to change to the viewpoint C from the user. Specifically, the operation of the display control unit 14 when the receiving unit 15 receives an operation indicating an instruction to change to the viewpoint C from the user is the case where the operation indicating the instruction to change to the viewpoint E is received from the user. Since it is the same, description is abbreviate | omitted.

  The display image in FIG. 4A includes an omnidirectional image 32 and a viewpoint list image 42. The omnidirectional image 32 is an omnidirectional image that includes an omnidirectional partial image that is an area including the viewpoint C in the center. The viewpoint list image 42 displays a thumbnail corresponding to the omnidirectional partial image corresponding to the viewpoint included in the omnidirectional image 32 in the center of the viewpoint list image 42, and the omnidirectional partial images corresponding to the viewpoints B to D. This is a viewpoint list image in which the size of the thumbnails is enlarged.

  Next, before specifically explaining the display control method of the display control apparatus 10 of the first embodiment with reference to the flowchart, here, the coordinates representing the position on the omnidirectional image and the viewpoint list information are described. This will be described in detail. FIG. 5 is a diagram for explaining the position of the viewpoint on the omnidirectional image of the first embodiment. The display control device 10 according to the present embodiment displays the display device 20 on the display device 20 in such a form that the user watching the display device 20 is actually looking at a predetermined direction from the shooting point. Therefore, the display control apparatus 10 according to the present embodiment uses the omnidirectional image as a spherical image by associating the point on the omnidirectional image with the point on the sphere. Since the omnidirectional image is a planar image as shown in FIG. 2, the curvature is different from that of the spherical surface. However, by creating the omnidirectional image using the Mercator projection, the display control apparatus 10 can make the point on the omnidirectional image correspond to the point on the spherical surface.

  As shown in FIG. 5, the coordinates on the omnidirectional image are represented by an azimuth angle (horizontal coordinate) from the center point of the sphere and an elevation angle (vertical coordinate) from the center point of the sphere. Can do. The azimuth angle takes a value in the range of 0 ° to 360 ° in the clockwise direction. The elevation angle takes a value in the range of −90 ° to 90 °. Accordingly, the position of the viewpoint on the omnidirectional image can be represented by the azimuth angle and the elevation angle. The omnidirectional partial image is an image including an arbitrary viewpoint on the omnidirectional image in the center of the image. Therefore, the omnidirectional partial image can be represented by the coordinates of the central viewpoint of the image (azimuth angle and elevation angle) and the angle of view (camera zoom value).

  Here, a method in which the display control unit 14 adjusts the angle of view of the omnidirectional image displayed on the display device 20 by adjusting the camera angle and the camera position will be described. 6 and 7 are diagrams for explaining the relationship among the angle of view, the camera angle, and the camera position. FIG. 6 exemplifies ranges (ad) and specific values (AD) regarding the relationship between the angle of view, the camera angle, and the camera position. FIG. 7 shows the geometric positional relationship between the ranges (ad) in FIG. 6 and specific values (AD). The circumference of FIG. 7 shows the omnidirectional image. The angle of view represents the range of the scene (the size of the image) when viewing the viewpoint included in the center of the omnidirectional image from the center point of the sphere by the angle from the center point. The omnidirectional images shown in the range of this scene correspond to the omnidirectional images 30, 31, and 32 shown in FIGS.

  A specific example of adjusting the angle of view according to the camera angle and the camera position will be described. The example of A (angle of view 60 °) can be realized by setting the camera position to the center of the sphere (0.0) and the camera angle to 60 °. The example of B (angle of view 120 °) can be realized by setting the camera position backward by 1.0 from the center of the sphere to −1.0 and setting the camera angle to 60 °. The example of C (view angle 240 °) can be realized by setting the camera position backward by 1.0 from the center of the sphere to −1.0 and setting the camera angle to 120 °. The example of D (angle of view 286 °) can be realized by moving the camera position backward 1.38 from the center of the sphere to −1.38 and setting the camera angle to 120 °.

  As described above, the display control unit 14 can adjust the angle of view of the omnidirectional image displayed on the display device 20 by adjusting the camera angle and the camera position. In the description of this embodiment, the angle of view of the omnidirectional image is fixed to 120 ° (example B). Note that the angle of view of the omnidirectional image is not limited to 120 ° and may be arbitrary.

  FIG. 8 is a diagram illustrating an example of viewpoint list information of the display control apparatus 10 according to the first embodiment. The viewpoint list information includes a viewpoint, an azimuth angle, an elevation angle, an angle of view, and a thumbnail. The viewpoint is information for identifying the viewpoint. The azimuth indicates the horizontal position of the viewpoint on the omnidirectional image. The elevation angle indicates the vertical position of the viewpoint on the omnidirectional image. The angle of view indicates a range of the omnidirectional partial image including the viewpoint in the center of the image. The thumbnail is a reduced image of the omnidirectional partial image of the viewpoint displayed on the viewpoint list image. In the description of the present embodiment, in order to make it easy to understand that the processing for changing the display of the omnidirectional image and the processing for changing the display of the viewpoint list image work together, in the example of FIG. The difference in the position of G is caused only in the azimuth angle. Therefore, in the description of the present embodiment, the field angle is described as a horizontal field angle (horizontal field angle).

  Note that the values of the azimuth angle, the elevation angle, and the field angle of the viewpoint list information may be arbitrarily set. An example in which the difference in viewpoint position occurs not only in the azimuth angle (horizontal coordinate) but also in the elevation angle (vertical coordinate) will be described in the third embodiment.

  Next, a display control method of the display control apparatus 10 according to the first embodiment will be described. FIG. 9 is a flowchart illustrating an example of the display control method according to the first embodiment. FIG. 10 is a diagram illustrating a change example of the display image displayed on the display device 20 according to the first embodiment. With reference to FIGS. 8 to 10, the processing when the omnidirectional image of the display image is changed from the omnidirectional image 30 centered on the viewpoint D to the omnidirectional image 31 centered on the viewpoint E will be described. To do.

  The display control unit 14 determines the area of the omnidirectional image to be displayed on the display device 20, that is, the omnidirectional image to be displayed (step S1). When displaying the omnidirectional image for the first time, the display control unit 14 determines the omnidirectional image 30 including the viewpoint D as the omnidirectional image to be displayed on the display device 20. This is because, in the viewpoint list information of FIG. 8, the viewpoint D is used as a reference (0 °) for the azimuth angle and elevation angle in the omnidirectional image. Further, when the receiving unit 15 receives an input indicating a change in viewpoint from the user (step S <b> 6, which will be described later), the display control unit 14 displays on the display device 20 according to the input indicating the change in the viewpoint. Determine the spherical image. After that, the flow for the display shown in FIG. 10A will be described.

  Next, the display control unit 14 determines the order of thumbnails of the omnidirectional partial images displayed on the viewpoint list image (step S2). Here, the case where the omnidirectional image displayed on the display device 20 is determined to be the omnidirectional image 30 including the viewpoint D in step S1 will be specifically described. First, the display control unit 14 determines the order of the thumbnails according to the azimuth angle (horizontal coordinate) of the displayed omnidirectional image 30. In the example of the viewpoint list information in FIG. 8, the order of thumbnails is as follows: viewpoint D (0 °), viewpoint E (50 °), viewpoint F (100 °), The viewpoint G (150 °), the viewpoint A (200 °), the viewpoint B (250 °), and the viewpoint C (300 °). The azimuth angle is expressed in a clockwise direction. Next, the display control unit 14 rearranges the order of the thumbnails so that the thumbnail of the omnidirectional partial image corresponding to the viewpoint D becomes the center of the viewpoint list image. That is, the display control unit 14 distributes the thumbnails to the left and right within a range of 180 ° from the center with respect to the thumbnail of the omnidirectional partial image corresponding to the viewpoint D. Since the azimuth angle of the viewpoint D is 0 °, the left side is 0 ° to −180 ° (180 ° to 360 °), and the right side is 0 ° to + 180 °. Therefore, the order of thumbnails is as follows: viewpoint A (200 ° (−160 °)), viewpoint B (250 ° (−110 °)), viewpoint C (300 ° (−60 °)), viewpoint D (0 °), The viewpoint E (50 °), the viewpoint F (100 °), and the viewpoint G (150 °).

  Next, the display control unit 14 determines the size of the thumbnail of the omnidirectional partial image displayed in the viewpoint list image (step S3). Here, the case where the omnidirectional image displayed on the display device 20 is determined to be the omnidirectional image 30 including the viewpoint D in the center in step S1 will be specifically described. The display control unit 14 enlarges the size of the thumbnail of the omnidirectional partial image corresponding to the viewpoint included in the omnidirectional image 30. Referring to the viewpoint item information in FIG. 8, the azimuth angle of the viewpoint D is 0 ° and the angle of view is 120 °. Further, since the elevation angles of all the viewpoints A to G are 0 °, the viewpoint does not fall outside the range of the omnidirectional image 30 due to the difference in elevation angle. Therefore, the display control unit 14 enlarges the thumbnail of the omnidirectional partial image corresponding to the viewpoint whose azimuth angle is included in the range of the angle of view of the omnidirectional image 30. Since the azimuth angle of the viewpoint D is 0 ° and the angle of view of the omnidirectional image 30 including the viewpoint D in the center is 120 °, the display control unit 14 has an azimuth angle of + 60 ° to −60 ° (300 °). To be included in the omnidirectional image 30. Next, from the viewpoint list information, the display control unit 14 sets the viewpoint C (azimuth angle 300 degrees), viewpoint D (azimuth angle 0 degrees), viewpoint as the viewpoints whose azimuth angles are +60 degrees to -60 degrees (300 degrees). E (azimuth angle 50 °) is specified. Through the above procedure, the display control unit 14 enlarges the thumbnails of the omnidirectional partial images corresponding to the viewpoint C (azimuth angle 300 °), the viewpoint D (azimuth angle 0 °), and the viewpoint E (azimuth angle 50 °). Determine the thumbnail to enlarge.

  Next, the display control unit 14 determines a thumbnail of the omnidirectional partial image displayed on the viewpoint list image displayed on the display device 20 (step S4). Specifically, the thumbnail of the omnidirectional partial image to be displayed on the viewpoint list image is determined according to the omnidirectional image determined in step S1 and the size of the viewpoint list image. That is, a predetermined number of thumbnails of omnidirectional partial images of other viewpoints that are close in distance to the viewpoint included in the omnidirectional image determined in step S1 are selected in order of increasing distance. Here, it is assumed that the predetermined number is four.

  Regarding the thumbnails of the omnidirectional partial images displayed in the viewpoint list image, the order of the thumbnails determined in step S3 is viewpoint A (200 ° (−180 °)), viewpoint B (250 ° (−110 °)), The case of the viewpoint C (300 ° (−60 °)), the viewpoint D (0 °), the viewpoint E (50 °), the viewpoint F (100 °), and the viewpoint G (150 °) will be described. In this case, since viewpoint D (0 °) is the center of the viewpoint list image, viewpoint B (250 ° (−110 °)), viewpoint C (300 ° (−60 °)), viewpoint D (0 °) ), Thumbnails of omnidirectional partial images corresponding to viewpoint E (50 °) and viewpoint F (100 °) are displayed. That is, the display of thumbnails of the omnidirectional partial images corresponding to the viewpoint A (200 ° (−180 °)) and the viewpoint G (150 °) is omitted by “.

  The display control unit 14 displays a display image including the omnidirectional image determined in step S1 and the viewpoint list image created in steps S2 to S4 on the display device 20 (step S5). The display shown in FIG. 10A is performed by such flow control.

  Next, a flow for changing the display from FIG. 10A to the display of FIG. 10B will be described. First, the reception unit 15 determines whether or not an input indicating a change of the omnidirectional image (change of viewpoint) has been received from the user (step S6). When the input indicating the viewpoint change is not received (step S6, No), the process of detecting the input indicating the change of the omnidirectional image (change of the viewpoint) according to the user operation is continued.

  When the input indicating the change of the viewpoint is received (step S6, Yes), the process returns to step S1. Here, the case where the reception unit 15 receives an input indicating that the display of the omnidirectional image is changed to the omnidirectional image including the viewpoint E in the center will be specifically described. The display control unit 14 determines the omnidirectional image to be displayed on the display device 20 as the omnidirectional image 31 including the viewpoint E in the center (step S1).

  Next, the display control unit 14 determines the order of thumbnails of the omnidirectional partial images displayed on the viewpoint list image (step S2). First, the display control unit 14 determines the order of thumbnails according to the azimuth (horizontal coordinate). In the example of the viewpoint list information in FIG. 8, the order of thumbnails is as follows: viewpoint D (0 °), viewpoint E (50 °), viewpoint F (100 °), The viewpoint G (150 °), the viewpoint A (200 °), the viewpoint B (250 °), and the viewpoint C (300 °). The azimuth angle is expressed in a clockwise direction. Next, the display control unit 14 rearranges the order of the thumbnails so that the thumbnail of the omnidirectional partial image 31 corresponding to the viewpoint E is at the center of the viewpoint list image. That is, the display control unit 14 distributes the thumbnails to the left and right within a range of 180 ° from the center with respect to the thumbnail of the omnidirectional partial image 31 corresponding to the viewpoint E. Since the azimuth angle of the viewpoint E is 50 °, the left side is 50 ° to −130 ° (230 ° to 360 ° and 0 ° to 50 °), and the right side is 50 ° to + 230 °. Therefore, the order of thumbnails is as follows: viewpoint B (250 ° (−110 °)), viewpoint C (300 ° (−60 °)), viewpoint D (0 °), viewpoint E (50 °), viewpoint F (100 °). ), Viewpoint G (150 °), and viewpoint A (200 ° (−180 °)).

  Next, the display control unit 14 determines the size of the thumbnail of the omnidirectional partial image displayed in the viewpoint list image (step S3). As illustrated in FIG. 10B, the display control unit 14 enlarges the size of the thumbnail of the omnidirectional partial image corresponding to the viewpoint included in the omnidirectional image 31. Referring to the viewpoint item information in FIG. 8, the azimuth angle of the viewpoint E is 50 ° and the angle of view is 120 °. Further, since the elevation angles of all the viewpoints A to G are 0 °, the viewpoint does not fall outside the range of the omnidirectional image 31 due to the difference in elevation angle. Therefore, the display control unit 14 enlarges the thumbnail of the omnidirectional partial image corresponding to the viewpoint whose azimuth is included within the range of the angle of view of the omnidirectional image 31. The display control unit 14 has an azimuth angle of −10 ° (350 °) to + 110 ° because the azimuth angle of the viewpoint E is 50 ° and the angle of view of the omnidirectional image 31 including the viewpoint E in the center is 120 °. Are included in the omnidirectional image 31. Next, the display control unit 14 determines, from the viewpoint list information, as viewpoints having an azimuth angle of −10 ° (350 °) to + 110 °, a viewpoint D (azimuth angle 0 °), a viewpoint E (azimuth angle 50 °), and a viewpoint. Specify F (100 °). Through the above procedure, the display control unit 14 enlarges the size of the thumbnail of the omnidirectional partial image corresponding to the viewpoint D (azimuth angle 0 °), the viewpoint E (azimuth angle 50 °), and the viewpoint F (100 °). Decide what to do.

  Next, the display control unit 14 determines a thumbnail of the omnidirectional partial image displayed on the viewpoint list image displayed on the display device 20 (step S4). Since viewpoint E (50 °) is the center of the viewpoint list image, viewpoint C (300 ° (−60 °)), viewpoint D (0 °), viewpoint E (50 °), viewpoint F (100 °), A thumbnail of the omnidirectional partial image corresponding to the viewpoint G (150 °) is displayed. That is, the thumbnail display of the omnidirectional partial images corresponding to the viewpoint B (250 ° (−110 °)) and the viewpoint A (200 ° (−180 °)) is omitted by “.

  In summary, the display image displayed on the display device 20 is changed from FIG. 10A to FIG. 10B. In addition, in this Embodiment 1, although it has shown about the case where an elevation angle is the same in this way, with reference to FIG. 14 shown by the below-mentioned modification, even if it is a case where a viewpoint with a different elevation angle is included. The thumbnails of the viewpoints b, d, e, f, and h, which are viewpoints within the range where the omnidirectional image 60 is displayed, may be enlarged and displayed.

  The display control unit 14 of the display control device 10 according to the present embodiment displays a new omnidirectional image in response to an input indicating a change in the omnidirectional image (which can be said to be a change in the area in which the omnidirectional image is displayed). The thumbnail of the celestial sphere partial image corresponding to the area is displayed in the center of the viewpoint list image, and the thumbnail size of the omnidirectional partial image including the viewpoint included in the omnidirectional image of the display image in the center is displayed. Since the image is enlarged, the relationship between the display range of the image displayed on the screen and information indicating a list of viewpoints on the image can be clarified.

(Second Embodiment)
Next, the display control apparatus 10 of 2nd Embodiment is demonstrated. The display control unit 14 of the display control apparatus 10 of the present embodiment converts the size of the thumbnail of the omnidirectional partial image including the viewpoint included in the omnidirectional image of the display image in the center into the omnidirectional image of the display image. The size of the thumbnail of the omnidirectional partial image including the viewpoint not included in the center is made smaller. That is, the display control apparatus 10 of the first embodiment emphasizes the thumbnails of viewpoints displayed on the display apparatus 20 with viewpoint list images, but the display control apparatus 10 of the second embodiment is displayed on the display apparatus 20. Emphasize thumbnails of viewpoints that have not been viewed with viewpoint list images.

  Since the configuration of the display control apparatus 10 of the present embodiment is the same as that of the display control apparatus 10 of the first embodiment, description thereof is omitted. Since the description of the present embodiment is the same as the description of the first embodiment, differences from the first embodiment will be specifically described based on specific examples.

  FIG. 11 is a diagram illustrating an example of a display image displayed on the display device 20 according to the second embodiment. FIG. 11 is a diagram corresponding to FIG. 3 referred to in the description of the first embodiment. However, the viewpoint list image 50 is different from the viewpoint list image 40 of the first embodiment. That is, the display control unit 13 of the present embodiment enlarges the thumbnails of the omnidirectional partial images of the viewpoints A, B, F, and G that are not included in the omnidirectional image 30. Further, the display control unit 13 of the present embodiment does not enlarge the thumbnails of the omnidirectional partial images of the viewpoints C, D, and E included in the omnidirectional image 30. Although the thumbnail of the omnidirectional partial image at the viewpoint D is omitted, the thumbnail of the omnidirectional partial image at the viewpoint D may be displayed without being omitted.

  FIGS. 12A to 12C are diagrams showing examples of changing the display image displayed on the display device 20 of the second embodiment. FIG. 12 is a diagram corresponding to FIG. 4 referred to in the description of the first embodiment. The image shown in FIG. 12B is a case where the viewpoint D is included in the center of the omnidirectional image, and shows the display image before the change. When the reception unit 15 receives an instruction to change to the viewpoint E from the user, the display control unit 14 displays the display image illustrated in FIG. 12C including the omnidirectional image 31 and the viewpoint list image 51. First, the omnidirectional image 31 and the viewpoint list image 51 will be described. The omnidirectional image 31 is an omnidirectional image in which a region is set so that the display including the viewpoint E is included. The viewpoint list image 51 displays “...” in which the thumbnail of the omnidirectional partial image corresponding to the viewpoint included in the omnidirectional image 31 such as the viewpoint E is omitted in the center of the viewpoint list image 51, It is a viewpoint list image in which the size of the thumbnails of the omnidirectional partial images corresponding to the viewpoints B, C, G, and A that are not included in the spherical image 31 is enlarged.

  When the reception unit 15 receives an instruction to change to the viewpoint C from the user, that is, when the screen illustrated in FIG. 12A is displayed, the display control unit 14 displays the omnidirectional image 32 and the viewpoint. A display image including the list image 52 is displayed. The omnidirectional image 32 is an omnidirectional image in which a region is set so that the display includes the viewpoint C. The viewpoint list image 52 displays “...” In which the thumbnail of the omnidirectional partial image corresponding to the viewpoint included in the omnidirectional image 32 such as the viewpoint C is omitted in the center of the viewpoint list image 52. 4 is a viewpoint list image obtained by enlarging the thumbnail size of the omnidirectional partial images corresponding to viewpoints G, A, E, and F that are not included in the spherical image 32. FIG. In the second embodiment, the case where the elevation angles are the same is shown. However, referring to FIG. 14 shown in the modification example described later, even if the viewpoints include different elevation angles. The thumbnails of the viewpoints a, c, g, i, which are viewpoints outside the range where the omnidirectional image 60 is displayed, may be enlarged and displayed.

  The display control unit 14 of the display control apparatus 10 according to the present embodiment enlarges the size of the thumbnail of the omnidirectional partial image that includes the viewpoint that is not included in the omnidirectional image of the display image in the center, so that the viewpoint list image is displayed. Distinguish the size of thumbnails. Thereby, the relationship between the display range of the image displayed on the screen and the information indicating the list of viewpoints on the image can be clarified. In addition, it is possible to emphasize that there is another viewpoint by displaying thumbnails of viewpoints that are not displayed in comparison with viewpoints that are already displayed in the omnidirectional image.

(Third embodiment)
Next, the display control apparatus 10 of 3rd Embodiment is demonstrated. The display control unit 14 of the display control device 10 according to the present embodiment controls the display when a plurality of viewpoints having different elevation angles as well as azimuth angles are present in the omnidirectional image. Different from the control device 10. Since the configuration of the display control apparatus 10 of the present embodiment is the same as that of the display control apparatus 10 of the first embodiment, description thereof is omitted. Since the description of the present embodiment is the same as the description of the first embodiment, differences from the first embodiment will be specifically described based on specific examples.

  FIG. 13 is a diagram illustrating an example of viewpoint list information of the display control apparatus 10 according to the third embodiment. The viewpoint list information includes a viewpoint, an azimuth angle, an elevation angle, a horizontal angle of view, a vertical angle of view, and a thumbnail. The viewpoint is information for identifying the viewpoint. The azimuth indicates the horizontal position of the viewpoint on the omnidirectional image. The elevation angle indicates the vertical position of the viewpoint on the omnidirectional image. The horizontal angle of view indicates a horizontal range of the omnidirectional image including the viewpoint in the center of the image. The vertical angle of view indicates a range in the vertical direction of the omnidirectional image including the viewpoint at the center of the image. The thumbnail is a reduced image of the omnidirectional partial image including the viewpoint to be displayed in the viewpoint list image in the center. The coordinates on the omnidirectional image of the viewpoints a to i included in the viewpoint list information of FIG. 13 can be expressed by using the horizontal coordinate as the azimuth and the vertical coordinate as the elevation angle.

  FIG. 14 is a diagram illustrating an example of a display image displayed on the display device 20 according to the third embodiment. The display image includes an omnidirectional image 60 and a viewpoint list image 70. The omnidirectional image 60 in FIG. 14 is an example in the case where the viewpoint e on the omnidirectional image is included in the center of the image. That is, this is an example in which the display device 20 displays the omnidirectional image 60 corresponding to the display with the viewpoint e at the center. In addition to the viewpoint e, the omnidirectional image 60 includes a viewpoint b on the left side of the image, a viewpoint h on the right side of the image, a viewpoint d on the upper side of the center of the image, and a viewpoint f on the lower side of the center of the image. Note that the viewpoint a, viewpoint c, viewpoint g, and viewpoint i are outside the display range of the omnidirectional image 60. Thus, the omnidirectional image exists even outside the displayed range, and the omnidirectional image 60 means the omnidirectional image shown in the display area.

  The viewpoint list image 70 in FIG. 14 is an example when displayed at the lower part of the screen of the display device 20. The viewpoint list image 70 displays thumbnails of omnidirectional partial images corresponding to the viewpoints a to i. The display control unit 14 displays the thumbnail of the omnidirectional partial image 60 corresponding to the viewpoint e at a predetermined position in the viewpoint list image 70. In the display control apparatus 10 of the present embodiment, the predetermined position is the center position of the viewpoint list image.

  Next, the size of the thumbnail of the viewpoint list image 70 will be described. The display control unit 14 of the present embodiment determines the thumbnail of the viewpoint list image 70 whose size is to be enlarged as follows. First, the display control unit 14 determines that the viewpoint included in the omnidirectional image 60 is the viewpoint b based on the horizontal and vertical angles of view of the omnidirectional image 60 and the coordinates (azimuth angle and elevation angle) of each viewpoint. , Viewpoint d, viewpoint e, viewpoint f, and viewpoint h. The display control unit 14 determines whether the viewpoint is included in the image based on the horizontal angle of view of the omnidirectional image 60 in the horizontal direction (azimuth angle) of the coordinates of the viewpoint. For the vertical direction (elevation angle) of the coordinates of the viewpoint, the display control unit 14 determines whether the viewpoint is included in the image based on the vertical angle of view of the omnidirectional image 60. Note that the description of the method for determining whether or not the viewpoint is included in the omnidirectional image of the display image is the same as the description of the first embodiment, and thus detailed description thereof is omitted.

  The display control unit 14 determines to enlarge the thumbnail of the viewpoint e included in the center of the omnidirectional image 60. Next, the display control unit 14 determines whether or not to enlarge the thumbnail of the omnidirectional partial image that includes the viewpoint in the center for viewpoints other than the center of the omnidirectional image 60. The display control unit 14 refers to the elevation angle of each viewpoint in the viewpoint list information of FIG. 13 among the viewpoints included in the display range of the omnidirectional partial image 60, and includes the viewpoint e included in the center of the omnidirectional partial image 60 and A viewpoint having the same elevation angle (0 °) or an elevation angle within a certain predetermined range (for example, ± 10 °) is specified. The viewpoints are viewpoint b and viewpoint h. The display control unit 14 determines to enlarge the size of the thumbnail of the omnidirectional partial image including the viewpoint b in the center and the thumbnail of the omnidirectional partial image including the viewpoint h in the center. In summary, the display control unit 14 enlarges the size of the thumbnails of the omnidirectional partial images corresponding to the viewpoint b, the viewpoint e, and the viewpoint h.

  In other words, among the viewpoints included in the display range of the omnidirectional image 60, the display control unit 14 has a viewpoint in which the coordinates of the viewpoint e included in the center of the omnidirectional image 60 of the display image are different from the coordinates in the vertical direction. The thumbnail size of the omnidirectional partial image including (view point d and view point f) in the center is not enlarged. Further, as in the case of the first embodiment, the display control unit 14 does not enlarge the thumbnail of the omnidirectional partial image that includes a viewpoint that is not included in the display range of the omnidirectional image 60 in the center.

  The display control unit 14 of the display control apparatus 10 according to the present embodiment includes the coordinates of the viewpoint included in the center of the omnidirectional image of the display image and the coordinates in the vertical direction among the viewpoints included in the omnidirectional image of the display image. However, the thumbnail size of the omnidirectional partial image including different viewpoints in the center is not enlarged. According to the display control apparatus 10 of the present embodiment, even when the difference in viewpoint occurs in the horizontal direction and the vertical direction, information indicating the display range of the image displayed on the screen and the list of viewpoints on the image The relationship with can be clarified.

  As a modification of the present embodiment, when the horizontal coordinate is different, the size of the thumbnail of the omnidirectional partial image of the different viewpoint is not enlarged, and the vertical coordinate is the same or predetermined. The size of the thumbnail of the omnidirectional partial image of the viewpoint included in the range may be enlarged. In such a case, in the thumbnails of the viewpoint list image 70 shown in FIG. 14, the thumbnails of the viewpoints e, d, and f are enlarged, and the thumbnails of the viewpoints b and h are not enlarged.

  Finally, the hardware configuration of the main part of the display control apparatus 10 of the first to third embodiments will be described. FIG. 15 is a diagram illustrating an example of a hardware configuration of a main part of the display control apparatus 10 according to the first to third embodiments. The display control device 10 according to the first to third embodiments includes a control device 81, a main storage device 82, an auxiliary storage device 83, an input device 84, and a communication device 85. The control device 81, the main storage device 82, the auxiliary storage device 83, the input device 84, and the communication device 85 are connected to each other via a bus 86.

  The control device 81 executes the program read from the auxiliary storage device 83 to the main storage device 82. The main storage device 82 is a memory such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The auxiliary storage device 83 is, for example, a hard disk. The input device 84 is an interface for inputting information to the display control device 10. The communication device 85 is an interface for connecting to a network.

  A program executed by the display control apparatus 10 of the first to third embodiments is an installable or executable file such as a CD-ROM, a memory card, a CD-R, and a DVD (Digital Versatile Disk). The program may be recorded on a computer-readable recording medium and provided as a computer program product. The program executed by the display control apparatus 10 of the first to third embodiments may be provided by storing it on a computer connected to a network such as the Internet and downloading it via the network. Moreover, you may provide or distribute via networks, such as the internet, without downloading the program performed with the display control apparatus 10 of 1st-3rd embodiment. Further, the program of the display control apparatus 10 of the first to third embodiments may be provided by being incorporated in advance in a ROM or the like.

  The program executed by the display control device 10 according to the first to third embodiments is a functional block (acquisition unit 11, generation unit 13, display) that can be realized as a program among the functional blocks of the display control device 10 described above. The module configuration includes the control unit 14 and the receiving unit 15).

  As the actual hardware, each module is loaded onto the main storage device 82 when the control device 81 reads and executes the program from the recording medium. That is, each module is generated on the main storage device 82. Note that some or all of the functional blocks of the display control apparatus 10 may be realized by hardware such as an IC (Integrated Circuit) without being realized by a program.

DESCRIPTION OF SYMBOLS 10 Display control apparatus 11 Acquisition part 12 Storage part 13 Generation part 14 Display control part 15 Reception part 20 Display apparatus 81 Control apparatus 82 Main storage apparatus 83 Auxiliary storage apparatus 84 Input apparatus 85 Communication apparatus 86 Bus

JP 2010-148028 A

Claims (10)

On the computer,
Generating a first image that is an image of a partial region of the omnidirectional image from the omnidirectional image and includes an arbitrary viewpoint on the omnidirectional image;
A display step of displaying a display image including a second image that is a part of the omnidirectional image and a plurality of the first images on a display device;
A reception step of receiving an input indicating a change in the display image in response to an operation on the plurality of first images or the second image displayed in the display step;
And execute
The display step includes
The second image displayed by the display step is changed in response to an input indicating a change to the second image, and the first image corresponding to the changed region of the second image is at a predetermined position. The order is changed so as to be displayed, and the first image including the viewpoint included in the changed second image and the first image including the viewpoint not included in the changed second image are regions. A program that displays in different display formats. The program according to claim 1, wherein the predetermined position is a center position. The viewpoint is represented by horizontal coordinates and vertical coordinates,
The display step includes
The program according to claim 1 or 2, wherein the order of the first images is arranged based on the coordinates in the horizontal direction. The operation on the second image is:
The program according to any one of claims 1 to 3, which is an operation of instructing the second image as the viewpoint. The operations for the plurality of first images are:
2. An operation of selecting the first image from the plurality of first images, or an operation of changing the first image displayed at the predetermined position by scrolling on the plurality of first images. The program of any one of thru | or 4. On the computer,
Generating a first image that is an image of a partial region of the omnidirectional image from the omnidirectional image and includes an arbitrary viewpoint on the omnidirectional image;
A display step of displaying a display image including a second image that is a part of the omnidirectional image and a plurality of the first images on a display device;
And execute
The display step includes
A program for displaying the first image including a viewpoint included in the second image and the first image including a viewpoint not included in the second image in display formats having different area sizes . A generating unit that generates a first image that is an image of a partial area of the omnidirectional image and includes an arbitrary viewpoint on the omnidirectional image from an omnidirectional image;
A display control unit for displaying a display image including a second image that is a part of the omnidirectional image and a plurality of the first images on a display device;
A receiving unit that receives an input indicating a change in the display image in response to an operation on the plurality of first images or the second image displayed by the display control unit;
With
The display control unit
The second image displayed by the display control unit is changed in response to an input indicating a change to the second image, and the first image corresponding to the changed region of the second image is at a predetermined position. The first image including the viewpoint included in the changed second image and the first image including the viewpoint not included in the changed second image. display control device the size of the region is displayed by different display formats. A generating unit that generates a first image that is an image of a partial area of the omnidirectional image and includes an arbitrary viewpoint on the omnidirectional image from an omnidirectional image;
A display control unit for displaying a display image including a second image that is a part of the omnidirectional image and a plurality of the first images on a display device;
With
The display control unit
A display control apparatus that displays the first image including a viewpoint included in the second image and the first image including a viewpoint not included in the second image in display formats having different area sizes . Generating a first image that is an image of a partial region of the omnidirectional image from the omnidirectional image and includes an arbitrary viewpoint on the omnidirectional image;
A display step of displaying a display image including a second image that is a part of the omnidirectional image and a plurality of the first images on a display device;
A reception step of receiving an input indicating a change in the display image in response to an operation on the plurality of first images or the second image displayed in the display step;
Including
The display step includes
The second image displayed by the display step is changed in response to an input indicating a change to the second image, and the first image corresponding to the changed region of the second image is at a predetermined position. The order is changed so as to be displayed, and the first image including the viewpoint included in the changed second image and the first image including the viewpoint not included in the changed second image are regions. display control method size of displays with different display formats. Generating a first image that is an image of a partial region of the omnidirectional image from the omnidirectional image and includes an arbitrary viewpoint on the omnidirectional image;
A display step of displaying a display image including a second image that is a part of the omnidirectional image and a plurality of the first images on a display device;
Including
The display step includes
A display control method for displaying the first image including a viewpoint included in the second image and the first image including a viewpoint not included in the second image in display formats having different area sizes .

Images