JPWO2019031397A1 - Image display device, image display method, and image display program - Google Patents

Abstract

In an image display device that displays a part of the image on the screen, after switching the image to be displayed on the screen, the user can continue to view the image without being confused when resuming the already displayed display. Provided is an image display device that can The image display device displays a display based on a display unit that can display a partial range of each image, an information input unit that inputs a signal according to a user's operation or operation, and a signal that is input from the information input unit. A display range setting unit for setting the display range of the image in the unit, a history information storage unit for storing information representing the display state of the image being displayed as history information, and a display unit in accordance with an instruction to switch the image. An image switching unit for switching images, and when a signal indicating an instruction to return the image to be displayed on the display unit to the already displayed image is input from the information input unit, the display range setting unit displays the history information. Based on this, the display range in the already displayed image is set.

Description

  The present invention relates to an image display device, an image display method, and an image display program.

  In recent years, an image (a so-called omnidirectional image) of a 360 ° subject around a shooting point has been widely used in business fields such as tourism guidance and real estate preview as well as entertainment fields such as games. Yes. The omnidirectional image is displayed in a panorama on a general two-dimensional display, or is processed into a virtual reality (VR) content and displayed in a three-dimensional manner. For example, a part of the omnidirectional image is displayed on a glasses-type or goggles-type display device called a head-mounted display (hereinafter also referred to as “HMD”), and according to the movement of the head of the user wearing the HMD, By changing the range of the image to be displayed on the screen, the user can feel as if he / she entered the space of the displayed image.

  As a technique related to the display of an omnidirectional image, Patent Document 1 discloses an image reproduction method for reproducing a spherical omnidirectional image captured by a spherical omnidirectional camera by an image reproducing apparatus having a gyro sensor. An image reproduction method is disclosed in which an event corresponding to a video is activated at the line of sight according to the line of sight of the viewer during reproduction.

JP, 2006-134742, A

  When panoramic or VR images such as omnidirectional images are displayed in panorama or VR, a general image display device displays a part of the omnidirectional image (hereinafter also referred to as a display range) on the screen. The display range changes according to the user's operation and the user's movement. As this display range, when an omnidirectional image is newly displayed or switched to another omnidirectional image, a range set in advance is set first. Therefore, when the user switches the display from a state in which a certain image is displayed to another image, and then performs an operation to return the display to an already displayed image, such as the immediately preceding image, the immediately preceding image is initialized. Display resumes from the set display range. For this reason, the user may be temporarily confused and confused about what image he / she is looking at even though the same image as the image he / she was looking at is displayed on the screen.

  The present invention has been made in view of the above, and in an image display device that displays a partial range in an image on a screen, after switching the image to be displayed on the screen, the display is returned to the already displayed image. An object of the present invention is to provide an image display device, an image display method, and a program that allow a user to continue viewing an image with a natural feeling without being confused even when the user restarts.

  In order to solve the above problems, an image display device according to one embodiment of the present invention is an image display device that displays a plurality of images including a moving image or a still image, and is a partial range of each of the plurality of images. Among the images to be displayed on the display unit based on the signal input from the information input unit, a signal input from the information input unit, and a display unit capable of displaying When a signal indicating an instruction to switch an image to be displayed on the display unit is input from the information input unit and a display range setting unit that sets a display range that is a range to be displayed on the display unit, the display A history information storage unit that stores information representing a display state of an image displayed on the unit as history information, an image switching unit that switches an image to be displayed on the display unit in accordance with an instruction to switch the image, and And when the signal indicating an instruction to return the image to be displayed on the display unit to the already displayed image is input from the information input unit, the display range setting unit is stored in the history information storage unit. The display range in the already displayed image is set based on the stored history information.

  In the image display device, the display range setting unit is a signal representing an instruction to switch an image to be displayed on the display unit from the information input unit to an image in which history information is not stored in the history information storage unit. Is input based on a reference coordinate set in advance in an image in which the history information is not stored and based on a signal input from the information input unit as needed. The position of the display range with respect to coordinates may be changed, and the history information storage unit may store the displacement of the display range with respect to the reference coordinates as the history information.

  In the image display device, a common tag is assigned to different first and second images of the plurality of images, and the first image is displayed on the display unit while the first image is displayed on the display unit. A tag display setting unit that superimposes and displays a tag attached to the image or information indicating that the tag is attached to the first image, and the tag or the tag is sent from the information input unit. When a signal for selecting information indicating that the information is given is input, the image switching unit may switch the image to be displayed on the display unit to the second image.

  In the image display device, a common tag is assigned to a first group and a second group of frames that are separated from each other among a plurality of frames constituting one moving image included in the plurality of images, and the first group Tag display for displaying a tag attached to the first group of frames or information indicating that the tag is attached superimposed on the first group of frames while the frame is displayed on the display unit A setting unit, and when the signal for selecting the tag or information indicating that the tag is attached is input from the information input unit, the image switching unit displays an image to be displayed on the display unit. You may switch to the second group of frames.

  An image processing method according to another aspect of the present invention is an image processing method executed by an image display device that displays a plurality of images including a moving image or a still image, and the image display device includes: A display unit capable of displaying a part of the range, and an information input unit for inputting a signal in accordance with a user operation or an operation by the user, and the display based on the signal input from the information input unit Step (a) of setting a display range which is a range to be displayed on the display unit among images to be displayed on the unit, and a signal indicating an instruction to switch the image to be displayed on the display unit from the information input unit. Step (b) of storing information representing the display state of the image displayed on the display unit as history information when input, and an image to be displayed on the display unit in accordance with an instruction to switch the image Step (c), and in step (a), a signal representing an instruction to return the image to be displayed on the display unit to the already displayed image is input from the information input unit The display range in the already displayed image is set based on the history information stored in step (b).

  An image display program according to another aspect of the present invention is an image display program that is executed by an image display device that displays a plurality of images including a moving image or a still image, and the image display device includes: A display unit capable of displaying a partial range of each, and an information input unit for inputting a signal according to a user's operation or a user's operation, based on the signal input from the information input unit, Step (a) of setting a display range that is a range to be displayed on the display unit among images to be displayed on the display unit, and a signal representing an instruction to switch the image to be displayed on the display unit from the information input unit Is input according to the step (b) of storing information representing the display state of the image displayed on the display unit as history information and an instruction to switch the image. Step (c) of switching an image to be displayed on the display unit is executed, and in step (a), an instruction is given from the information input unit to return the image to be displayed on the display unit to the already displayed image. When a signal to represent is input, the display range in the already displayed image is set based on the history information stored in step (b).

  According to the present invention, when switching the image to be displayed on the display unit, the information indicating the display state of the image displayed so far is stored as history information, and the image to be displayed on the display unit is changed to the already displayed image. When returning, the display range of the already displayed image is set based on the history information, so that the already displayed image can be displayed from a range familiar to the user. Therefore, the user can continue to view the image with a natural feeling without being confused.

1 is a block diagram illustrating a schematic configuration of an image display device according to a first embodiment of the present invention. It is a schematic diagram for demonstrating the relationship of the image file memorize | stored in the image data memory | storage part shown in FIG. It is a schematic diagram for demonstrating the setting method of a display range. It is a schematic diagram for demonstrating the setting method of a display range. 3 is a flowchart illustrating an operation of the image display device illustrated in FIG. 1. 3 is a flowchart illustrating an operation of the image display device illustrated in FIG. 1. It is a schematic diagram which illustrates the floor plan of real estate properties. FIG. 2 is a schematic view illustrating an image displayed on the image display device illustrated in FIG. 1. FIG. 2 is a schematic view illustrating an image displayed on the image display device illustrated in FIG. 1. FIG. 2 is a schematic view illustrating an image displayed on the image display device illustrated in FIG. 1. It is a block diagram which shows schematic structure of the image display apparatus which concerns on the 2nd Embodiment of this invention. It is a schematic diagram which shows the state with which the image display apparatus shown in FIG. 10 was mounted | worn by the user. It is a schematic diagram which shows an example of the screen displayed on the image display apparatus shown in FIG.

  Hereinafter, an image display device, an image display method, and an image display program according to embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited by these embodiments. Moreover, in description of each drawing, the same code | symbol is attached | subjected and shown to the same part.

(First embodiment)
FIG. 1 is a block diagram showing a schematic configuration of an image display apparatus according to the first embodiment of the present invention. The image display device 10 according to the present embodiment is configured by a general-purpose terminal device such as a personal computer, a notebook PC, a tablet terminal, a smartphone, or a portable game machine. As shown in FIG. 1, the image display apparatus 10 includes a communication interface 11, a display unit 12, an operation input unit 13, a storage unit 14, and a processor 15 that performs various arithmetic processes.

  The communication interface 11 connects the image display device 10 to a communication network such as the Internet, a LAN, or a telecommunication line, and transmits / receives information to / from other devices connected to the communication network. The communication interface 11 is configured using, for example, a soft modem, a cable modem, a wireless modem, an ADSL modem, or the like.

  The display unit 12 is a display including a display panel formed by, for example, liquid crystal or organic EL (electroluminescence) and a driving unit. The display unit 12 may be a display externally attached to the main body of the personal computer, or may be a display built in a notebook PC or tablet terminal.

  The operation input unit 13 is an input device such as an operation button, a keyboard, a mouse, and a touch panel provided on the display unit 12, and inputs a signal corresponding to an operation performed from the outside to the processor 15. That is, the operation input unit 13 functions as an information input unit that inputs information according to an operation by the user.

  The storage unit 14 is a computer-readable storage medium such as a semiconductor memory such as a ROM or a RAM. In addition to the operating system program and the driver program, the storage unit 14 stores application programs that execute various functions, various parameters that are used during the execution of these programs, and the like. Specifically, the storage unit 14 includes a program storage unit 141 that stores these programs and the like, an image data storage unit 142 that stores image data, and a history information storage unit 143. In addition, the storage unit 14 may store data of various contents such as sound output together when displaying an image.

  The image data storage unit 142 stores image data (image files) of a plurality of images (moving images or still images) displayed on the display unit 12. The image data may be previously read into the image display device 10 via a communication cable or a portable storage medium, or may be read as needed via the communication interface 11.

  The moving image or still image displayed on the display unit 12 may be an image (a so-called omnidirectional image) in which a subject at 360 ° around the shooting point is captured, or a fish-eye lens or a wide-angle lens (or an ultra-wide-angle lens). An image having a wide angle of view (for example, an angle of view of 200 ° or more) may be used. Or you may change the range of the subjects to be stored in one image between a moving image and a still image. Here, the general viewing angle of a stationary person is about 180 ° to the left and right (about 90 ° on one side), but when exercising (eg walking), other than the central region of the field of view (right and left The visibility of the edges, etc.) decreases. Therefore, for moving images, an image showing only a predetermined range centered on the moving direction of the user is taken, and for a still image, a spherical image is taken into consideration, such as an operation in which the user stops and looks back and looks around. It is also good.

FIG. 2 is a schematic diagram for explaining the relationship between image files stored in the image data storage unit 142. The image data storage unit 142 stores image files of moving images I _A and I _B composed of a plurality of frames I _{A (1)} , I _{A (2)} , ..., I _{B (1)} , I _{B (2)} , ... Still image I _C and I _D image files are stored. These image files are subjected to marking processing (processing for assigning tags) for associating different images with each other. For example, if the tag T ₀₁ is assigned to a part of the frame _{I A (t1) ~I A (} t1 + 5) of the video I _A, in the frame I _{B (1)} of the beginning of the video I _B, the same Tag T ₀₁ is assigned. Thus, the frame I _A in the video _{I A (t1) ~I A (} t1 + 5) and the video I _B are being linked. The frame I _A in the video _{I A (t3) ~I A (} t3 + 6) is the tag T ₀₃ is applied, a still image same tag T ₀₃ to I _C is tied. Thus, the frame I _A in the video I _A and _{(t3) ~I A (t3 +} 6) and still picture I _C are being linked. Arbitrary information such as the name of the shooting location and the location information of the shooting location can be used as the tag.

The association of images by tags may be performed between different moving images (for example, moving image I _A and moving image I _B ) or between a moving image and a still image (for example, moving image I _A and still image I _C ). It is good, and it may be performed between still images. For example, the same tag T ₀₄ is assigned to the still image I _C and the still image _ID . Thereby, the still image I _C and the still image _ID are linked. Furthermore, frames that are separated from each other in the same moving image may be associated with a tag. For example, the frame I _A in the video _{I A (t2) ~I A (} t2 + 2) and the frame I _{A (t4),} the same tag T ₀₂ is applied. As a result, the frames IA _{(t2) to} IA _{(t2 + 2)} and the frame IA _(t4) are linked.

  The history information storage unit 143 stores history information that is information related to the image being displayed on the display unit 12 when an instruction to switch the image to be displayed on the display unit 12 is input. The history information includes information for specifying an image displayed on the display unit 12 (an image number or a frame number in a moving image; hereinafter, also referred to as image identification information), and information indicating an image display state on the display unit 12 ( Hereinafter also referred to as status information). The state information will be described in detail later.

  The processor 15 is configured using, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), and comprehensively controls each unit of the image display device 10 by reading various programs stored in the program storage unit 141. At the same time, various arithmetic processes for displaying various images are executed.

  Specifically, the processor 15 reads an image display program stored in the program storage unit 141 to display an image on the display unit 12 based on the image data stored in the image data storage unit 142 and allows the user to view the image. Execute various functions for The function units realized by the processor 15 reading the image display program include an instruction receiving unit 151 and a display control unit 152. The instruction receiving unit 151 receives a signal input from the operation input unit 13 and inputs an instruction corresponding to the signal to the display control unit 152. The display control unit 152 controls image display on the display unit 12 in accordance with an instruction input from the instruction receiving unit 151. Specifically, the display control unit 152 includes a display range setting unit 153, an image switching unit 154, and a tag display setting unit 155.

  The display range setting unit 153 sets a range (hereinafter referred to as a display range) to be displayed on the display unit 12 at a time among images based on the image data stored in the image data storage unit 142. For example, when the image to be displayed is an omnidirectional image, the entire celestial sphere image is not displayed on the display unit 12 at the same time, and the user does not feel unnatural considering the general field of view of the person. A range (for example, a range where the angle of view in the horizontal direction is 180 ° or less) is extracted from the omnidirectional image as a display range and displayed on the display unit 12.

3A and 3B are schematic diagrams for explaining a method of setting a display range. 3A and 3B show a state in which the image I ₁ that is an omnidirectional image is projected onto the celestial sphere centered at the center point P1, and the celestial sphere cut in the horizontal direction is viewed from above. Has been. Hereinafter, the azimuth angle on the celestial sphere is φ, and the elevation angle is θ (not shown).

In the image I ₁ , the size of the display range V (solid line portion) displayed on the display unit 12 is, for example, a range of 180 ° in the azimuth direction (φ ₀ = 180 °) and 120 in the direction of the elevation angle. A fixed value is set such as a range of ° (θ ₀ = 120 °). This setting may be changed by the user.

On the other hand, the position of the display range V in the image I ₁ can be changed according to a signal input from the operation input unit 13. Specifically, when the history information regarding the image I ₁ is not stored in the history information storage unit 143, the display range setting unit 153 first sets the reference point (for example, the center point) P3 of the display range V to the image I ₁ . On the other hand, the position of the display range V is initialized so as to coincide with the preset reference coordinate P2 (see FIG. 3A). Thereafter, when the user 1 performs an operation of changing the display range V using the operation input unit 13 (see FIG. 1), the display range setting unit 153 causes the reference point of the display range V to be the amount corresponding to the operation. P3 is moved (see FIG. 3B). As a result, the position of the display range V is shifted, and the area in the image I ₁ newly set as the display range V is displayed on the display unit 12. Position of the display range V is represented as a displacement of the reference point P3 in the display range V relative to the reference coordinate P2 of the image I _1. In other words, it is represented by a change (Δφ, Δθ) in the direction of the vector u ′ from the center point P1 of the celestial sphere toward the reference point P3. This vector u ′ corresponds to the user's line-of-sight direction u when the user 1 is virtually set at the center point P1 of the celestial sphere.

State information of the history information stored in the history information storage section 143 described above, the displacement of the reference point P3 in the display range V relative to the reference coordinate P2 of the image I _1, i.e., the change in direction of the vector u '([Delta] [phi, Δθ).

  The image switching unit 154 switches the image to be displayed on the display unit 12 in accordance with the instruction input from the instruction receiving unit 151. Specifically, an image (see FIG. 2) associated with the image displayed when the instruction is input is extracted, and the image file is opened and displayed. Note that the display range V immediately after image switching is set to an initially set position (see FIG. 3A).

  The tag display setting unit 155 superimposes a tag attached to the image (a still image or a frame in a moving image) or an icon indicating that the tag is attached to the image displayed on the display unit 12. To display.

  Next, the operation of the image display device 10 will be described. In the following, as an example, a case will be described in which an image (still image and moving image) showing the inside of a real estate property is panorama displayed. 4 and 5 are flowcharts showing the operation of the image display apparatus 10. FIG. 6 is a schematic diagram illustrating the floor plan of a real estate property. 7 to 9 are schematic views illustrating images displayed on the image display device 10. Note that an image file of an image to be displayed on the image display device 10 may be stored in the image data storage unit 142 in advance, or may be stored in a server connected to the image display device 10 via a communication network. You may make it read.

  In the following, it is assumed that moving image shooting is performed along the moving image shooting route R1, and still image shooting is performed at the still image shooting points P1 and P2, as shown in FIG. Of the moving images shot along the moving image shooting route R1, the frame in which the vicinity of the marking point M1 is shown is linked to the moving image showing the washroom by a common tag. A frame in which the vicinity of the marking point M2 is reflected is linked to a still image in which a living dining kitchen (LDK) is reflected by a common tag. The frame in which the vicinity of the marking point M3 is reflected is linked to the still image in which the scenery outside the veranda is reflected by a common tag. Further, the still images taken at the still image shooting points P1 and P2 are also associated with a common tag.

First, in step S100, the image display apparatus 10 opens a moving image file and reproduces the moving image. For example, when playing back a moving image that captures the moving image shooting route R1, a scene I _m (1) facing the hallway from the entrance is displayed on the display unit 12, as shown in FIG. Thereafter, when the reproduction of the moving image is continued, the LDK scene I _m (2), the veranda scene I _m (3), and the like are sequentially displayed on the display unit 12.

  The display range V displayed on the display unit 12 is initially set at the beginning of the reproduction of the moving image, that is, the reference coordinates P2 of the displayed image and the reference point P3 of the display range V coincide with each other. (See FIG. 3A). On the other hand, when a signal indicating an instruction to change the display range V is input from the operation input unit 13, the display range V moves (see FIG. 3B). The operation for changing the display range V may be an operation of clicking or tapping an arbitrary position on the screen with a mouse, or an operation of clicking or tapping an icon displayed on the screen. good. The user can use the operation input unit 13 to proceed with viewing while looking at a desired direction in the image.

  In subsequent step S101, the image display device 10 acquires a time from the start of the reproduction of the moving image to the present (reproduction time), in other words, a frame number of the moving image.

  In subsequent step S102, the image display apparatus 10 determines whether or not an instruction to return to the image displayed immediately before has been input. For example, when a predetermined pointer operation (click operation, tap operation, etc.) using the operation input unit 13 on the screen of the display unit 12 or an operation of pressing a specific operation button or key is performed, the image display device 10 It is determined that an instruction to return to the previous image has been input. When an instruction to return to the previous image is input (step S102: Yes), the process proceeds to step S130 described later. On the other hand, when an instruction to return to the immediately preceding image is not input (step S102: No), the process proceeds to step S103.

  Here, the image displayed immediately before is an image displayed on the display unit 12 before the currently displayed image is displayed, and is an image (moving image) different from the currently displayed image. (Or still image), when a frame is skipped in the currently displayed moving image, the frame before skip is also included. Note that the determination in step S102 is omitted when the image display device 10 has never switched images, that is, displayed images with different image files or skipped frames in a moving image.

  In step S103, the image display apparatus 10 determines whether or not the current playback time is within the playback time of the marked frame. That is, it is determined whether or not a tag is attached to the currently displayed frame. If it is not within the playback time of the marked frame (step S103: No), the process proceeds to step S119.

  In step S119, the image display apparatus 10 determines whether or not to end the image display. For example, when the VR video being displayed reaches the final frame and a predetermined operation for ending the image display is performed using the operation input unit 13, it is determined that the image display is to be ended. When it is determined that the image display is to be ended (step S119: Yes), the image display device 10 ends the image display. On the other hand, if it is determined not to end the display of the image (step S119: No), the process returns to step S100. For example, when the displayed VR video has not yet reached the final frame (step S119: No), the playback of the VR video is continued (step S100).

  On the other hand, when the current playback time is within the playback time of the marked frame (step S103: Yes), the image display device 10 searches for an image (link destination image) linked to the frame being displayed. (Step S104). That is, a still image or a frame in a moving image to which the same tag as the tag given to the frame being displayed is assigned is searched.

  In subsequent step S <b> 105, the image display apparatus 10 determines whether or not the linked image is included in the currently displayed image (moving image) file. When the linked image is not included in the currently displayed image file (step S105: No), the image display device 10 opens the image file including the linked image (step S106). If the image file is stored in the server, the image file is read from the server.

Subsequently, the image display device 10 displays the tag attached to the currently displayed image (frame) on the screen (step S107). Or you may display the icon and text which show that the tag is provided instead of the tag. For example, in the scenes I _m (1), I _m (2), and I _m (3) shown in FIG. 7, icons T1 to T3 indicating that a tag is attached are displayed.

  In subsequent step S108, the image display apparatus 10 determines whether or not the displayed tag (or icon or text) has been selected. Specifically, when a predetermined pointer operation (click operation or the like) using the operation input unit 13 is performed on the displayed tag, it is determined that the tag is selected.

When the displayed tag (or icon or text) is not selected (step S108: No), the process proceeds to step S119. For example, when the icon T1 displayed in the scene I _m (1) shown in FIG. 7 is not selected, if it is not determined to end the image display (step S119: No), the moving image is reproduced as it is (step S119). S100), scene I _m (2), scene I _m (3), etc. are continuously displayed.

  On the other hand, when the displayed tag (or icon or text) is selected (step S108: Yes), the image display device 10 pauses the reproduction of the currently displayed moving image (step S109).

  In subsequent step S110, the image display device 10 acquires state information indicating the state of the display range V in the temporarily stopped moving image and stores it as history information. That is, as shown in FIG. 3B, the displacement of the reference point P3 of the display range V with respect to the reference coordinate P2 of the image, that is, the change in the direction of the vector u ′ (Δφ, Δθ) is stored.

  In subsequent step S111, the image display device 10 determines whether or not the image file opened in step S106 is a moving image. When the image file is not a moving image, that is, when it is a still image (step S111: No), the image display device 10 displays a still image on the display unit 12 based on the image file (step S112).

For example, when the icon T2 displayed in the scene I _m (2) of the LDK shown in FIG. 7 is selected (see step S108), the state information of the scene I _m (2) is stored (see step S109). As shown in FIG. 8, the LDK still image I _s (A) is displayed on the display unit 12. In this still image I _s (A), icons T11 to T15 for inputting an instruction to display another image associated with the still image I _s (A) and an instruction to return to the immediately preceding image are input. A return mark TR1 is displayed. A still image corresponding to the operated icon is displayed on the display unit 12 by a pointer operation using the operation input unit 13 for any of these icons T11 to T15. For example, when an operation of selecting the icon T11 is performed, the still image I _s (B) illustrated in FIG. 9 is displayed on the display unit 12. Also in this still image I _s (B), icons T21 to T24 and a return mark TR2 are displayed.

  In subsequent step S113, the image display device 10 displays an end tag on the screen on which the still image is displayed. This end tag is for inputting an instruction to end the display of the still image and resume the reproduction of the moving image.

  In subsequent step S114, the image display device 10 determines whether or not an instruction to return to the image displayed immediately before has been input. For example, in FIG. 8, when a pointer operation using the operation input unit 13 is performed on the return mark TR1 displayed on the screen, it is determined that an instruction to return to the previous image is input. When an instruction to return to the previous image is input (step S114: Yes), the process proceeds to step S130 described later. On the other hand, when an instruction to return to the immediately preceding image is not input (step S114: No), the process proceeds to step S115.

  In step S115, the image display apparatus 10 determines whether the end tag displayed in step S113 has been selected. The method for determining whether or not a tag has been selected is the same as in step S108.

  When the end tag is not selected (step S115: No), the process returns to step S112. In this case, the same still image continues to be displayed. On the other hand, when the end tag is selected (step S115: Yes), the image display apparatus 10 hides the still image (step S116), and then the process proceeds to step S119.

If the image file opened in step S106 is a moving image (step S111: Yes), the image display device 10 jumps to the frame associated with the tag selected in step S108 among the opened image files. (Step S117). Then, the display on the display unit 12 is switched to the moving image of the linked destination (step S118). Thereafter, if it is not determined to end the display of the image (step S119: No), the reproduction of the moving image is started from the jumped frame (step S100). For example, when the tag T ₀₁ displayed during the reproduction of the frames I _{A (t1) to} I _{A (t1 + 5)} of the moving image I _A shown in FIG. 2 is selected, the moving image I assigned with the same tag T ₀₁ is selected. Jump to frame IB ₍₁₎ at the beginning of _B and play the movie.

  In step S105, when the image to which the frame being displayed is linked is included in the image (moving image) file being displayed (step S105: Yes), the image display device 10 changes the image (frame) being displayed. The assigned tag is displayed on the screen (step S120).

  In subsequent step S121, the image display apparatus 10 determines whether or not the displayed tag is selected. The method for determining whether or not a tag has been selected is the same as in step S108.

When the tag is selected (step S121: Yes), the image display device 10 skips to the linked frame (step S122). Thereafter, the process proceeds to step S100, and the reproduction of the moving image is started from the skipped frame. For example, if the video I _A frame _{I A (t2) ~I A (} t2 + 2) tag T ₀₂ displayed on the screen during the display of that shown in FIG. 2 is selected, the same tag T ₀₂ is applied The moving image is reproduced while skipping to the frame I _{A (t4)} of the moving image I _A. On the other hand, when the tag is not selected (step S121: No), the process returns to step S100.

  When an instruction to return to the immediately preceding image is input in step S102 or S114 (steps S102 and S114: Yes), the image display device 10 uses the history information stored in step S110 to display the image displayed immediately before ( Image identification information and status information of the return destination image are acquired (step S130 in FIG. 5).

In subsequent step S131, the image display apparatus 10 determines whether or not the return destination image is a moving image based on the image identification information in the history information. If the return image is not a moving image, that is, if it is a still image (step S131: No), the image display device 10 opens the image file of the return destination still image (step S132). For example, when the return mark TR2 is selected on the screen on which the still image I _s (B) shown in FIG. 9 is displayed, the image file of the still image I _s (A) displayed just before that is opened. .

  In subsequent step S133, the image display device 10 sets the display range V in the return image based on the state information in the history information. For example, as shown in FIG. 3B, when the direction change of the vector u ′ is Δφ, the display is switched to another image, and then an instruction to return to the display of the image displayed immediately before is input. The display range V is set so that the change in the direction of the vector u ′ relative to the initial setting becomes Δφ again.

  In subsequent step S134, the image display apparatus 10 hides the currently displayed still image. Thereafter, the process proceeds to step S112, and the return still image is displayed in a state before the image switching (see FIG. 8).

If it is determined in step S131 that the return destination image is a moving image (step S131: Yes), the image display device 10 jumps to the playback time (frame) in the return destination movie (step S135). For example, when the return mark TR1 is selected on the screen on which the still image I _s (A) shown in FIG. 8 is displayed, the scene I _m (2) of the moving image displayed immediately before that is displayed (see FIG. 7). Jump to.

  In subsequent step S136, the image display device 10 switches the display to the moving image of the return destination. Further, in step S137, the image display device 10 sets the display range V in the return destination image based on the state information in the history information. The method for setting the display range V is the same as in step S133. Thereafter, the process proceeds to step S100, and reproduction of the return destination video is started (see FIG. 7).

  As described above, according to the first embodiment of the present invention, when an instruction to switch the image to be displayed on the display unit 12 is input, the state information indicating the state of the display range V of the displayed image is displayed. When an instruction to return to the image displayed immediately before is input after switching between images, the display range V is set based on the stored state information. You can continue to appreciate images with a natural feeling without being confused.

(Second Embodiment)
FIG. 10 is a block diagram showing a schematic configuration of an image display apparatus according to the second embodiment of the present invention. FIG. 11 is a schematic diagram illustrating a state where the image display device according to the present embodiment is mounted on the user. FIG. 12 is a schematic diagram illustrating an example of a screen displayed on the image display apparatus according to the present embodiment.

  The image display device 20 according to the present embodiment is configured by a small terminal device such as a smartphone or a portable game machine, for example. As shown in FIG. 11, the image display device 20 is attached to the goggle-type holder 2 and is used by being mounted on the head of the user 1. Alternatively, a dedicated device (so-called head mounted display) in which the image display device and the holder are integrated may be used as the image display device 20. The image display device 20 according to the present embodiment causes the user 1 to stereoscopically view an image (moving image or still image) displayed on the screen, and changes the image displayed on the screen according to the movement of the head of the user 1. This is a so-called VR display device that allows the user 1 to experience the feeling of entering a virtual space.

  As shown in FIG. 10, the image display device 20 includes a communication interface 21, a display unit 22, an operation input unit 23, a sensor unit 24, a photographing unit 25, a storage unit 26, and a processor 27. Among these, the hardware configuration of the communication interface 21, the display unit 22, the operation input unit 23, the storage unit 26, and the processor 27 includes the communication interface 11, the display unit 12, the operation input unit 13, and the imaging unit illustrated in FIG. 25, the storage unit 26, and the processor 27 have the same configuration. Further, as the operation input unit 23, a remote controller wirelessly connected to the main body of the image display device 20 may be used in addition to the operation buttons and the touch sensor provided in the display unit 12.

  The sensor unit 24 includes various sensors such as a gyro sensor and an acceleration sensor, detects the orientation and movement of the image display device 20, and inputs a detection signal to the processor 27. That is, the sensor unit 24 functions as an information input unit that inputs information according to the user's operation.

The imaging unit 25 is a camera built in a terminal device such as a smartphone.
Similar to the storage unit 14 shown in FIG. 1, the storage unit 26 includes a program storage unit 261, an image data storage unit 262, and a history information storage unit 263.

The functional units executed by the processor 27 include an attitude information acquisition unit 271, an instruction reception unit 272, and a display control unit 273.
The posture information acquisition unit 271 acquires information representing the orientation and posture of the image display device 20 (hereinafter referred to as posture information) based on the detection signal input from the sensor unit 24. The orientation and orientation of the image display device 20 represent the orientation and orientation of the head of the user 1 wearing the image display device 20, in other words, the user's 1 line-of-sight direction.

The instruction receiving unit 272 receives a signal input from the operation input unit 23 and inputs an instruction corresponding to the signal to the display control unit 273.
The display control unit 273 includes a display range setting unit 274, an image switching unit 275, and a tag display setting unit 276.

  The display range setting unit 274 sets a display range V (see FIGS. 3A and 3B) to be displayed on the display unit 22 based on the posture information acquired by the posture information acquisition unit 271. Specifically, the display range V is set by moving the reference point P3 in accordance with the change in the line-of-sight direction u of the user 1.

  The operations of the image switching unit 275 and the tag display setting unit 276 are the same as the operations of the image switching unit 154 and the tag display setting unit 155 shown in FIG.

  In the holder 2 shown in FIG. 11, two lenses are respectively attached at positions corresponding to the left and right eyes of the user 1. When making the user 1 view the image stereoscopically, as shown in FIG. 12, two images with parallax are displayed in the two regions 22 a and 22 b in the display unit 22, respectively. The user 1 can recognize the images shown in the regions 22a and 22b three-dimensionally by viewing the regions 22a and 22b through the two lenses in the holder 2.

  The operation of the image display device 20 is the same as that of the first embodiment (see FIGS. 4 and 5) as a whole. Selection of a tag, a return mark, or the like displayed on the display unit 22 may be performed using, for example, a remote controller. Alternatively, the photographing unit 25 may be activated to capture the user 1's gesture as a video and display it superimposed on the image displayed on the display unit 22 so that the tag can be selected using the gesture.

(Modification)
In the first and second embodiments, a tag attached to an image (still image or moving image) being displayed on the display unit or an icon indicating that the tag is attached is superimposed on the image. When an operation for selecting a tag or an icon is performed, the display is switched to another image associated with the tag. However, the operation for switching the image to be displayed on the display unit is not limited to this.

  For example, a time bar indicating the progress of moving image playback may be displayed on the display unit, and the playback location (frame) of the moving image may be switched according to an operation of selecting a desired position on the time bar. Alternatively, a list of thumbnails of images related to the image being displayed on the display unit may be displayed on a part of the screen, and the display may be switched to the image for which the thumbnail has been selected in accordance with an operation for selecting the thumbnail. As long as the video is being displayed, the thumbnail to be displayed may be a still image linked to one of the frames in the video, a thumbnail of another video, or a thumbnail of each frame in the same video. There may be. Further, as long as a still image is being displayed, it may be a thumbnail of another still image, or a thumbnail of a frame in a moving image associated with the still image.

  When an operation for switching an image to be displayed on the display unit (for example, an operation for tapping one point on the time bar displayed on the screen or an operation for tapping a thumbnail) is performed on the image display device, the image display device Stores the history information (image identification information and status information) of the image (or frame) that has been displayed so far, and then switches the image to be displayed on the display unit. Thereafter, when an operation to return the image to be displayed on the display unit to the previous image is performed, the image display device sets the display range of the image (see FIG. 3B) based on the stored history information, and the previous image Is displayed. Alternatively, when an image switching operation is performed on the image display device, if the image display device has already stored history information regarding the switching destination image (such as an image with a thumbnail selected), the image display device The display range is set based on the history information, and the switching destination image is displayed. In the latter case, the display range is set based on the history information of the image, not only the image displayed immediately before, but also when jumping back to the image displayed a while ago and returning. The user can see a familiar screen and can continue to view the image without being confused.

  The present invention is not limited to the first and second embodiments and modifications described above, but by appropriately combining a plurality of components disclosed in the first and second embodiments and modifications. Various inventions can be formed. For example, some constituent elements may be excluded from all the constituent elements shown in the first and second embodiments and modifications, and the first and second embodiments and modifications may be used. You may form combining the shown component suitably.

DESCRIPTION OF SYMBOLS 1 User 2 Holder 10, 20 Image display apparatus 11, 21 Communication interface 12, 22 Display part 13, 23 Operation input part 14, 26 Storage part 15, 27 Processor 22a, 22b Area 24 Sensor part 25 Shooting part 141, 261 Program storage Unit 142, 262 Image data storage unit 143, 263 History information storage unit 151, 272 Instruction reception unit 152, 273 Display control unit 153, 274 Display range setting unit 154, 275 Image switching unit 155, 276 Tag display setting unit 271 Posture information Acquisition department

Claims (6)

An image display device that displays a plurality of images including moving images or still images,
A display unit capable of displaying a partial range of each of the plurality of images;
An information input unit for inputting a signal in accordance with a user operation or an operation by the user;
A display range setting unit that sets a display range that is a range to be displayed on the display unit, among images to be displayed on the display unit, based on a signal input from the information input unit;
When a signal indicating an instruction to switch an image to be displayed on the display unit is input from the information input unit, information indicating a display state of the image displayed on the display unit is stored as history information. A history information storage unit;
In accordance with an instruction to switch the image, an image switching unit that switches an image to be displayed on the display unit;
With
When a signal indicating an instruction to return the image to be displayed on the display unit to the already displayed image is input from the information input unit, the display range setting unit is stored in the history information storage unit An image display device that sets the display range in the already displayed image based on the history information. When the display range setting unit receives from the information input unit a signal representing an instruction to switch an image to be displayed on the display unit to an image in which history information is not stored in the history information storage unit, The display range is set based on preset reference coordinates in an image in which the history information is not stored, and the display range relative to the reference coordinates is set based on a signal input from the information input unit as needed. Change the position,
The history information storage unit stores a displacement of the display range with respect to the reference coordinates as the history information.
The image display device according to claim 1. A common tag is assigned to different first and second images of the plurality of images,
While the first image is displayed on the display unit, a tag attached to the first image or information indicating that the tag is attached is displayed superimposed on the first image. A tag display setting unit,
When a signal for selecting the tag or information indicating that the tag is attached is input from the information input unit, the image switching unit sets an image to be displayed on the display unit as the second image. Switch,
The image display device according to claim 1. Among the plurality of frames constituting one moving image included in the plurality of images, a common tag is given to the first group and the second group of frames separated from each other,
While the frame of the first group is displayed on the display unit, a tag attached to the frame of the first group or information indicating that the tag is attached is superimposed on the frame of the first group. A tag display setting unit to be displayed,
When a signal for selecting the tag or information indicating that the tag is attached is input from the information input unit, the image switching unit displays an image to be displayed on the display unit in the second group of frames. Switch to
The image display device according to claim 1. An image processing method executed by an image display device that displays a plurality of images including moving images or still images,
The image display device includes:
A display unit capable of displaying a partial range of each of the plurality of images;
An information input unit for inputting a signal in accordance with a user operation or an operation by the user;
With
(A) setting a display range that is a range to be displayed on the display unit among images to be displayed on the display unit based on a signal input from the information input unit;
When a signal indicating an instruction to switch an image to be displayed on the display unit is input from the information input unit, information indicating a display state of the image displayed on the display unit is stored as history information. Step (b);
(C) switching an image to be displayed on the display unit in accordance with an instruction to switch the image;
Including
In step (a), when a signal indicating an instruction to return the image to be displayed on the display unit to the already displayed image is input from the information input unit, the history stored in step (b) An image display method for setting the display range in the already displayed image based on information. An image display program to be executed by an image display device that displays a plurality of images including moving images or still images,
The image display device includes:
A display unit capable of displaying a partial range of each of the plurality of images;
An information input unit for inputting a signal in accordance with a user operation or an operation by the user;
With
(A) setting a display range that is a range to be displayed on the display unit among images to be displayed on the display unit based on a signal input from the information input unit;
When a signal indicating an instruction to switch an image to be displayed on the display unit is input from the information input unit, information indicating a display state of the image displayed on the display unit is stored as history information. Step (b);
(C) switching an image to be displayed on the display unit in accordance with an instruction to switch the image;
And execute
In step (a), when a signal indicating an instruction to return the image to be displayed on the display unit to the already displayed image is input from the information input unit, the history stored in step (b) An image display program for setting the display range in the already displayed image based on information.

Images