JP2020057297A - Image display apparatus, image display method, and program - Google Patents

Abstract

To improve convenience upon browsing an image associated with positional information.SOLUTION: An image display apparatus 1 has an output unit 17, an input unit 16, an operation determining unit 112, a peripheral image acquiring unit 113, and a display control unit 115. The output unit 17 displays a captured image associated with positional information. The input unit 16 detects reception of an operation. The operation determining unit 112 determines whether or not change of a display mode of the captured image is instructed based on the operation detected by the input unit 16 when the output unit 17 displays the captured image. The peripheral image acquiring unit 113 acquires, from an outside of the image display apparatus 1, a peripheral image based on the positional image associated with the captured image. The display control unit 115, controls the peripheral image acquired by the peripheral image acquiring unit 113 to be displayed on the output unit 17 when the operation determining unit 112 determines affirmative.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an image display device, an image display method, and a program.

  2. Description of the Related Art Conventionally, there has been known a display processing device that displays information related to an image obtained by photographing when the image is viewed. For example, a display processing device that displays peripheral information of a position associated with an image as information related to the image is known (for example, see Patent Document 1).

JP 2010-218373 A

  However, the peripheral information in the general technology such as the technology disclosed in Patent Literature 1 is mainly an image of a map of a surrounding area or text information of a spot located in the map. For this reason, when a general technique is used, there is an aspect that the user lacks convenience in understanding the situation around the shooting position.

  The present invention has been made in view of such a situation, and an object of the present invention is to improve convenience when browsing an image associated with position information.

In order to achieve the above object, an image display device of one embodiment of the present invention
Display means for displaying a first image associated with position information;
Detecting means for detecting reception of an operation;
A determination unit configured to determine whether or not an instruction to change a display mode of the first image has been given based on an operation detected by the detection unit when the first image is displayed by the display unit;
Obtaining means for obtaining, from the outside of the apparatus, a second image captured at a position corresponding to the position information associated with the first image, when the determination means determines that the determination is affirmative;
Display control means for controlling the second image acquired by the acquisition means to be displayed on the display means,
It is characterized by having.

  According to the present invention, it is possible to improve convenience when browsing an image associated with position information.

1 is a system configuration diagram illustrating a configuration of an image display system S according to an embodiment of the present invention. FIG. 1 is a block diagram illustrating a hardware configuration of an image display device 1 according to an embodiment of the present invention. FIG. 3 is a functional block diagram illustrating a functional configuration for executing a peripheral image display process among the functional configurations of the image display device 1 of FIG. 2. 4 is a first flowchart illustrating a flow of a peripheral image display process performed by the image display device 1 of FIG. 1 having the functional configuration of FIG. 3 in the image display system S. 4 is a second flowchart illustrating a flow of a peripheral image display process performed by the image display device 1 of FIG. 1 having the functional configuration of FIG. 3 in the image display system S. FIG. 9 is a first schematic diagram illustrating a display example in a peripheral image display process. FIG. 14 is a second schematic diagram illustrating a display example in the peripheral image display processing. FIG. 14 is a third schematic diagram illustrating a display example in the peripheral image display processing. 13 is a flowchart illustrating a flow of a peripheral image display process according to a first modification.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[Overall system configuration]
FIG. 1 is a system configuration diagram showing a configuration of an image display system S according to an embodiment of the present invention.
As shown in FIG. 1, the image display system S includes an image display device 1, an imaging device 2, an image database 3, and a network 4. The image display device 1 and the imaging device 2 are communicably connected. Further, the image display device 1 and the image database 3 are communicably connected. These communications may be communications conforming to any standard. For example, communication between the image display device 1 and the imaging device 2 is realized by communication conforming to Wi-Fi (registered trademark). Further, for example, communication between the image display device 1 and the image database 3 is realized by communication via a network 4 which is the Internet.

The image display device 1 is a device having an image display function, a communication function, and a position positioning function. The image display device 1 is realized by, for example, a smartphone.
The imaging device 2 is a device having a shooting function and a communication function. The imaging device 2 is realized by, for example, a digital camera.

The image database 3 is a database that stores wide-angle landscape images (for example, omnidirectional images and omnidirectional images obtained by shooting 360 degrees around the shooting position) obtained by shooting in various places. The wide-angle landscape image is generated by, for example, shooting using a wide-angle fisheye lens. Alternatively, this wide-angle landscape image is generated by, for example, joining images captured from different viewpoint positions. Then, for example, the image database 3 is realized by a predetermined business operator accumulating the wide-angle landscape images generated in this manner.
For example, the image database 3 is realized by including a server device to which the technology described in the following <Reference> is applied.

<References>
JP 2011-527483 A

In the image display system S having such a configuration, the imaging device 2 generates an image (hereinafter, referred to as a “photographed image”) according to a user's photographing operation.
Further, the image display device 1 acquires a captured image from the imaging device 2 and associates the acquired captured image with position information. As the position information, for example, information measured by the image display device 1 is used. Here, in the present embodiment, it is assumed that the same user carries the image display device 1 and the imaging device 2 at a short distance. Therefore, the position information measured by the image display device 1 is substantially the position information indicating the position where the imaging device 2 has taken an image.

The image display device 1 acquires from the image database 3 a surrounding landscape image (hereinafter, referred to as a “peripheral image”) captured at a position corresponding to the position information.
Then, the image display device 1 further displays a peripheral image when a predetermined operation is received from the user when the captured image is displayed. For example, the captured image and the peripheral image are switched and displayed, or the captured image and the peripheral image are combined and displayed. The user who has referred to the peripheral image displayed in this way can easily understand the situation around the shooting position visually.
That is, according to the image display system S, it is possible to improve convenience when browsing an image associated with position information.

  Next, a hardware configuration and functional blocks of the image display device 1 that performs main processing in the image display system S will be described with reference to FIGS.

[Hardware configuration]
FIG. 2 is a block diagram showing a hardware configuration of the image display device 1 according to an embodiment of the image display device of the present invention.
As shown in FIG. 2, the image display device 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, An input unit 16, an output unit 17, a storage unit 18, a communication unit 19, an imaging unit 20, a positioning unit 21, and a drive 22 are provided.

  The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 18 to the RAM 13. These programs are, for example, programs such as an operating system and firmware for operating each unit of the image display device 1, or application software for realizing the function blocks described later.

  The RAM 13 also stores data and the like necessary for the CPU 11 to execute various processes.

  The CPU 11, the ROM 12, and the RAM 13 are mutually connected via a bus 14. The bus 14 is also connected to an input / output interface 15. The input / output interface 15 is connected to an input unit 16, an output unit 17, a storage unit 18, a communication unit 19, an imaging unit 20, a positioning unit 21, and a drive 22.

The input unit 16 is configured by various buttons and the like, receives an instruction operation of the user, and inputs various information according to the received instruction operation.
The output unit 17 includes a display, a speaker, and the like, and outputs images and sounds. Note that, for example, the input unit 16 and the output unit 17 may be integrally realized by a touch panel.
The storage unit 18 is configured by a hard disk, a DRAM (Dynamic Random Access Memory), or the like, and stores data of various images.
The communication unit 19 controls communication performed with another device. For example, it controls communication performed with the imaging device 2 and communication performed with the image database 3 described above with reference to FIG.

Although not shown, the imaging unit 20 includes an optical lens unit and an image sensor.
The optical lens unit includes a lens that condenses light, such as a focus lens and a zoom lens, for photographing a subject.
The focus lens is a lens that forms a subject image on the light receiving surface of the image sensor. A zoom lens is a lens that changes the focal length freely within a certain range.
The optical lens unit is also provided with a peripheral circuit for adjusting setting parameters such as focus, exposure, white balance, and the like, as necessary.

The image sensor includes a photoelectric conversion element, an AFE (Analog Front End), and the like.
The photoelectric conversion element includes, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element or the like. A subject image is incident on the photoelectric conversion element from the optical lens unit. Therefore, the photoelectric conversion element photoelectrically converts (photographs) a subject image, accumulates image signals for a certain period of time, and sequentially supplies the accumulated image signals to the AFE as analog signals.
The AFE executes various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal. A digital signal is generated by various signal processing, and is output as an output signal of the imaging unit 20.
Such an output signal of the imaging unit 20 is appropriately supplied as data of a captured image to the CPU 11, an image processing unit (not shown), and the like.

  The positioning section 21 measures the current position of the image display device 1. For example, the positioning unit 21 receives a radio wave transmitted from a satellite in a GPS (Global Positioning System) and converts it into an electric signal. Further, the positioning unit 21 measures the current position (for example, the current position specified by latitude, longitude, and altitude) of the image display device 1 indicated by GPS based on the converted electric signal.

  In the drive 22, a removable medium 31 composed of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted. The program read from the removable medium 31 by the drive 22 is installed in the storage unit 18 as needed. In addition, the removable medium 31 can store various data such as image data stored in the storage unit 18 in the same manner as the storage unit 18.

[Function block]
FIG. 3 is a functional block diagram showing a functional configuration for executing a peripheral image display process among the functional configurations of the image display device 1 of FIG.
Here, as described above, the peripheral image display processing refers to a series of processing for further displaying a peripheral image when a captured image is displayed.

When performing the peripheral image display processing, as shown in FIG. 3, in the CPU 11, the captured image acquisition unit 111, the operation determination unit 112, the peripheral image acquisition unit 113, the image processing unit 114, the display control unit 115, , Works.
When the peripheral image display process is performed, a captured image storage unit 181 and a peripheral image storage unit 182 are set in one area of the storage unit 18.
The data necessary for realizing the peripheral image display processing is appropriately transmitted and received between these functional blocks at an appropriate timing, even if not particularly mentioned below.

The captured image storage unit 181 stores data of a captured image acquired from the imaging device 2. The captured image is acquired by a captured image acquisition unit 111 described later, and stored in the captured image storage unit 181.
The peripheral image storage unit 182 stores the peripheral image data obtained from the image database 3. The peripheral image is acquired by a peripheral image acquiring unit 113 described later, and is stored in the peripheral image storage unit 182.

The photographed image acquisition unit 111 acquires a photographed image from the imaging device 2 by communication via the communication unit 19.
The captured image is transmitted to the image display device 1 and captured by the captured image acquisition unit 111, for example, when the image capturing device 2 captures an image. In this case, when the captured image acquisition unit 111 acquires a captured image, the captured image acquisition unit 111 acquires from the positioning unit 21 position information indicating the current position of the image display device 1 at that time.

  Alternatively, the captured image is transmitted to the image display device 1 together with, for example, information indicating the time at which the imaging device 2 performed the image capturing, and is acquired by the captured image acquiring unit 111. In this case, the photographed image acquiring unit 111 specifies the photographing time based on the information transmitted together with the photographed image. Then, the captured image obtaining unit 111 obtains, from the log information of the positioning unit 21, position information indicating the current position of the image display device 1 measured at the positioning time corresponding to the specified shooting time.

Then, the captured image acquiring unit 111 stores the acquired captured image in the captured image storage unit 181 by storing the acquired captured image in the captured image storage unit 181 in association with position information indicating the acquired current position. .
In the present embodiment, since the image display device 1 and the imaging device 2 are assumed to be carried by the same user and used at a short distance, the position information measured by the positioning unit 21 is substantially The point that becomes the position information indicating the position where the imaging device 2 has taken the image is as described above.

  The operation determination unit 112 determines the content of an instruction from the user based on the instruction operation performed on the image display device 1 by the user. The instruction operation from the user is received by various buttons for realizing the input unit 16 or a touch panel. The instruction content from the user determined by the operation determination unit 112 is output to each functional block related to the instruction content.

  The peripheral image acquisition unit 113 acquires a peripheral image. For example, when a captured image to be displayed is selected, the peripheral image acquisition unit 113 reads out the position information associated with the captured image. Further, the peripheral image acquisition unit 113 transmits request information including the position information to the image database 3. Then, the peripheral image acquiring unit 113 acquires a peripheral image corresponding to the transmitted position information from the image database 3 as a response to the request information. The captured image acquisition unit 111 stores the acquired peripheral image in the peripheral image storage unit 182 by storing the acquired peripheral image in the peripheral image storage unit 182.

The image processing unit 114 performs image processing for displaying a peripheral image. For example, the image processing unit 114 specifies the correspondence between the captured image and the peripheral image by performing image analysis on the captured image and the peripheral image. Specifically, the image processing unit 114 specifies a position corresponding to the captured image in the peripheral image (that is, a position corresponding to the same capturing range in the same capturing direction as the captured image in the peripheral image).
In this case, for example, the image processing unit 114 detects an edge or a feature point of the subject in each of the captured image and the peripheral image, and specifies the position corresponding to the captured image in the peripheral image by performing the matching processing. . Alternatively, if information indicating the shooting direction at the time of shooting is associated as a geotag or the like with the shot image and the peripheral image, the image processing unit 114 may generate the shot image in the surrounding image based on the information indicating the shooting direction. Identify the corresponding location.

Then, the image processing unit 114 outputs a position corresponding to the captured image in the specified peripheral image to the display control unit 115.
Note that the image processing unit 114 may further have a function of performing other image processing. For example, the image processing unit 114 performs a process of combining the captured image and the peripheral image based on the position corresponding to the captured image in the specified peripheral image so that the user does not feel uncomfortable when the user observes the image. It may have a function. Note that an image obtained by appropriately combining the captured image and the peripheral image in this manner is hereinafter referred to as a “synthesized image”.

  The display control unit 115 controls an image to be displayed on the output unit 17 according to an instruction from the user. For example, the display control unit 115 performs control for displaying the captured image selected by the user on the output unit 17. Further, when a predetermined operation is performed by the user, the display control unit 115 switches between the captured image and the peripheral image based on the position corresponding to the captured image in the peripheral image output from the image processing unit 114. Control for displaying on the output unit 17 is performed. Alternatively, the display control unit 115 performs control for displaying a composite image on the output unit 17 when a predetermined operation is performed by the user.

  The predetermined operation is, for example, an operation for instructing a process involving a change in the display range of a captured image. More specifically, for example, a reduction operation (for example, pinch-in), an enlargement operation (for example, pinch-out) of a captured image, and a movement operation (for example, swipe) of a captured image are performed.

The display control unit 115 also displays on the output unit 17 a user interface for displaying each image and receiving an operation instruction from a user.
A display example of a screen displayed on the output unit 17 under the control of the display control unit 115 will be described in the column of [Display Example] described below with reference to FIGS. 6, 7, and 8.

[motion]
FIGS. 4 and 5 are flowcharts illustrating the flow of the peripheral image display process performed by the image display apparatus 1 of FIG. 1 having the functional configuration of FIG. 3 in the image display system S. FIG. 7 also illustrates the processing of the image database 3 that is performed in response to the processing by the image display device 1.
The peripheral image display processing is started by a user's operation instruction to the input unit 16. For example, the processing is started by a start instruction of application software for realizing the peripheral image display processing.

  First, the display control unit 115 displays a list of captured images stored in the captured image storage unit 181 on the output unit 17 (step S11).

  The operation determination unit 112 determines whether any of the captured images displayed in the list has been selected by a user's operation instruction (step S12). If none of the photographed images displayed in the list is selected by the user's operation instruction (No in step S12), the process returns to the photographed image list display in step S11. On the other hand, if any of the captured images displayed in the list is selected by the user's operation instruction (Yes in step S12), the process proceeds to step S13.

  The peripheral image acquiring unit 113 reads out the position information indicating the photographing position, which is associated with the photographed image selected in step S12 (step S13).

  The peripheral image acquisition unit 113 generates request information including the position information read in step S13, and transmits the generated request information to the image database 3 (step S14).

  On the other hand, the image database 3 determines whether or not the transmitted request information has been received (Step S31). If the request information has not been received (No in step S31), the image database 3 returns to the request waiting state. On the other hand, if the request information has been received (Yes in step S31), the process proceeds to step S32.

  The image database 3 searches for a peripheral image from its own database based on the position information included in the request information (step S32). Specifically, a peripheral image in which a position corresponding to the position information included in the request information is set as a shooting position is searched.

  The image database 3 transmits the peripheral image specified by the search in step S32 to the image display device 1 as a response to the request information transmitted in step S14 (step S33).

  On the other hand, the peripheral image acquisition unit 113 determines whether a peripheral image has been received as a response to the request information (Step S15). If the peripheral image has not been received (No in step S15), the peripheral image is in a reception standby state. On the other hand, if a peripheral image has been received (Yes in step S15), the process proceeds to step S16 in FIG.

  Proceeding to FIG. 5, the display control unit 115 displays the captured image selected in step S12 on the output unit 17 (step S16). The user can browse the displayed captured image.

  The operation determining unit 112 determines whether a predetermined operation instruction from a user who has viewed the captured image displayed in step S16 is detected (step S17). Here, the predetermined operation is an operation for instructing a process involving a change in the display range of a captured image, such as an enlargement operation, a reduction operation, or a movement operation, as described above. If the predetermined operation instruction has not been detected (No in step S17), the display of the selected captured image is maintained. On the other hand, when the predetermined instruction operation is detected (Yes in step S17), the process proceeds to step S18.

  The operation determining unit 112 determines whether or not to display the outside of the photographing range of the photographed image when the display range of the photographed image is changed based on the predetermined operation instruction from the user detected in step S17. A determination is made (step S18). That is, it is determined whether or not the content of the predetermined operation instruction from the user is a display instruction outside the shooting range of the shot image. If the instruction is not a display instruction outside the shooting range of the shot image (No in step S18), the process proceeds to step S21. On the other hand, when the instruction is a display instruction outside the shooting range of the shot image (Yes in step S18), the process proceeds to step S19.

  The image processing unit 114 extracts an image area corresponding to the display instruction detected in step S17 from the peripheral image (step S19). For example, when the display instruction of the photographing range adjacent to the photographed image is given in step S17, the image area corresponding to the photographing range adjacent to the photographed image is extracted from the peripheral image.

  The display control unit 115 displays the image area extracted from the peripheral image in step S19 on the output unit 17 (step S20). Thus, an image corresponding to the display instruction detected in step S17 can be displayed.

  The display control unit 115 displays an image area corresponding to the display instruction detected in step S17 on the output unit 17 in the captured image (step S21). Thereby, similarly to step S19, an image corresponding to the display instruction detected in step S17 can be displayed.

  The operation determination unit 112 determines whether an end instruction of the present process from the user has been detected (step S22). If the end instruction has not been detected (No in step S22), the process returns to step S17 and is repeated. On the other hand, when an end instruction is detected (Yes in step S22), the present process ends.

  As described above, the peripheral image display processing is realized, and when a predetermined operation is received from the user at the time of displaying the captured image, the peripheral image can be further displayed. As a result, it is possible to improve the convenience when browsing the image associated with the position information.

[Display example]
In the peripheral image display processing, an example of screen transition of a display displayed on the output unit 17 under the control of the display control unit 115 will be described with reference to FIGS. 6, 7, and 8. FIG. Here, FIGS. 6, 7, and 8 are schematic diagrams each showing a display example in the peripheral image display processing.
As shown in FIGS. 6, 7, and 8, in this example, it is assumed that the output unit 17 of the image display device 1 is realized as a touch panel arranged on the front of a smartphone. Further, in this example, the display control unit 115 performs display by dividing the touch panel into two display areas, a display area AR1 and a display area AR2.

  The captured image selected by the user is displayed by the processing in step S16 described above. For example, as shown in FIG. 6A, a map including a position corresponding to the shooting position of the shot image is displayed in the display area AR1. On this map, an icon 51 indicating the shooting position is displayed. Further, the captured image is displayed in the display area AR2. In this photographed image, the person serving as the subject is displayed as the subject 52 together with the image serving as the background.

As another example, for example, as shown in FIG. 6B, a captured image is displayed in the display area AR1, and information (corresponding to the captured position of the captured image, as shown as information 53 in the display area AR2). Here, text indicating the name of the shooting position is displayed.
The display example shown in FIG. 6A and the display example shown in FIG. 6B may be switched and displayed according to a user's operation instruction.

  In addition, as shown in FIG. 7C, an operation button 54 (here, an operation button with a text “peripheral”) for receiving a predetermined operation instruction from the user is displayed. When the operation of pressing the operation button 54 is received, the display content of the display area AR1 is changed as shown in FIG. 7D.

  Specifically, as shown in FIG. 7D, the captured image is displayed in the display area AR1. Then, in the conceptual area AR3, peripheral images corresponding to the captured images acquired from the image database 3 are arranged in a positional relationship. Here, the conceptual area AR3 is not an area where display is actually performed, but a conceptual area illustrated for explaining the processing of the present embodiment.

  In this case, as shown in FIG. 7E, a user's operation instruction is received in the display area AR1. For example, a swipe that slides in the horizontal direction while the user's hand 60 is in contact with the touch panel is received as an operation instruction from the user. Since this is a predetermined operation, it is determined as Yes in step S17 described above. In addition, since the display instruction is out of the photographing range, the determination in step S18 is Yes.

  Then, as shown in FIG. 7F, the display is switched from the photographed image to the peripheral image arranged in the conceptual area AR3. For example, when a swipe for instructing a movement to the left with respect to the captured image is received, an image on the left side of the position corresponding to the captured image in the peripheral image (the east side when facing directly from the capturing position of the peripheral image) Are extracted and displayed in the display area AR1 (steps S19 and S20). After that, an image at another position of the peripheral image is displayed in response to a user's operation instruction.

  Further, for example, in the display of FIG. 7F, when a swipe for instructing a rightward movement is directly received with respect to the captured image, the display instruction is not a display outside the imaging range, and thus the determination is No in step S18. Then, the photographed image is displayed again (step S21).

In this way, the image display device 1 further displays the peripheral image when a predetermined operation is received from the user when the captured image is displayed. The user who has referred to the peripheral image displayed in this way can easily understand the situation around the shooting position visually.
That is, according to the image display device 1, it is possible to improve convenience when browsing an image associated with position information.

[Configuration example]
The image display device 1 configured as described above includes an output unit 17, an input unit 16, an operation determination unit 112, a peripheral image acquisition unit 113, and a display control unit 115.
The output unit 17 displays a captured image associated with the position information.
The input unit 16 detects reception of an operation.
The operation determining unit 112 determines whether or not an instruction to change the display mode of the captured image has been given based on the operation detected by the input unit 16 when the output unit 17 displays the captured image.
The peripheral image acquisition unit 113 acquires a peripheral image based on the position information associated with the captured image from outside the image display device 1.
When the operation determination unit 112 determines that the determination is affirmative (that is, when the operation determination unit 112 displays the captured image by the output unit 17, the display control unit 115 determines the captured image based on the operation detected by the input unit 16). If it is determined that the display mode has been instructed to change), control is performed so that the peripheral image acquired by the peripheral image acquisition unit 113 is displayed on the output unit 17.
Accordingly, the image display device 1 can further display the peripheral image when a predetermined operation is received from the user when the captured image is displayed. That is, according to the image display device 1, the convenience at the time of browsing the image associated with the position information can be improved.

The image display device 1 further includes a communication unit 19.
The communication unit 19 communicates with the image database 3 outside the device.
The peripheral image acquisition unit 113 transmits the position information associated with the captured image to the image database 3 via the communication unit 19, and acquires a peripheral image based on the position information from the image database 3 as a response.
This allows the user to browse the peripheral images stored in the image database 3.

The position information associated with the captured image is information indicating a position where the captured image was captured,
The peripheral image is a surrounding landscape image including the position where the captured image is captured.
This allows the user to view both the captured image and the surrounding landscape image including the position where the captured image was captured.

The display control unit 115 performs control such that the captured image and the surrounding image, which is a landscape image, are synthesized so as not to cause a sense of discomfort when observed, and displayed on the output unit 17.
This allows the user to view the synthesized image and the landscape image without any discomfort.

The operation determination unit 112 determines that the operation detected by the input unit 16 is affirmative when the operation includes a reduction operation of the displayed captured image or a movement operation of the displayed captured image.
Thus, when the captured image is reduced or moved in accordance with the operation, even if the captured image is out of the display range, the user can browse the peripheral image instead.

The image display device 1 further includes an imaging unit 20.
The imaging unit 20 generates a photographed image by photographing.
The captured image acquisition unit 111 performs control so as to associate position information with the captured image generated by the imaging unit 20.
As a result, it is possible to obtain a captured image associated with the position information.

The image display device 1 further includes an image processing unit 114.
The image processing unit 114 analyzes the captured image.
The peripheral image acquisition unit 113 acquires a peripheral image based on the analysis result of the image processing unit 114.
This makes it possible to acquire a peripheral image based on a captured image to which no positional information is associated.

[Modification]
It should be noted that the present invention is not limited to the above-described embodiment, but includes modifications and improvements as long as the object of the present invention can be achieved. For example, the above-described embodiment may be modified to provide a modification as exemplified below. Further, modifications as exemplified below can be combined.

[Modification 1]
In the above-described embodiment, when there is a display instruction outside the photographing range in the photographed image (Yes in step S18), an image area corresponding to the display instruction is extracted from the peripheral image (step S19). Then, the image area extracted from the peripheral image is displayed (Step S20). That is, the captured image and the peripheral image are switched and displayed. However, the present invention is not limited to this, and a composite image may be displayed when it is determined to be Yes in step S18.

  The flow of the peripheral image display processing in this case will be described with reference to FIG. FIG. 9 is a flowchart illustrating the flow of the peripheral image display process according to the present modification. Steps for performing the same processing as in the above-described embodiment are denoted by the same reference numerals in the drawings, and description thereof will not be repeated.

  When steps S11 to S15 (and also steps S31 to S33) described above with reference to FIG. 4 are completed, in the present modified example, the process proceeds to step S16 in FIG. Thereafter, if it is determined Yes in step S18, the process proceeds to step S41 in the present modified example.

  The image processing unit 114 generates a composite image by combining the captured image and the peripheral image based on the position corresponding to the captured image in the peripheral image so that the user does not feel uncomfortable when observing the image ( Step S41).

The display control unit 115 displays an image area corresponding to a display instruction from the user on the output unit 17 for the composite image generated in step S18.
As a result, it is possible to display a synthesized image that is synthesized so as not to cause a sense of discomfort, instead of switching between the captured image and the peripheral image and displaying the captured image. Therefore, for example, when a moving operation of the captured image such as a swipe is received from the user, the display can be continuously changed without causing the user to feel the switching between the captured image and the peripheral image. Further, for example, when a reduction operation of a captured image such as a pinch-in is received from a user, a composite image having an image corresponding to the captured image as a center and a surrounding image as a periphery can be displayed.

[Modification 2]
In the above-described embodiment, the position information based on the positioning by the positioning unit 21 is associated with the captured image. Then, a peripheral image corresponding to the captured image is specified based on the position information. The invention is not limited to this, and the peripheral image corresponding to the captured image may be specified by another method. For example, the image processing unit 114 and the image database 3 may perform image matching using a technique such as template matching between the captured image and a plurality of peripheral images stored in the database. Then, by image matching, the matched peripheral image may be specified as a peripheral image corresponding to the captured image.
This makes it possible to identify the peripheral image even when the captured image is not associated with the position information based on the positioning.

  Further, when position information is used as in the above-described embodiment, other information may be used. For example, when information indicating the photographing time at the time of photographing is associated with the photographed image as a geotag or the like, the position information is the corresponding peripheral image, and the peripheral image photographed in the time zone corresponding to this time information is also included. An image may be specified. Thereby, when there are peripheral images captured at different time zones (for example, daytime zone and nighttime zone) at the same shooting position in the image database 3, a peripheral image that is more consistent with the shot image is determined. Can be identified. Therefore, when the captured image and the peripheral image are switched and displayed, or when a composite image is generated, a natural display that does not cause a sense of discomfort can be realized.

[Modification 3]
The processing content may be modified according to the function of each device included in the image display system S. For example, when the imaging device 2 has a positioning function, the position information measured by the imaging device 2 may be associated with the captured image instead of the position information measured by the image display device 1. .
Further, if the image display system S further includes a device having a position positioning function, such as a GPS logger or a smart watch, for example, the position at which the device having the position positioning function has positioned the captured image is determined. Information may be associated. In this case, the association of the position information may be performed immediately in real time at the time of shooting, or may be performed by specifying the corresponding position information based on the shooting time and the positioning time thereafter. In the case where the image display device 1 is used alone, the process of the above-described embodiment may be performed on the image captured by the image display device 1 using the image display device 1.

[Modification 4]
In the above-described embodiment, when a captured image to be displayed is selected, steps S13 to S13 to S15 (and also steps S31 to S33) are performed. That is, the peripheral image is acquired prior to the display of the captured image. However, the present invention is not limited to this, and the peripheral image may be acquired at another timing. For example, the peripheral image may be acquired at the time when the captured image is acquired before the peripheral image display processing is performed. Alternatively, the determination may be Yes in step S18, and the peripheral image may be acquired when the peripheral image becomes necessary.

[Other Modifications]
Further, in the above-described embodiment, the image display device 1 to which the present invention is applied has been described using a smartphone as an example, but is not particularly limited thereto.
The present invention can be applied to general electronic devices having an image display function. For example, the present invention is applicable to personal computers, television receivers, video cameras, portable navigation devices, mobile phones, portable game machines, and the like.

The series of processes described above can be executed by hardware or can be executed by software.
In other words, the functional configuration in FIG. 3 is merely an example and is not particularly limited. That is, it is sufficient that the image display device 1 has a function capable of executing the above-described series of processes as a whole, and what kind of functional block is used to realize this function is not particularly limited to the example of FIG. .
In addition, one functional block may be configured by hardware alone, may be configured by software alone, or may be configured by a combination thereof.
The functional configuration according to the present embodiment is realized by a processor that executes arithmetic processing, and the processor that can be used in the present embodiment includes single processing units, multiprocessors, and various processing units such as multicore processors. In addition to the above, a combination of these various processing apparatuses and processing circuits such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field-Programmable Gate Array) is included.

When a series of processing is executed by software, a program constituting the software is installed in a computer or the like from a network or a recording medium.
The computer may be a computer embedded in dedicated hardware. Further, the computer may be a computer that can execute various functions by installing various programs, for example, a general-purpose personal computer.

  The recording medium including such a program is constituted not only by the removable medium 31 of FIG. 2 distributed separately from the apparatus main body to provide the program to the user, but also It is composed of a recording medium and the like provided to the user. The removable medium 31 is configured by, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk includes, for example, a CD-ROM (Compact Disk-Only Memory), a DVD (Digital Versatile Disk), a Blu-ray (registered trademark) Disc (Blu-ray Disc), and the like. The magneto-optical disk is composed of an MD (Mini-Disk) or the like. The recording medium provided to the user in a state where the recording medium is incorporated in the apparatus main body in advance includes, for example, the ROM 12 in FIG. 2 in which the program is recorded, the hard disk included in the storage unit 18 in FIG.

  In this specification, the steps of describing a program recorded on a recording medium are not only performed in chronological order according to the order, but are not necessarily performed in chronological order. This also includes the processing to be executed.

  Although some embodiments of the present invention have been described above, these embodiments are merely examples and do not limit the technical scope of the present invention. The present invention can take other various embodiments, and various changes such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and their modifications are included in the scope and gist of the invention described in this specification and the like, and are also included in the invention described in the claims and equivalents thereof.

Hereinafter, the inventions described in the claims at the time of filing the application of the present application are additionally described.
[Appendix 1]
An image display device,
Display means for displaying a first image associated with position information;
Detecting means for detecting reception of an operation;
A determination unit configured to determine whether or not an instruction to change a display mode of the first image has been given based on an operation detected by the detection unit when the first image is displayed by the display unit;
Acquiring means for acquiring a second image based on position information associated with the first image from outside the image display device;
A display control unit configured to control the display unit to display the second image acquired by the acquisition unit when the determination unit determines affirmatively;
An image display device comprising:
[Appendix 2]
Further comprising communication means for communicating with a database outside the device,
The acquisition unit transmits position information associated with the first image to the database via the communication unit, and acquires a second image based on the position information from the database in response to the position information. The image display device according to supplementary note 1.
[Appendix 3]
The position information associated with the first image is information indicating a position where the first image was captured,
3. The image display device according to claim 1, wherein the second image is an image around a position where the first image is captured.
[Appendix 4]
4. The image display device according to claim 3, wherein the display control unit controls the first image and the second image to be combined and displayed on the display unit.
[Appendix 5]
The determination unit determines that the operation is affirmative when the operation detected by the detection unit includes a reduction operation of the displayed first image or a movement operation of the displayed first image. The image display device according to any one of supplementary notes 1 to 4.
[Appendix 6]
Imaging means for generating the first image by photographing;
Position information control means for controlling to associate position information with the first image generated by the imaging means;
The image display device according to any one of supplementary notes 1 to 5, further comprising:
[Appendix 7]
The image processing apparatus further includes image analysis means for analyzing the first image,
7. The image display device according to claim 1, wherein the acquisition unit acquires the second image based on an analysis result of the image analysis unit.
[Appendix 8]
An image display method performed by a computer,
A display step of displaying a first image associated with the position information on the display means;
A detection step of detecting acceptance of the operation;
A determining step of determining whether or not an instruction to change a display mode of the first image has been given based on the operation detected by the detecting step when the first image is displayed by the displaying step;
An obtaining step of obtaining a second image based on position information associated with the first image from outside the computer;
A display control step of controlling to display the second image acquired by the acquiring step on the display means when it is determined to be affirmative by the decision step;
An image display method comprising:
[Appendix 9]
Computer
Display means for displaying a first image associated with the position information on the display unit;
Detecting means for detecting reception of an operation,
A determining unit configured to determine whether or not an instruction to change a display mode of the first image has been given based on an operation detected by the detecting unit when the first image is displayed by the display unit;
Acquiring means for acquiring a second image based on position information associated with the first image from outside the computer;
A display control unit configured to control the display unit to display the second image acquired by the acquisition unit when the determination unit determines that the determination is affirmative;
A program characterized by functioning as a program.

  DESCRIPTION OF SYMBOLS 1 ... Image display apparatus, 2 ... Imaging apparatus, 3 ... Image database, 4 ... Network, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Bus, 15 input / output interface, 16 input unit, 17 output unit, 18 storage unit, 19 communication unit, 20 imaging unit, 21 positioning Unit, 22 drive, 31 removable media, 111 captured image acquisition unit, 112 operation determination unit, 113 peripheral image acquisition unit, 114 image processing unit, 115 ··· Display control unit 181 · Captured image storage unit 182 · Peripheral image storage unit · S · Image display system

Claims (9)

An image display device,
Display means for displaying a first image associated with position information;
Detecting means for detecting reception of an operation;
A determination unit configured to determine whether or not an instruction to change a display mode of the first image has been given based on an operation detected by the detection unit when the first image is displayed by the display unit;
Acquiring means for acquiring a second image based on position information associated with the first image from outside the image display device;
A display control unit configured to control the display unit to display the second image acquired by the acquisition unit when the determination unit determines affirmatively;
An image display device comprising: Further comprising communication means for communicating with a database outside the device,
The acquisition unit transmits position information associated with the first image to the database via the communication unit, and acquires a second image based on the position information from the database in response to the position information. The image display device according to claim 1. The position information associated with the first image is information indicating a position where the first image was captured,
The image display device according to claim 1, wherein the second image is an image around a position where the first image is captured.   The image display device according to claim 3, wherein the display control unit controls the first image and the second image to be combined and displayed on the display unit.   The determination unit determines that the operation is affirmative when the operation detected by the detection unit includes a reduction operation of the displayed first image or a movement operation of the displayed first image. The image display device according to claim 1. Imaging means for generating the first image by photographing;
Position information control means for controlling to associate position information with the first image generated by the imaging means;
The image display device according to any one of claims 1 to 5, further comprising: The image processing apparatus further includes image analysis means for analyzing the first image,
The image display device according to claim 1, wherein the acquisition unit acquires the second image based on an analysis result of the image analysis unit. An image display method performed by a computer,
A display step of displaying a first image associated with the position information on the display means;
A detection step of detecting acceptance of the operation;
A determining step of determining whether or not an instruction to change a display mode of the first image has been given based on the operation detected by the detecting step when the first image is displayed by the displaying step;
An obtaining step of obtaining a second image based on position information associated with the first image from outside the computer;
A display control step of controlling to display the second image acquired by the acquiring step on the display means when it is determined to be affirmative by the decision step;
An image display method comprising: Computer
Display means for displaying a first image associated with the position information on the display unit;
Detecting means for detecting reception of an operation,
A determination unit configured to determine whether or not an instruction to change a display mode of the first image has been given based on an operation detected by the detection unit when the first image is displayed by the display unit;
Acquiring means for acquiring a second image based on position information associated with the first image from outside the computer;
A display control unit configured to control the display unit to display the second image acquired by the acquisition unit when the determination unit determines that the determination is affirmative;
A program characterized by functioning as a program.

Images