JP2019211810A - Image display system - Google Patents

Abstract

To view images of the same object at different points in time with a simple operation.SOLUTION: An image display system includes a plurality of portable terminals 1 used for photographing, a server 2 for storing a lot of image data (image files) obtained by the photographing, and an image display apparatus 3 for displaying the images. These are connected with each other via a network N. After photographing, the portable terminal 1 can transmit the obtained image file to the server 2 via the network. At this time, time information, positional information and pose information when generating the image file are transmitted. The server 2 stores the above mentioned image files, the time information, the positional information and the pose information as grouped photographing data D.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image display system that displays a desired image using a large number of image data.

  An image based on this image data (for example, an image of a landscape) is obtained by obtaining image data captured by a device at a location away from the user via a server in a personal computer or a mobile terminal (smartphone or the like) owned by the user. ) Can be seen. This image data includes data captured in the past in addition to the current data. Here, even when the same part is imaged, the image is not necessarily captured under the same imaging condition. If the imaging condition is different, the obtained image is different even though the substantially same landscape is captured. .

  Patent Document 1 compares a landscape image at a certain point in time with a landscape image at a different point in time, and identifies and shows a portion that is recognized as having changed in this landscape. For example, at a different point in time An image display system that performs processing such as specifying an image at this location and simultaneously displaying the image is described. In this technique, image data obtained by a large number of mobile terminals is stored in a server, and various kinds of information are shared between the mobile terminals and stored together with the image data. As a result, as described above, images at different time points can be appropriately compared, and the display as described above can be performed.

JP 2006-133701 A

  The image display system described above is effective for accurately recognizing temporal changes of structures in the landscape, but in practice, such a strictness is not required, and the user can In many cases, it is only required to view images at different times. In such a case, the operation required in the image display system described above is not easy for the user and is not convenient.

  For this reason, an image display device and an image display system that can view images at different times of the same object by a simple operation are desired.

  This invention is made | formed in view of such a condition, and it aims at providing the technique which can solve the said subject.

  The image display system of the present invention is an image display system that displays an image selected by a user among images captured by an imaging terminal at different times on a display unit, and is generated when the imaging terminal captures an image. Captured image file, time information indicating the date and time when the imaging was performed, position information indicating the position of the imaging terminal when the imaging was performed, and the imaging terminal when the imaging was performed A storage unit that stores a plurality of pieces of posture information representing postures in a three-dimensional space as imaging data that is a group of data, and a range in each of the time information, the position information, and the posture information is set by a user. , The time information, the position information, and the posture information are images based on the imaging data within the set ranges, respectively. A control unit for displaying on the display unit together with the date and time based on the serial time information, characterized by comprising a.

  With the above configuration, it is possible to view images of the same object at different points in time with a simple operation.

1 is a diagram illustrating a configuration of an entire image display system according to an embodiment of the present invention. It is a figure which shows the structure of the imaging terminal used in the image display system which concerns on embodiment of this invention. It is a figure which shows the structure of the server used in the image display system which concerns on embodiment of this invention. It is a figure which shows the structure of the image display apparatus used in the image display system which concerns on embodiment of this invention. It is a flowchart which shows operation | movement in the image display apparatus used in the image display system which concerns on embodiment of this invention. It is an example of the image display in the image display system which concerns on embodiment of this invention. It is another example of the image display in the image display system which concerns on embodiment of this invention.

  Hereinafter, an image display system according to an embodiment of the present invention will be described. FIG. 1 is a diagram showing a configuration of the entire image display system. This image display system includes a plurality of portable terminals (imaging terminals) 1 used for imaging, a server 2 that stores a large number of image data (image files) obtained by the imaging, and an image display device that displays images. 3 is provided. These are connected to each other via a network N. Here, the image display device 3 is used only for displaying an image, and for example, a normal personal computer or the like can be used. Further, one of the plurality of portable terminals 1 or the server 2 can also serve as the image display device 3.

  FIG. 2 is a diagram illustrating a configuration of a portable terminal (imaging terminal) 1 for obtaining an image file or the like stored in the server 2. The portable terminal 1 includes a control unit 10 that is used for imaging and performs overall control, and an imaging unit 11 that captures surrounding images and obtains the image data as an image file. In addition, a network connection unit 12 is provided for connecting to a network to exchange various data including the image file. In addition, an operation panel 13 and a display unit 14 that are used for operations by the user of the portable terminal 1 are provided, and a storage unit 15 that stores the above-described image file and various data is provided. The operation panel 13 and the display unit 14 are actually provided as a touch panel display in which these are integrated.

  Moreover, the time recognition part 16 for recognizing the present time is provided, and, thereby, the control part 10 recognizes a date and time as current time information. Further, a position information recognition unit 17 for recognizing the current position by receiving a GPS signal is also provided. This position information is represented by latitude, longitude, and altitude, for example. A posture recognition unit 18 for recognizing the current posture of the subject is also provided. The posture recognition unit 18 is configured by combining an orientation sensor and a posture sensor described in, for example, JP-A-2015-45583, and detects geomagnetism and acceleration in a three-dimensional space at the center of the visual field in the imaging unit 11. Detect orientation. If the imaging unit 11 is fixed in the mobile terminal 1, the orientation of the center of the visual field in the three-dimensional space is uniquely determined if the orientation of the central axis of the mobile terminal 1 in the three-dimensional space is determined. Alternatively, the center axis of the mobile terminal 1 or the orientation of a predetermined axis in a mobile terminal 1 may be detected instead. Here, the direction detected in this way is referred to as the attitude of the mobile terminal 1. Here, not the temporal change of the posture, but the posture at the time when the imaging unit 11 performs the imaging is particularly detected. This posture information is represented by, for example, an azimuth angle and an elevation angle.

  When the imaging unit 11 captures an image, the portable terminal 1 (the control unit 10) can transmit an image file obtained by the operation to the server 2 via the network N by the operation of the operation panel 13 by the user. At this time, together with this image file, time information, position information, and posture information when this image file is created (imaged) are transmitted.

  FIG. 3 is a diagram illustrating a configuration of the server 2. The server 2 is connected to the mobile terminal 1 via the network connection unit 21 and is provided with a storage unit 22 that stores data transmitted from the mobile terminal 1 as described above and a control unit 20 that controls the whole. Here, there are a plurality of mobile terminals 1 that transmit data to the server, and data is transmitted from each mobile terminal 1 at a plurality of points in time, so that the storage unit 22 has a large capacity to store all of these data. It is composed of a hard disk and semiconductor memory. In addition, an operation panel 23 and a display unit 24 for the user to operate the server 2 are provided.

  In the server 2, the image file, the time information, the position information, and the posture information are stored in the storage unit 22 as a group of imaging data D. For this reason, for example, when a specific range is specified in the time information, the position information, and the posture information, the control unit 20 recognizes the presence or absence of the imaging data D in which each of these pieces of information is all within this range, or This imaging data D can be extracted.

  FIG. 4 is a diagram illustrating a configuration of the image display device 3. Also in this image display device 3, a control unit 30, a network connection unit 31, an operation panel 32, a display unit 33, and a storage unit 34 are provided, and a control unit 30 that controls these components is provided. As in the case of the portable terminal 1, a touch panel display in which the operation panel 32 and the display unit 33 are integrated may be used. As described above, a normal personal computer can be used as the image display device 3, and the portable terminal 1 and the server 2 may also serve as the image display device 3. In this image display system, the target image is displayed on the display unit 33.

  Among the above data stored in the storage unit 22, in particular, the position information and the posture information reflect the positional relationship between the captured image and the imaging unit 11 (the portable terminal 1). For this reason, if there are two pieces of image data D having the same position information and posture information, the images of the image file in these are obtained by imaging the same object in the same manner. For this reason, for example, if certain imaging data D (display imaging data) is selected, the image is displayed on the display unit 14, and there is imaging data D (similar imaging data) whose position information and orientation information are close to this display imaging data. The image of the similar imaging data can be estimated as an image of the same target at a time different from that currently displayed on the display unit 14. For this reason, such similar imaging data is extracted from all the imaging data stored in the storage unit 22, and when there are a plurality of similar imaging data, a desired date and time is selected, and the display unit 14 is selected. And can be compared with the original image by the display imaging data.

  FIG. 5 is a flowchart showing the operation at this time. Here, first, the control unit 30 in the image display device 3 obtains a list of imaging data D stored in the storage unit 22 from the server 2 side (S1). Here, the image display device 3 obtains only time information, position information, and posture information with a small data capacity in all the imaging data D, and does not need to obtain an image file with a large data capacity. This list is displayed on the display unit 33 (S2), and the user is made to select one of them as an image (display imaging data) to be displayed using the operation panel 32 (S3). Selection in this case is performed by an arbitrary method such as designation of a file name, date and time (time information), and position (position information). Thereafter, the control unit 30 obtains the image file of the display imaging data from the server 2 (storage unit 22) via the network N (S4), and displays the image based on the image file together with the date and time based on the time information. 33 is displayed (S5).

  Next, the control unit 30 refers to the previously obtained list (S1), and sets the imaging data whose date and time is earlier than the display imaging data and whose position information and orientation information are close to the display imaging data as similar imaging data. Extract. Here, when the position information is close, when the distance between two points based on the position information is smaller than a predetermined threshold (first threshold), the position information is close when two directions based on the position information If the angle difference is smaller than a predetermined threshold (second threshold), it can be determined. If similar imaging data is not found (S6: No), the process ends.

  When there is similar imaging data (S6: Yes), the control unit 30 corresponds to the length of the line segment in addition to the image display and the date and time display (S5) on the display unit 33. The time-lapse display expressed as described above is displayed together with the display of the image of the display imaging data (S7). FIG. 6A shows an example of display on the display unit 33 in this case. Here, in the upper target image display unit 33A, an image based on the display imaging data is displayed, and on the lower side, the date and time of the display imaging data is displayed on the date and time display unit 33B. Further below that, a time passage display 33C showing a time passage so that the time interval corresponds to the length in the horizontal direction is displayed. In the period display 33D, the period shown in the displayed time passage display 33C is specifically displayed. The rightmost side in the time lapse display 33C corresponds to the date and time of the display imaging data displayed on the date and time display unit 33B. Among these, the marker M indicating the date and time of the image currently displayed on the target image display unit 33A is It is displayed so that it exists on the rightmost side. The left side corresponds to the past going back from this point. Among these, marking (imaging data existence mark L) is displayed at a location corresponding to the date and time of imaging data existing as similar imaging data.

  The user can operate the operation panel 32 (or touch panel display) to move the marker M to the left side (past side) from the position in FIG. When the resetting of the position of the marker M is performed (S8: Yes), the control unit 30 replaces the similar imaging data having time information according to the new position of the marker M with the display imaging data. In the same manner as in the above (S5) (S9). FIG. 6B is an example of display on the display unit 33 in this state. Thereafter, this operation (S8, S9) is repeated every time the marker M is reset.

  In the above example, the positions of the markers M are two types, that is, the case of FIG. 6A and the case of FIG. However, an operation considering the moving speed of the marker M is also possible. For example, when the movement speed of the marker M is slow and there is a lot of similar imaging data within the time range indicated by the time lapse display 33C, until the marker M after the movement finally stops, The time-lapse display unit 33C can sequentially read similar imaging data corresponding to the position of the moving marker M without changing, and sequentially switch and display these images on the target image display unit 33A. In this case, the user can more appropriately recognize the temporal change of the imaging target (landscape). On the other hand, when the movement speed of the marker M is fast, only two types of images corresponding to the start point (FIG. 6A) and the end point (FIG. 6B) of the marker M are displayed on the target image display unit 33A. be able to. Such an operation can be appropriately performed according to the determination timing of the position of the marker M and the time interval.

  In the example of FIG. 6, only one target image display unit 33 </ b> A that displays a target image is provided. However, a plurality of target image display units 33 </ b> A can be provided simultaneously for each target object (target landscape). FIGS. 7A and 7B show a case where four target image display units 33A1 to 33A4 are used. FIG. 7A shows a display immediately after displaying the display imaging data (S7). FIG. 7B is an example of display (S9) after the marker M is reset. Here, the date and time of the imaging data (display imaging data) in each of the target image display units 33A1 to 33A4 is common (simultaneous), and the position information or the posture information is greatly different from each other, and therefore different objects (landscapes). Are recognized as being simultaneously captured.

  On the other hand, the time-lapse display unit 33C and the marker M are shared by the target image display units 33A1 to 33A4. However, similar imaging data is obtained for each display imaging data shown in each of the target image display units 33A1 to 33A4, and the date and time of the obtained similar imaging data generally differs for each display imaging data. For this reason, here, in the time lapse display section 33C, the imaging data presence mark L is displayed as imaging data existence marks L1 to L4 that are different for each display imaging data (target image display section). The By such display, the user can recognize for each image displayed on the target image display units 33A1 to 33A4 which point in time the corresponding image in the past can be viewed.

  As described above, it is possible to perform the simple operation to view the images of the same object at different points in time using the image display system.

  In the above example, the target image display unit and the time lapse display unit are displayed as shown in FIGS. 6 and 7, but as long as the user can recognize them and use them in the same way, These configurations are arbitrary.

  Further, in the above example, the display as shown in FIGS. 6 and 7 is performed by the image display device 3 in FIG. 1. However, as described above, the same display and operation are performed by the portable terminal 1 and the server. 2 can also be performed. Furthermore, any device that is connected to a server that stores imaging data via a network and includes a similar display unit can be used as an image display device.

  In the above example, a single server 2 (storage unit 22) is used. However, a plurality of servers and storage units for storing imaging data may be provided distributed over a network.

  In the above example, the device that acquires the imaging data is the portable terminal. However, an imaging device that is fixed on the ground and connected to the network can also be used as the imaging terminal. In this case, since the position is fixed, the position information recognition unit is not necessary. Furthermore, if the posture at the time of imaging of the imaging terminal is fixed, the posture recognition unit described above is also unnecessary. In these cases, the control unit of the server can automatically recognize the position information and the posture information by identifying the imaging terminal.

1 Mobile terminal (imaging terminal)
2 Server 3 Image display device 10, 20, 30 Control unit 11 Imaging unit 12, 21, 31 Network connection unit 13, 23, 32 Operation panel 14, 24, 33 Display unit 15, 22, 34 Storage unit 16 Time recognition unit 17 Position information recognition unit 18 Posture recognition unit 33A, 33A1, 33A2, 33A3, 33A4 Target image display unit 33B Date and time display unit 33C Time lapse display 33D Period display D Imaging data L, L1 to L4 Imaging data presence mark M Marker N network

Claims (6)

An image display system that displays an image selected by a user among images captured by an imaging terminal at different times on a display unit,
An image file generated by the imaging terminal imaging, time information indicating the date and time when the imaging was performed, position information indicating the position of the imaging terminal when the imaging was performed, and the imaging A storage unit that stores a plurality of pieces of posture information representing the posture of the imaging terminal in the three-dimensional space at the time when the imaging is performed as imaging data that is a group of data;
A range in each of the time information, the position information, and the posture information is set by a user, and an image based on the imaging data in the range in which the time information, the position information, and the posture information are set, respectively. A control unit for displaying on the display unit together with the date and time based on the time information of the imaging data;
An image display system comprising: The controller is
When displaying an image based on display imaging data that is one of the imaging data on the display unit,
The imaging data other than the display imaging data, and the imaging data in which the difference between the position information and the posture information in the display imaging data is different from the position information and the posture information. The image display system according to claim 1, wherein similar image data is extracted, and the date and time based on the time information in the similar image data is displayed on the display unit. In the display unit,
The image display system according to claim 2, wherein the date and time is displayed by a marker in a time passage display in which a time interval is displayed corresponding to a length. In the display unit,
3. The display according to claim 2, wherein the marker is set to be selectable by a user in the time lapse display, and an image based on the imaging data corresponding to the date and time corresponding to the selected marker is displayed. 4. The image display system according to 3. The control unit simultaneously displays images corresponding to a plurality of different display imaging data on the display unit, and extracts the similar imaging data corresponding to each of the plurality of display imaging data,
5. The image forming system according to claim 3, wherein the marker is displayed in a different manner for each corresponding display imaging data in the time-lapse display. A server comprising the storage unit and separated from the imaging terminal;
An image display device connected to the server via a network and including the control unit and the display unit;
The image display system according to any one of claims 1 to 5, further comprising: