JP2016218626A - Image management apparatus, image management method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image management apparatus configured to properly manage a captured image.SOLUTION: An image management apparatus includes: an image storage unit 101 which stores a plurality of image groups comprising a series of images obtained by capturing an object in different time zones, and position-related information which is information on an imaging position of one or more images and is associated with one or more images constituting the image groups; an image designation receiving unit 107 which receives designated one or more images of the image groups stored in the image storage unit 101; a corresponding image acquisition unit 108 which acquires an image associated with the designated one or more images received by the image designation receiving unit 107, by the position-related information, from an image group different from the image group including the designated image, of the image groups stored in the image storage unit 101; and an output unit 109 which outputs the image acquired by the corresponding image acquisition unit 108.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an apparatus for managing captured images.

  As a conventional technique, it is an inspection auxiliary device that assists in the inspection of a building, and a panoramic image generation unit that generates a panoramic image of a building by pasting together a plurality of inspection images in which the building is partially photographed, A panoramic image display unit that displays the entire panoramic image, a main image display unit that displays a partial area in the panoramic image, and a control unit that controls the position of the partial region displayed in the main image display unit in accordance with an input from the user And a display area drawing unit that visually draws a region corresponding to the partial area in the whole view image displayed on the whole view image display unit in conjunction with the partial area currently displayed on the main image display unit, (For example, refer to Patent Document 1).

JP 2014-11720 A (first page, FIG. 1 etc.)

  However, the conventional image management apparatus has a problem that it is not possible to manage properly captured images.

  For example, in a conventional image management apparatus, it is not possible to easily acquire and output an image obtained by photographing a region that is substantially the same as the region photographed in this image in the past for one photographed image. There was a problem. To give a specific example, since there are many images that are very similar in an image of a part of a building, such as a part of a wall, the same area as that of one image is taken from images taken in the past. It is very difficult to find an image that is considered to be taken, and as a result, it has been very difficult to acquire and output an image obtained by photographing almost the same region in the past.

  The present application has been made to solve the above-described problems, and an object thereof is to provide an image management apparatus and the like that can appropriately manage captured images.

  The image management apparatus according to the present invention includes a plurality of image groups configured by a series of a plurality of images captured at different time zones, and information associated with one or more images configuring each image group. An image storage unit that stores position-related information that is information related to a position at the time of shooting for one or more images, and an image designation that accepts designation of one or more images in the image group stored in the image storage unit An image group including the designated image among the image groups stored in the image storage unit and the image associated with the position-related information to one or more images received by the reception unit and the image designation reception unit The image management device includes a corresponding image acquisition unit that acquires images from different image groups and an output unit that outputs an image acquired by the corresponding image acquisition unit.

  With such a configuration, it is possible to appropriately manage images taken in different time zones. For example, it is possible to output images corresponding to shooting positions shot at different time zones.

  The image management apparatus according to the present invention is an image management apparatus according to the image management apparatus, wherein the position-related information includes position information of a camera that has captured an image and information indicating a shooting direction.

  With this configuration, it is possible to easily output an image corresponding to a shooting position and a shooting direction from images shot at different time zones.

  In the image management apparatus of the present invention, the image management apparatus uses an image acquisition unit that acquires a series of a plurality of images and two or more images of the series of images acquired by the image acquisition unit. A position-related information acquisition unit that acquires position-related information for one or more images, an image group including a series of images including an image from which the position-related information acquisition unit has acquired the position-related information, and the image group The image management apparatus further includes an accumulation unit that accumulates position-related information acquired for each of one or more images constituting the image storage unit in an image storage unit.

  With this configuration, the position related information can be acquired from the captured image, and the position related information can be acquired without using position detection means such as GPS.

  The image management apparatus according to the present invention is an image management apparatus in which the position-related information is information relating to a relative position with respect to a position designated in advance.

  With this configuration, the position at the time of shooting can be indicated by a relative position.

  In the image management apparatus of the present invention, in the image management apparatus, the image acquisition unit includes a plurality of series of images having the same shooting start position as a plurality of series of images acquired for one target. An image management apparatus that acquires a plurality of images.

  With this configuration, it is possible to easily determine the correspondence between positions and directions for images taken in different time zones.

  In the image management apparatus of the present invention, in the image management apparatus, the image acquisition unit acquires a frame image one by one every one or more frame images from a plurality of continuous frame images constituting the moving image. An image management apparatus for acquiring a series of a plurality of images.

  With this configuration, a series of a plurality of images can be acquired from a moving image.

  In the image management device of the present invention, in the image management device, the image acquisition unit acquires a frame image one by one for every number of frame images corresponding to the moving speed of the camera when shooting a moving image. It is a management device.

  With this configuration, it is possible to appropriately acquire a series of images from a moving image according to the moving speed.

  The image management apparatus according to the present invention further includes a link setting unit that links images included in different image groups using position-related information in the image management apparatus, and the corresponding image acquisition unit receives the image designation reception. This is an image management apparatus that acquires images linked to one or more images that the unit has received designation.

  With such a configuration, it is possible to easily output corresponding images between image groups photographed in different time zones.

  The image management apparatus according to the present invention is an image management apparatus in which one or more images constituting an image group are stored in an image storage unit in association with sensor information.

  With this configuration, sensor information can be managed appropriately.

  In the image management apparatus according to the present invention, in the image management apparatus, one or more images constituting an image group are stored in an image storage unit in association with a feature amount acquired from each image. It is.

  With this configuration, it is possible to appropriately manage the image feature amount.

  According to the image management apparatus and the like according to the present invention, captured images can be appropriately managed.

Block diagram of an image management system in an embodiment of the present invention A flowchart for explaining the operation of the image management apparatus of the image management system The perspective view seen from the lower side of the moving body in which the camera of the image management system is installed Schematic diagram for explaining the operation of the image management apparatus of the image management system The figure which shows the image management table | surface of the image management apparatus of the image management system The figure which shows the link information management table | surface of the image management apparatus of the image management system The figure which shows the image management table | surface of the image management apparatus of the image management system Schematic diagram explaining position related information of the image management apparatus of the image management system The figure which shows the feature-value management table | surface of the image management apparatus of the image management system The figure which shows the sensor information management table | surface of the image management apparatus of the image management system The figure which shows the example of a display of the image management apparatus of the image management system The figure which shows the example of a display of the image management apparatus of the image management system The figure which shows an example of the external appearance of the computer system in embodiment of this invention The figure which shows an example of a structure of the computer system

  Hereinafter, embodiments of an image management apparatus and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

(Embodiment)
FIG. 1 is a block diagram of an image management system 1000 in the present embodiment.

  The image management system 1000 includes an image management apparatus 1 and a camera 2. The image management apparatus 1 and the camera 2 can be connected by, for example, wired communication or wireless communication so that information can be transmitted and received. It should be noted that information of an image taken by the camera 2 is accumulated in a removable storage medium (not shown) (for example, a memory card), and the image management apparatus 1 reads out the information stored in the storage medium. Also good.

  The image management apparatus 1 includes an image storage unit 101, an image acquisition unit 102, a position related information acquisition unit 103, a feature amount acquisition unit 104, a storage unit 105, a link setting unit 106, an image designation reception unit 107, a corresponding image acquisition unit 108, An output unit 109 is provided.

  The image storage unit 101 includes information associated with a plurality of image groups configured by a series of a plurality of images captured at different time zones, and one or more images configuring each image group. In addition, position related information, which is information regarding the position at the time of shooting, of each of the one or more images is stored. The series of images is usually a series of still images. This still image may be a frame image of a moving image. The object to be photographed is, for example, one or more structures, the inside of the structure (for example, in a room, a warehouse, or the like). The structure is, for example, a building or a structure. It is a building, a house or a building. The structure is, for example, a bridge, a tower, a multi-story parking lot, or the like. However, the photographing target may be an object that is not related to the structure. Further, the shooting target may be terrain or the like, or may be an arbitrary space or region (for example, a region). The imaging target may be considered as a management target, an observation target, a monitoring target, or the like.

  The one or more images associated with the position-related information are one or more images in a series of images constituting each image group, and preferably two or more. More preferably, all images constituting each image group are preferable. The position related information is, for example, information including information on the shooting position where the image is shot and information on the position of the shooting target. The shooting position is a position where an image is shot, for example, the position of the camera 2 that shot the image. The position of the imaging target is, for example, the position of the part (surface of the imaging target) that is centered in the captured image of the imaging target. The information on the shooting position and the information on the position of the shooting target are, for example, coordinates. The coordinates may be absolute coordinates represented by a combination of latitude and longitude, or may be relative coordinates based on a predesignated position or coordinate system. The predesignated position is, for example, a position where a series of images has been started. The position related information may further include information indicating a shooting direction when shooting each image. The shooting direction is, for example, the direction of the camera 2 that performs shooting, and specifically, the optical axis direction of the camera 2. Note that the rotation direction with the optical axis of the camera 2 as the rotation axis may also be considered as one element of the shooting direction of the camera 2. Information indicating the shooting direction is represented by a direction vector, for example. The shooting direction may be a direction specified in advance or a relative direction based on a coordinate system, or may be an absolute direction represented by an azimuth or the like. This relative direction is, for example, a shooting direction when shooting is started.

  In addition, in order to observe a subject to be photographed or to acquire position related information from an image as will be described later, when the position related information is arranged in order of photographing among two or more images associated with each other It is preferable that the two images adjacent to each other are images that are at least partially overlapped, that is, at least a portion or region that is common to at least a part is captured, and more preferably, more than half of the images overlap. It is preferable that a portion or region common to more than half of the image is captured. For example, when all of a series of multiple images are associated with position-related information, at least a part of the two images in which the shooting order is continuous overlap, preferably an image that overlaps more than half. It is preferable that

  In addition, when using relative position and direction information as shooting position and shooting direction information, when shooting one shooting target at different time zones, the shooting range of the camera 2 at the start of shooting matches. It is preferable to do so. For example, when relative position information is used as a shooting position, a plurality of image groups composed of a series of images shot at different time zones for a single shooting target are obtained by starting shooting from the same position. The image group is preferable. The same position here is preferably the same position in order to increase the accuracy of the position indicated by the position-related information. However, if high accuracy is not required, It may be considered as a position within the specified range. Also, when using relative direction information as shooting direction information, a plurality of image groups composed of a series of images shot at different time zones for one shooting target start shooting from the same shooting direction. The image group obtained in this way is preferable. The shooting direction in this case is also preferably the same direction in order to increase the accuracy of the shooting direction indicated by the position-related information. However, if high accuracy is not required, the same shooting direction may be set to approximately the same direction or Further, it may be considered as a shooting direction in which the difference in angle between the directions is within a pre-specified range.

  For example, the different time zones here are different time zones for observing and monitoring the situation of the imaging target in time series. The different time zones are usually time zones that do not overlap. The different time zones are, for example, a time zone where the start date and time and the end date and time are different, a time zone where the dates are different, and a time zone where the shooting years are different. For example, images taken at different times include images taken this year, images taken last year, images taken two years ago, and the like.

  The plurality of images constituting one image group are, for example, a series of a plurality of images photographed in one time zone for one photographing target. The series of images are, for example, a plurality of images obtained by shooting one shooting object by changing the combination of shooting position and shooting direction. The series of images may be a plurality of images taken at an arbitrary timing by changing the shooting position and the shooting direction in one time zone. The series of images are, for example, a plurality of still images obtained by shooting one shooting target at regular or indefinite time intervals. In each of the series of images, the entire subject may be photographed, or only a part of the subject may be photographed. The series of images is, for example, a plurality of images obtained by shooting different areas of one shooting target. However, the different regions here may partially overlap. The series of multiple images may be, for example, a plurality of continuous frame images that constitute a moving image in which a single photographing target is photographed by changing the combination of the photographing position and the photographing direction. Alternatively, the series of images may be a plurality of frame images obtained by acquiring one frame image at every other frame image from a plurality of continuous frame images constituting the moving image. Good. For example, the series of images may be a plurality of frame images obtained by thinning out a plurality of continuous frame images constituting a moving image. Here, every other one or more frame images may be every other number of frame images specified in advance, or every other number of frame images that are changed in accordance with conditions specified in advance. good.

  For example, position related information corresponding to one or more images of a plurality of images constituting the image group is stored in the image storage unit 101 in association with the two or more images. For example, position related information associated with each image constituting the image group is associated with each image and stored in the image storage unit 101. Each image group may be considered as a plurality of pieces of information including, for example, an image and position-related information corresponding to the image. For example, the position storage information obtained for one or more frame images constituting a plurality of frame images of a moving image constituting the image group is associated with the one or more frame images obtained from the position related information. Stored in

  Each image group may store information indicating the time zone when the image was taken, for example, information such as the shooting time, the date and year when the image was taken, and the like. In addition, each image group may be associated with an identifier of a photographing target. The identifier of the shooting target is, for example, a name of the shooting target, a code assigned to the shooting target, or the like. Instead of associating these shooting times and shooting target information with the image group and storing them in the image storage unit 101, these pieces of information may be associated with each image constituting the image group. Note that the image group may be associated with information indicating the image group explicitly, or may not be associated with the image group. When not associated with each other, for example, an image associated with a shooting time in one time zone or an image associated with a specific serial number or the like may be considered as one image group.

  The plurality of images stored in one image group may have information indicating the image acquisition order, a time stamp indicating the time when the images were acquired, and the like. The time here may be a relative time or an absolute time.

  The image stored in the image storage unit 101 may be a color image or a grayscale image, and the type of image, the file format of the image, etc. are not limited. Moreover, the number of pixels of an image, an aspect ratio, etc. are not ask | required.

  It is preferable that the angle of view when capturing a series of images is constant. However, it may not be constant.

  The process in which an image group and position related information are accumulated in the image storage unit 101 does not matter. In the following, a case where the storage unit 105 stores an image group and position related information will be described as an example.

  One or more images, preferably all images, of each image group stored in the image storage unit 101 may store sensor information related to the images in association with each other. The sensor information is information acquired by a sensor (not shown) provided in the camera 2 or a device (not shown) that transports the camera 2 when each image is taken. The sensor information is, for example, information having one or more of temperature, humidity, radio wave level, radioactivity level, etc., and each sensor information is a temperature sensor, humidity sensor, radio wave level sensor, radioactivity level sensor, etc. Is the acquired information. Instead of associating sensor information with one or more images, sensor information may be associated with each image group and stored. For example, in this case, sensor information acquired by the sensor at one point in time from the start of shooting to the end of shooting, before the start of shooting, or at one point after the end of shooting may be associated with each image group.

  One or more images, preferably all images, stored in the image storage unit 101 may be stored in association with image feature amounts. The image feature amount is, for example, one or more pieces of information such as the luminance, color tone, and luminance distribution of the image.

  The image storage unit 101 may store link information for linking images included in different image groups. The link information is information indicating association between images. The image associated with the link information is, for example, an image captured at a corresponding shooting location or an image shot at a corresponding shooting location and shooting direction. Further, the corresponding captured image and shooting direction will be described later. Further, the image associated with the link information may be, for example, an image obtained by capturing a corresponding part of the subject, specifically the same part. The link information is, for example, information including a pointer indicating a storage location for a plurality of linked images, address information, and the like. The link information may be information having associated image identifiers of a plurality of linked images. Note that link information to another image may be associated with one image. Link information is widely understood.

  The image storage unit 101 is preferably a nonvolatile recording medium, but can also be realized by a volatile recording medium. The same applies to other storage units.

  The image acquisition unit 102 acquires a series of a plurality of images. The image acquisition unit 102 acquires, for example, a series of images taken by the camera 2 and the like. The series of images acquired by the image acquisition unit 102 is, for example, a series of images obtained by shooting the shooting target within one time zone. The series of images are, for example, images taken by the camera 2 or the like at regular or indefinite time intervals. For example, the image acquisition unit 102 may acquire a series of a plurality of images by receiving a series of a plurality of images, that is, a series of a plurality of still images. Further, the image acquisition unit 102 may acquire a plurality of frame images constituting the moving image as a series of a plurality of images by receiving a moving image formed of a plurality of frame images. Alternatively, a moving image may be considered as an image group.

  Further, the image acquisition unit 102 receives a moving image, and acquires one frame image at a time every one or more frame images from a plurality of continuous frame images constituting the received moving image. The image may be acquired as a series of images. Here, the number of one or more frame images serving as an interval for acquiring the frame images may be constant or indefinite. Indefinite is a concept that includes being variable. Acquiring one frame image every one or more frame images means acquiring frame images sequentially with one or more frame images separated, or acquiring a frame image at every one or more frame image intervals. You may think that. Acquisition of a frame image may be considered as extraction of a frame image.

  For example, the image acquisition unit 102 may acquire one frame image from the received moving image one by one for every variable number of frame images. For example, the image acquisition unit 102 may acquire one frame image from the moving image received from the camera 2 or the like every other frame image corresponding to the moving speed of the camera 2 when shooting the moving image. . For example, as the moving speed of the camera 2 increases, the number of frame images that are targets for the image acquisition unit 102 to acquire one frame image is set to decrease continuously or stepwise.

  For example, the image acquisition unit 102 acquires the moving speed of the camera 2 when the frame image is captured, and each time the moving speed is acquired, the moving speed of the camera 2 prepared in advance according to the acquired speed, Using information such as a table that shows the correspondence with the number of frame images from which one frame image is acquired, the number of frame images from which one frame image is acquired, that is, the frequency at which the frame image is acquired is determined. You may do it. For example, the moving speed may be acquired at every time interval specified in advance, may be acquired at every indefinite time interval, or may be acquired at every trigger specified in advance. The image acquisition unit 102 may acquire the moving speed of the camera 2 when a frame image is captured every one or more predetermined number of frame images. The image acquisition unit may acquire frame images every number of frame images corresponding to the moving speed, that is, at intervals corresponding to the moving speed, every time the moving speed is acquired. That is, the frequency of acquiring one frame image and the interval of acquiring frame images may be updated according to the moving speed. For example, after acquiring one frame image, in order to determine the next frame image to be acquired, the moving speed at the time of shooting the acquired frame image and the frame images before and after the acquired frame image is acquired, and according to this moving speed You may make it acquire the back frame image by the determined number of frames. The number of frame images is also determined by the image acquisition unit 102 using an arithmetic expression for calculating the number of frame images to be acquired with a predetermined moving speed as an argument. The number of frame images may be determined from the speed.

  In addition, the table as described above, the arithmetic expression, and the like are such that at least a part of adjacent frame images, more preferably half or more overlap, when arranged in the shooting order among the frame images to be acquired, Set to be an arithmetic expression. For example, the number of frame images from which one frame image of this table is acquired and the calculation formula may be set according to the angle of view of the camera 2, the frame rate of the moving image, and the like.

  In addition, instead of the moving speed at the time of shooting each frame image of the camera 2 described above, an average speed or a maximum speed at the time of shooting a moving image by the camera 2 may be used. Or you may make it use the maximum speed of the moving body (not shown) which moves the camera 2 at the time of imaging | photography. In these cases, the frame images may be sequentially acquired every frame image of the number of frames determined according to the average speed and the maximum speed.

  The moving speed at the time of shooting by the camera 2 may be acquired using a moving speed acquired using a speed sensor or the like (not shown) provided in the camera 2. For example, when the image acquisition unit 102 acquires a moving image, information on the moving speed of the camera at the time of capturing each frame image, information on the average speed, the maximum speed, and the like may be acquired. For example, the image acquisition unit 102 acquires information on the moving speed of the camera associated with the time acquired at the time of shooting, and corresponds to a time that matches the time at which each frame image was shot from the information on the moving speed. The attached movement speed information may be acquired as movement speed information when each frame image is captured.

  Further, the moving speed at the time of shooting by the camera 2 may be acquired using a moving image shot by the camera 2. For example, by calculating a motion vector, an optical flow, or the like of two or more frame images in which the shooting order is continuous, the moving speed of the shooting target captured in the frame image of the moving image can be acquired. If it does not move, it can be considered that the moving speed of the object to be photographed corresponds to the moving speed of the camera 2, and therefore the moving speed of the camera 2 can be acquired from this moving speed.

  In addition, in a plurality of frame images in which the shooting order is continuous with respect to one frame image, the movement amount of the pixels constituting the image and the feature points of the image in the frame image is detected using a motion vector, an optical flow, etc. Then, the frame image immediately before the movement amount becomes equal to or greater than a predetermined threshold value may be sequentially acquired. Even in such a case, it may be considered that the frames are sequentially acquired as every other frame image of one or more frame images corresponding to the moving speed of the camera 2 at the time of photographing.

  The series of images acquired by the image acquisition unit 102 is, for example, an image in which at least a part, preferably half or more, of two images adjacent to each other in the shooting order overlap. Is preferred. An image having an adjacent shooting order may be considered to mean a set of one image and the next closest image in the shooting order. Since the overlapping of images has been described above, detailed description thereof is omitted here.

  For example, the image acquisition unit 102 may receive a series of a plurality of images or moving images by receiving a series of a plurality of images or moving images transmitted by the camera 2 or the like through wired communication or wireless communication. Further, the image acquisition unit 102 may receive a series of multiple images and moving images by reading out the series of multiple images and moving images recorded on a removable recording medium such as a memory card by the camera 2. good. For example, the image acquisition unit 102 may receive a still image or a frame image constituting a moving image taken by the camera 2 in real time immediately after shooting, or may receive a series of a plurality of images or moving images collectively. good. The acceptance here is a concept including reception, reading, and the like.

  For example, when acquiring a series of a plurality of images, the image acquisition unit 102 may acquire sensor information corresponding to at least one of the images. The sensor information is information acquired by one or more sensors provided in the camera 2 or the like. For example, the image acquisition unit 102 may receive sensor information associated with one or more of a series of multiple images from the camera 2 or the like. The sensor information associated with the image is, for example, sensor information acquired at the same time as the time when the image was captured, and specifically corresponds to the same acquisition time as the capture time associated with the image. It is attached sensor information. Alternatively, sensor information linked to each image by the camera 2 or the like may be used. Note that the image acquisition unit 102 may acquire (for example, receive from the camera 2) the acquired sensor information before, after, or in the middle of capturing a series of images.

  In addition, when acquiring one or more of a series of images, the image acquisition unit 102 acquires information indicating a shooting position corresponding to each of the one or more images and information indicating a shooting direction. Also good. For example, the information indicating the shooting position here is position information acquired by the GPS or the like provided in the camera 2 when shooting each image, for example, coordinates of the position. Further, the information indicating the photographing direction is information indicating, for example, the inclination in the triaxial direction acquired by a gyro sensor, a triaxial acceleration sensor, a tilt sensor, or the like provided in the camera 2. The information indicating the shooting position corresponding to each image and the information indicating the shooting direction are, for example, information indicating the shooting position and shooting direction associated with the acquisition time that matches the acquisition time of each image.

  The image acquisition unit 102, as a series of a plurality of images obtained by shooting one imaging target in different time zones, a series at which the above-described shooting start position or a position at which shooting starts and a shooting direction are the same. It is preferable to acquire a plurality of images.

  The image acquisition unit 102 can be realized by a device driver of an input unit, control software for a menu screen, or the like. Further, the process in which the image acquisition unit 102 performs a process for extracting a frame image from a moving image can be realized by using an MPU, a memory, or the like.

  The position related information acquisition unit 103 acquires position related information for one or more images using two or more images of a series of multiple images acquired by the image acquisition unit 102. For example, the position-related information acquisition unit 103 preferably uses all two or more images in the series of images acquired by the image acquisition unit 102 for one or more images in the series of images. The position related information is acquired for the image. For example, the position related information acquisition unit 103 uses two or more images of the series of images acquired by the image acquisition unit 102 and uses one or more of the two or more images for the position related information. To get. For example, the position related information acquisition unit 103 may acquire position related information for all the images using all of the series of images acquired by the image acquisition unit 102.

  When acquiring position-related information using a part of two or more images in a series of images, two or more images in the series of images used by the position-related information acquisition unit 103 are in the order of shooting. In the case of arrangement, it is preferable that at least a part of adjacent images, preferably half or more, be composed of overlapping images. An image adjacent when arranged in the order of photographing may be considered to mean a set of one image and the next closest image in the order of photographing. In addition, the two or more images include images taken at the shooting start position so that relative positions can be matched between a series of images shot at the same shooting target in different time zones. Is preferred.

  For example, the position-related information acquisition unit 103 acquires one frame image every other one or more frame images from a plurality of consecutive frame images constituting the moving image acquired by the image acquisition unit 102, and acquires the acquired frame image The position related information of each frame image is acquired using two or more of the above. The number of one or more frame images here is the same as the number of frame images in the process of acquiring one frame image from the moving image every other one or more frame images in the image acquisition unit 102. The detailed explanation is omitted.

  For example, the position-related information acquisition unit 103 selects every other one or more frame images corresponding to the moving speed of the camera 2 when shooting a moving image from the moving image acquired by the image acquisition unit 102 from the camera 2 or the like. Frame images may be acquired one by one, and position related information of these frame images may be acquired using two or more acquired frame images. The acquisition of the frame image corresponding to the moving speed of the camera 2 is the same as the process of acquiring the frame image corresponding to the moving speed of the camera 2 by the image acquisition unit 102 described above, and thus the description thereof is omitted here.

  Hereinafter, a process in which the position-related information acquisition unit 103 acquires position-related information using two or more images in a series of images will be described. Here, a case where information on a shooting position and a shooting direction is acquired as position related information will be described as an example. Here, the internal parameters of the camera 2 used for capturing the image (for example, the focal length, angle of view, resolution, width of the image sensor, etc. of the camera) are known, and the position related information acquisition unit 103 A case where internal parameters can be used as appropriate will be described.

  The position-related information acquisition unit 103 acquires two images that are adjacent in the order in which the images were taken from two or more images out of a series of images acquired by the image acquisition unit 102. Usually, two images that are adjacent to each other in the order of shooting are acquired in order from the image shot at the shooting start position. The two or more images are preferably images that are at least partially overlapped, preferably half or more. Then, information about the shooting position and the shooting direction is acquired using the two acquired images. For example, the position related information acquisition unit 103 detects a plurality of feature points for two images, and associates the feature points between the two images. Then, a rotation matrix and a translation vector are acquired using a 5-point algorithm or the like using the set of corresponding points obtained by the association and the internal parameters of the camera 2. This rotation matrix and translation vector are information representing the relative shooting direction and the relative shooting position of the other image with respect to one of the two images. When the information indicating the shooting position and the shooting direction for one of the two images is known or specified in advance, the information indicating the shooting position and the information indicating the shooting direction, the rotation matrix, and the translation vector are used. Information indicating the shooting position and shooting direction of the other image can be acquired.

  Specifically, it is assumed that the two images initially used for calculating the rotation matrix and the translation vector are the images taken at the start of shooting, and the next image whose shooting order is closest to this image. The shooting position and the shooting direction of the image shot at the start of the shooting are specified in advance, and the acquired rotation matrix and translation vector are used when the image of the image shot at the start of the shooting is taken by the camera 2. Assuming that the rotation matrix and translation vector are acquired with reference to the position and direction of the image, the rotation matrix, the translation vector, and information indicating the position of the camera 2 at the start of imaging (for example, the origin on three-dimensional coordinates For any image taken at the start of shooting out of two or more images in a series of multiple images using information indicating the direction and the direction (for example, a direction vector). The shooting order is the most It is possible to acquire information indicating the position and direction of the camera 2 at the time of photographing the image close. The position information and the direction information acquired in this case are position related information of an image closest to the image captured at the start of imaging. The position information and direction information acquired in this case are relative position and direction information with reference to the position and direction of the camera 2 at the start of shooting. The rotation matrix and the translation vector value may be acquired as position related information. In addition, when the position and direction of the camera 2 at the start of photographing are information indicating absolute coordinates or absolute directions, it is also possible to acquire information on absolute positions and directions.

  After that, the same processing is performed on the image captured at the start of the image capturing for the image closest to the first image capturing order and the image closest to the second image capturing position. Get direction information. Thereafter, similarly, by performing the same processing in order for a set of images after the image with the second closest shooting order, two or more images in the series of images acquired by the image acquisition unit 102 are: It is possible to acquire position-related information each having information on the shooting position and direction. Note that the position related information acquisition unit 103 may acquire only the information on the shooting position.

  If the internal parameters of the camera 2 are unknown, the internal parameters of the camera 2 may be specified simultaneously using a 12-point algorithm.

  Refer to the following Non-Patent Documents 1 and 2 for details of the process of acquiring the position of the camera 2 and the shooting direction using the above five-point parameters and the like. Refer to the following Non-Patent Document 3 for the 5-point parameter. “(Non-Patent Document 1) Mamoru Miura, 5 others,“ 3D shape measurement from 2 viewpoints based on moving camera and its performance evaluation ”, Journal of the Institute of Image Information and Television Engineers, Vol.68, No.4, pp. J135-J143, 2014 ". “(Non-Patent Document 2) Shuntaro Yamazaki, 2 others,“ Self-position estimation method for first-person vision system ”, [online], IEICE Technical Report, The Institute of Electronics, Information and Communication Engineers, [May 12, 2015 Day search], Internet <URL: http://www.dh.aist.go.jp/~shun/research/localization/PRMU-2010-05.pdf>. “(Non-Patent Document 3) David Nister,“ An efficient solution to the five-point relative pose problem ”, IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 26 (6): 756-777, June 2004 "

  For example, the following method of Non-Patent Document 4 may be used as a similar process for acquiring position-related information from two images. "(Non-Patent Document 4) Kunihiro Nishimura, 1 outside," Camera position estimation system that can be used both indoors and outdoors without markers-3D position (position / posture) estimation technology using image processing-", [online ] [Search May 12, 2015], Internet <URL: http://www.ipa.go.jp/files/000007216.pdf>.

  Instead of acquiring two images in the order of shooting and acquiring position-related information, two images may be acquired in the reverse order of the shooting order to acquire position-related information.

  Further, when the position-related information acquisition unit 103 acquires position-related information from two or more of a series of multiple images, each image may be appropriately resized or the like.

  In addition, when the image acquisition unit 102 acquires one or more of a plurality of images in series, shooting corresponding to each of the one or more images acquired by the GPS provided in the camera 2, a gyro sensor, or the like. When the information indicating the position or the information indicating the shooting direction is acquired, the position related information acquisition unit 103 uses the one or more images instead of acquiring the position related information using two or more images as described above. Information indicating the shooting position and shooting direction associated with the image may be acquired as position related information for each of the one or more images.

  Further, a distance measuring sensor (not shown) is provided in the camera 2 or the like, and the image acquisition unit 102 or the like provides information on the distance between the shooting position and the shooting target acquired when the distance measuring sensor or the like takes each image. The position related information acquisition unit 103 uses the distance information, the shooting position information acquired above, and the shooting direction information as the position information of the position related information. Information on the position of the subject to be photographed, that is, information on the position of the portion in the image to be photographed may be calculated and acquired.

  The feature amount acquisition unit 104 acquires image feature amounts for one or more of a series of images acquired by the image acquisition unit 102. For example, the feature amount acquisition unit 104 acquires one or more pieces of information such as luminance, color tone, and luminance distribution for one or more images.

  The storage unit 105 includes an image group composed of a series of images acquired by the image acquisition unit 102 and position related information acquired by the position related information acquisition unit 103 for one or more images constituting the image group. Are stored in the image storage unit 101. For example, the storage unit 105 includes an image group composed of a series of images acquired by the image acquisition unit 102, and the position related information acquired by the position related information acquisition unit 103 for each image constituting the image group. Are stored in the image storage unit 101. In addition, for example, the storage unit 105 acquires an image group composed of a plurality of frame images constituting the moving image acquired by the image acquisition unit 102 and a position acquired for each of one or more frame images constituting the image group. The related information may be accumulated in the image storage unit 101. The storage unit 105 stores, for example, position-related information acquired for one image among a plurality of images constituting one image group in association with the one image.

  In addition, the storage unit 105 stores the sensor information acquired by the image acquisition unit 102 for one or more images included in the image group in the image storage unit 101 in association with the image from which the sensor information is acquired. May be.

  In addition, the storage unit 105 associates the feature amount acquired by the feature amount acquisition unit 104 with respect to one or more images among the images constituting the image group in the image storage unit 101 in association with the image from which the feature amount has been acquired. It may be stored.

  Further, the storage unit 105 may store a target identifier, which is information for identifying a shooting target, in association with each of a series of a plurality of images acquired by the image acquisition unit 102. Alternatively, a target identifier may be stored in association with an image group composed of a series of images. The target identifier is, for example, a name indicating the shooting target, a code for identifying the shooting target, an address of the shooting target, coordinates where the shooting target is located, and the like.

  Further, the storage unit 105 may store information on the shooting date and time in association with each of a series of images acquired by the image acquisition unit 102. Or you may accumulate | store the information of imaging | photography date corresponding to the image group comprised by a series of several images. The information on the shooting date and time here is, for example, information having a shooting date, year, time, and the like. When the shooting date and time associated with the image group has a time, this time may be a time representing a shooting period such as a shooting start time or a shooting end time. The date and time here may be the date and time associated in advance with the image acquired by the image acquisition unit 102 (for example, the date and time given to each image by the camera 2 or the like). The date and time at the time of acquisition may be used. This date and time can be acquired from, for example, a clock (not shown).

  The link setting unit 106 links images included in different image groups using position-related information. For example, the link setting unit 106 links images that are included in different image groups and that correspond to the associated position-related information. The image corresponding to the associated position-related information is, for example, a plurality of images in which each position indicated by the position-related information associated with the image satisfies a predetermined condition or an image. Each of the positions and the shooting directions indicated by the position-related information is a plurality of images that satisfy a predetermined condition. The predesignated condition for the position is, for example, a condition for detecting position-related information in which the position such as the shooting position and the shooting direction are substantially the same. The predesignated condition for the position is, for example, a condition that the position is within a predesignated range, or a condition that the distance between the positions is equal to or less than a threshold value. The predesignated condition for the photographing direction is, for example, a condition that the angle formed by the photographing direction is within a predesignated range.

  Further, the image corresponding to the associated position-related information is, for example, an image indicating the same position in the position-related information associated with the image, or an image indicating the same position and the same shooting direction in the position-related information. It may be. In this case, it may be considered that the position and the direction are the same even if the position and the direction have variations or errors specified in advance.

  For example, the link setting unit 106 corresponds to the position-related information regarding the position when the distance between the positions indicated by the position-related information respectively associated with two or more images is equal to or less than a predetermined threshold value. You may judge. Alternatively, when the difference between the coordinate values of the three-axis coordinates, which is information indicating the position of the position-related information respectively associated with two or more images, is equal to or less than the threshold value, the position-related information is related to the position. May be determined to correspond. In addition, when the angle between the directions indicated by the information indicating the shooting direction (for example, the direction vector) included in the position related information respectively associated with two or more images is equal to or less than the threshold value, It may be determined that the related information corresponds.

  Note that the link setting unit 106 has, for each image, one or more image groups different from the one image, the position closest to the position indicated by the position related information corresponding to the one image. Is associated with position-related information associated with position-related information (or position-related information indicating the closest direction and the closest direction to the position and direction indicated by the position-related information corresponding to one image). You may make it link an image.

  Note that the link setting unit 106 acquires the position and shooting direction from the position related information of other images constituting the same image group for images that are not associated with the position related information. Images that constitute different image groups may be linked by using as information related to position-related information regarding the position at the time of photographing. For example, the link setting unit 106, for one image that is not associated with the position related information of each image group, has the shooting order associated with the position related information from the images taken before and after that. The closest image is detected, and the position and shooting direction information of the one image is acquired using the position and shooting direction indicated by the position-related information of each detected image. For example, an intermediate position of each position and an intermediate angle in the shooting direction are acquired as information on the position and shooting direction at the time of shooting for one image. Linking is performed in the same manner as described above, using information on the position and shooting direction obtained for one image in this way as position-related information substantially associated with this one image. You may do it.

  Alternatively, when position related information is not associated with one image of an image group to which a link is to be set, the position related information of the image associated with the position related information that has the closest shooting order to that image Between the image of the other image group that is associated with the corresponding position related information and the image that is associated with the position related information that is closest to the image in the shooting order. You may make it link the image in the same positional relationship as a positional relationship. The positional relationship between the image for which a link is to be set and the image associated with the position-related information that has the closest shooting order to this image is, for example, the image that is to be set the most for this image. This is information indicating the order and order of shooting and the order (for example, the second image after shooting, etc.) for images associated with position-related information that are close in order.

  The link setting unit 106 may link the images of different image groups in one-to-one, one-to-many, or many-to-many.

  Linking means associating images. Specifically, link information, which is information for associating images, is acquired and stored in the image storage unit 101 or a storage unit (not shown). The link setting unit 106 may associate information indicating another image associated with this image with one image. In this case, the information indicating the other image is considered as link information. Also good. The link information may be considered as information specifying one or more images associated with one image or information for reading. The link is widely understood.

  The image designation accepting unit 107 accepts designation of one or more images in the image group stored in the image storage unit 101. The designation of the image may be performed in any way as long as the designated image can be specified. For example, reception of designation of an image may be reception of designation of an image itself, or may be reception of designation of an image identifier associated with an image, a date and time associated with an image, and the like. For example, the output unit 109 described below displays one or more images included in one or more image groups on a monitor or the like, and selects one or more images from the displayed one or more images. May be received from the user via an input device or the like (not shown), the image designation accepting unit 107 may accept designation of the selected image.

  Here, reception is a concept including reception of information input from an input device such as a keyboard, mouse, touch panel, reception of information transmitted via a wired or wireless communication line, and the like. The input means may be anything such as a numeric keypad, keyboard, mouse or menu screen. The image designation receiving unit 107 can be realized by a device driver of an input unit such as a numeric keypad or a keyboard, menu screen control software, or the like.

  The corresponding image acquisition unit 108 is configured to specify an image associated with the one or more images received by the image designation receiving unit 107 by the position related information in the image group stored in the image storage unit 101. Obtained from an image group different from the image group including the image. For example, the corresponding image acquisition unit 108 stores, in the image storage unit 101, an image associated with the position related information corresponding to the position related information associated with one or more images received by the image designation receiving unit 107. Acquired from an image group different from the image group including the designated image among the stored image groups. However, this case is limited to the case where the image whose designation is accepted by the image designation receiving unit 107 is associated with the position related information. As described above in the description of the link setting unit 106, the image corresponding to the associated position related information is, for example, a condition in which each position indicated by the position related information associated with the image is designated in advance. A plurality of images satisfying the above, and each position and shooting direction indicated by the position related information associated with the images have the same position related information associated with the plurality of images or images that satisfy the predetermined conditions. It is an image showing a position or an image showing the same position and the same shooting direction in position-related information. The conditions specified in advance are the same as described above. The different image groups here are preferably image groups obtained by photographing the same subject, for example, image groups associated with the same subject identifier when the position indicated by the position related information is relative coordinates. . The different image groups here may be, for example, all image groups or some image groups. For example, it may be one or more image groups designated by the user or the like.

  For example, the corresponding image acquisition unit 108 acquires position-related information associated with the one image that the image specification receiving unit 107 has received from the image storage unit 101 and acquires one or more image groups (for example, specified by the user). The position satisfies the condition specified in advance for the position of the position-related information acquired above from among the position-related information associated with one or more images constituting the image group). One or more pieces of position-related information satisfying a pre-specified condition with respect to a shooting direction included in the position-related information acquired above are detected, and one or more pieces of information associated with the detected one or more pieces of position-related information are detected. Get an image. The conditions specified in advance for the position and the shooting direction are, for example, the same conditions as those described above in the description of the link setting unit 106. When there are a plurality of images for which the image designation accepting unit 107 has accepted designation, the above-described processing may be performed for each image to obtain the image.

  For example, the corresponding image acquisition unit 108 acquires position-related information associated with the one image that the image specification receiving unit 107 has received from the image storage unit 101 and acquires one or more image groups (for example, specified by the user). Information indicating the position of the position-related information associated with the one image that has received the designation acquired from the position-related information associated with one or more images constituting the image group) (for example, , Coordinates) information indicating the position where the distance is equal to or less than the threshold, and information indicating the shooting direction included in the position related information associated with the one image that has received the designation acquired above (for example, direction One or more pieces of position-related information having information indicating a shooting direction in which the magnitude of an angle formed with respect to the direction indicated by the vector is equal to or less than a threshold value, and is associated with the detected one or more pieces of position-related information. To get one or more of the image. In addition, instead of determining whether or not the distance between the positions indicated by the information indicating the positions included in the two position-related information is equal to or less than the threshold value, the positions of the position-related information associated with the two images are indicated. It is determined whether the difference between the coordinate values of the three-axis coordinates, which are information, is less than or equal to the threshold value, respectively, and when all are less than or equal to the threshold value, the distance is handled in the same manner as when the distance is less than or equal to the threshold value Also good.

  Note that the corresponding image acquisition unit 108 performs, for each image group that is different from the image group that includes the one image, for the one image that the image designation receiving unit 107 has received the designation. The position related information indicating the closest position to the position indicated by the position related information corresponding to (or the position indicating the closest and closest position to the position and direction indicated by the position related information corresponding to one image) You may make it acquire the image matched with the positional relevant information matched with the related information).

  Note that the corresponding image acquisition unit 108 determines the position and shooting direction of an image that is not associated with position-related information using the position-related information of other images that form the image group including the image. An image corresponding to the designated image may be acquired from another image group using the position and shooting direction as information corresponding to the position-related information of the image. In such a case, it may be considered that an image associated with the position-related information is acquired. That is, the image associated with the position-related information may be considered not to be an image associated with the position-related information directly associated with the image, for example. For example, the correspondence image acquisition unit 108, for one image that is not associated with the position related information of each image group, from the images photographed before and after that, the photographing order associated with the position related information, respectively. Is detected, and the position and shooting direction information of the one image is acquired using the position and shooting direction indicated by the position-related information of each detected image. For example, an intermediate position of each position and an intermediate angle in the shooting direction are acquired as information on the position of one image and the shooting direction. Then, the information on the shooting position and shooting direction acquired for one image that is not associated with position-related information in this way is used as position-related information that is substantially associated with this one image. The image associated with the position related information corresponding to the position related information associated with the specified image may be detected and acquired. In this case, the image that is not associated with the position-related information and for which the information corresponding to the position-related information is acquired may be an image for which the image designation receiving unit 107 has received the designation. The image acquired with respect to the received image may be sufficient, and both may be sufficient.

  Alternatively, when the position-related information is not associated with the designated image, it is associated with the position-related information corresponding to the position-related information of the image associated with the position-related information that is closest to the image in the shooting order. For images in other image groups, an image having the same positional relationship as the positional relationship between the image that has received the designation and the image closest to the image capturing order may be acquired. The positional relationship between the image that has received the designation and the image that has the closest shooting order to this image and that is associated with the position-related information is, for example, the image that has been specified and has the closest shooting order to this image. , Information indicating the order and order of shooting with respect to the image associated with the position-related information and the order (for example, the second image after shooting).

  Note that the corresponding image acquisition unit 108 converts one or more images received by the image designation receiving unit 107 into one or more images received by the image designation receiving unit 107 as images associated with the position related information. On the other hand, one or more images linked by the link setting unit 106 may be acquired. Even in such a case, an image whose designation is accepted by the image designation accepting unit 107 and an image in an image group different from the image group having this image linked to this image correspond to the respective images, for example. If the attached position-related information is linked so as to correspond, the position-related information corresponding to the position-related information respectively associated with one or more images received by the image designation receiving unit 107 The image associated with the information can be acquired from an image group different from the image group having the image that has received the designation.

  In the following, as the image associated with the position related information corresponding to the position related information respectively associated with the one or more images for which the image designation receiving unit 107 has received the designation, A case will be described as an example where one or more images linked to one or more images for which designation has been received are acquired. If the corresponding image acquisition unit 108 does not acquire an image using a link, the link setting unit 106 may be omitted.

  The output unit 109 outputs the image acquired by the corresponding image acquisition unit 108. For example, an image whose designation is accepted by the image designation receiving unit 107 may be further output. The output unit 109 may output one or more images constituting one or more image groups stored in the image storage unit 101. The output unit 109 may output, for example, one or more image groups associated with the target identifier or date and time specified by the user, or one or more images included in the image group. Further, the output unit 109 may output sensor information and attribute information associated with the image.

  Output here means display on a monitor, projection using a projector, printing on a printer, transmission to an external device, storage on a recording medium, processing result to other processing devices or other programs, etc. It is a concept that includes delivery. The output unit 109 may or may not include an output device such as a monitor. The output unit 109 can be realized by output device driver software, or output device driver software and an output device.

  The camera 2 takes an image. For example, the camera 2 captures a plurality of still images or one or more moving images. The camera 2 captures a series of images, for example. The series of images are, for example, a plurality of still images taken at regular or indefinite time intervals. Further, the series of images may be, for example, a moving image composed of a plurality of frame images taken at a constant frame rate. However, the frame rate may be variable. In this case, the frame image corresponds to a series of a plurality of images. The camera 2 may capture a plurality of images including a series of a plurality of images. A plurality of images including a series of images may be considered as, for example, a moving image taken by the camera 2, and in this case, the series of images is one frame every one or more frame images from the moving image. It may be considered as a frame image acquired (extracted) one by one.

  When the camera 2 captures a plurality of still images, the time interval for capturing the still images may be constant or indefinite. For example, the plurality of still images captured by the camera 2 may be a plurality of still images captured by so-called time-lapse shooting. Note that a plurality of still images taken by time-lapse shooting may be considered as moving images having each still image as a frame image. Alternatively, the plurality of still images may be a plurality of still images that are sequentially captured in accordance with a user instruction, a trigger designated in advance, or the like.

  The image captured by the camera 2 may be a color image or a monochrome image. Further, the number of pixels of an image captured by the camera 2 is not limited. Further, the file format of the moving image to be shot does not matter. The camera 2 preferably has a fixed focal length or can travel. The camera 2 may be a camera that captures electromagnetic waves other than visible light, such as an infrared camera or an ultraviolet camera.

  An image taken by the camera 2 is transmitted to the image management apparatus 1 by, for example, a wired connection or a wireless connection. It may be transmitted to the image management apparatus 1 via a network. Or you may accumulate | store in the storage part etc. which are not shown in figure. The storage unit (not shown) may be a removable storage medium. An image stored in a storage unit (not shown) is appropriately transmitted to the image management apparatus 1 by, for example, wired connection or wireless connection. The same applies to other stored information. The storage unit (not shown) may be a removable storage medium. Each image is transmitted or stored in association with the date and time when the image was taken, for example. The date and time can be obtained from, for example, a clock (not shown).

  For example, the camera 2 may transmit the captured image to the image management apparatus 1 in real time each time a still image or a frame image is captured.

  The camera 2 may be provided with one or more sensors (not shown). One or more sensors are sensors which acquire the above-mentioned center information, for example. For example, a temperature sensor, a radio wave level sensor, a radioactivity level sensor, or the like. Information acquired by each sensor is transmitted to the image management apparatus 1 in association with an image captured by the camera 2, for example. Or it accumulate | stores in the storage part etc. which are not shown in correlation with the image which the camera 2 image | photographed. The information acquired by each sensor may be transmitted in association with the measurement time, or accumulated in a storage unit (not shown). The storage unit (not shown) may be a removable storage medium. Note that the camera 2 may include a sensor capable of detecting a moving speed, such as a speed sensor or an acceleration sensor, as one of the sensors described above.

  The camera 2 may be provided with means such as GPS for acquiring information indicating the current position. Information indicating the current position acquired by the GPS is transmitted to the image management apparatus 1 in association with an image captured by the camera 2 or stored in a storage unit (not shown), for example. Information indicating the current position acquired by GPS may be accumulated in the image management apparatus 1 or a storage unit (not shown) in association with the acquired date.

  The camera 2 may be installed on a moving body (not shown). For example, the mobile body may be an automobile, a vehicle such as a motorcycle, or the like. The mobile body may be an aircraft. The moving body may be a so-called drone, an unmanned aircraft capable of computer-controlling a remote operation or a flight route. By installing the camera in the drone, it is possible to photograph the appearance and part of a structure such as a building, which has been difficult to photograph, from various positions and directions. Note that this moving object may be considered as a part of the image management system 1000.

  Instead of moving the camera 2 by moving the moving body, the user may hold the camera 2 and move the camera 2.

  Next, an example of the operation of the image management apparatus 1 of the image management system 1000 will be described using the flowchart of FIG. Here, the image linked by the link setting unit 106 is associated with the position related information corresponding to the position related information respectively associated with one or more images for which the image designation receiving unit 107 has received the designation. An example will be described in which the corresponding image acquisition unit 108 acquires images as images of another image group, and the output unit 109 displays the images acquired by the corresponding image acquisition unit 108.

  (Step S101) The image acquisition unit 102 determines whether or not an input of an image has been received. . The received image may be a series of multiple images or a moving image. If accepted, the process proceeds to step S102. If not accepted, the process proceeds to step S118. In addition, when receiving the input of an image, you may further receive sensor information.

  (Step S102) The image acquisition unit 102 determines whether or not the image received in step S101 is a moving image. If it is a moving image, the process proceeds to step S103, and if it is not a moving image, the process proceeds to step S104.

  (Step S103) The image acquisition unit 102 extracts, that is, acquires one frame image at a time from one or more frame images from the moving image received in step S101, and extracts a series of a plurality of images. Each time a predetermined number of frame images are acquired, the moving speed of the feature points in the image is obtained from a plurality of previous and subsequent frame images, and the moving speed when the frame image is captured is estimated. Depending on the moving speed, the number of frame images indicating the frequency of acquiring one frame image may be determined thereafter. The initial value of this interval is, for example, a value designated in advance. The acquired image is temporarily stored in a storage unit (not shown).

  (Step S <b> 104) The image acquisition unit 102 resizes each of the series of images to a predetermined size. This resizing is usually a reduction.

  (Step S105) The image acquisition unit 102 substitutes 1 as the value of the counter m.

  (Step S106) The position related information acquisition unit 103 determines whether or not there is an (m + 1) th image in the series of images acquired by the image acquisition unit 102. If there is, the process proceeds to step S107, and if not, the process proceeds to step S114. It is assumed that the m-th image is an m-th image that is the mth earliest in the sequence of a plurality of images acquired by the image acquisition unit 102.

  (Step S107) The position related information acquisition unit 103 acquires the mth image and the (m + 1) th image from the series of images acquired by the image acquisition unit 102.

  (Step S108) The position related information acquisition unit 103 detects a feature point for each image acquired in step S107.

  (Step S109) The position related information acquisition unit 103 performs feature point matching between the m-th image and the m + 1-th image.

  (Step S110) The position related information acquisition unit 103 uses the result of matching in Step S109 to obtain information indicating the shooting position and direction of the (m + 1) th image based on the position and direction of shooting the mth image. get. Then, using this information, information indicating the shooting position and shooting direction with respect to the position and direction of shooting the first image is acquired for the (m + 1) th image. Here, the information indicating the shooting position and the shooting direction is the position related information acquired by the position related information acquisition unit 103. Here, since the shooting position and shooting direction when the m = 1st image is shot is a reference for other position-related information, a reference prepared in advance is used as the position-related information for the first image. Position-related information having information indicating the position and information indicating the shooting direction is acquired.

  Note that, here, as an example, the shooting position is acquired, but information indicating the position of the target being captured in the image is acquired using information on the distance from the image acquired by the camera 2 or the like to the shooting target. You may make it do.

  (Step S111) The position-related information acquisition unit 103 associates the information indicating the shooting position and shooting direction acquired in step S110 with the m + 1-th image as position-related information corresponding to the m + 1-th image, not shown. Temporarily store in a storage medium or the like.

  (Step S112) The link setting unit 106 acquires the shooting position acquired in step S110 for each image included in one or more image groups associated with the same target identifier acquired in the past and stored in the image storage unit 101. Then, an image associated with the position related information corresponding to the shooting direction is detected. For example, in the image constituting the image group acquired and stored immediately before, the shooting position acquired in step S110 has a shooting direction in which the angle formed with respect to the shooting direction acquired in step S110 is equal to or less than a threshold value. An image associated with the position related information having the shooting position closest to is detected.

  (Step S113) The link setting unit 106 links the (m + 1) th image with one or more images detected in step S112. For example, link information indicating linking between images is accumulated in the image storage unit 101 or a storage unit (not shown). If link information has already been accumulated, it is updated with new link information.

  (Step S114) The position related information acquisition unit 103 increments the value of the counter m by 1. Then, the process returns to step S106.

  (Step S115) The feature amount acquisition unit 104 acquires a feature amount for each of a series of images.

  (Step S116) The storage unit 105 uses the sensor information received in step 101 to acquire sensor information for each of a series of multiple images. For example, when each of the one or more pieces of sensor information received in step S101 is associated with the acquisition time at which the sensor information is acquired, it corresponds to the acquisition time closest to each acquisition time of the series of images. The attached sensor information is acquired as sensor information corresponding to each image.

  (Step S117) The accumulation unit 105 accumulates, in the image storage unit 101, an image group including a series of a plurality of images temporarily stored in step S111 and position-related information associated with each image. The accumulation unit 105 further stores the feature amount of each image acquired in step S115, the sensor information of each image acquired in step S116, the target identifier, and the acquisition time of each image in each image constituting the image group. You may accumulate | store by matching. Then, the process returns to step S101.

  (Step S118) The output unit 109 determines whether or not it is time to display images constituting one image group. For example, a reception unit (not shown) may determine whether or not an instruction to display one image group has been received, and if the instruction is received, it may be determined that the timing is to be output. In addition, when it is immediately after the image group is accumulated in step S117, it may be determined that it is the timing for displaying the images constituting the image group. If it is time to display, the process proceeds to step S119. If it is not time to display, the process returns to step S101.

  (Step S119) The output unit 109 displays images constituting one image group. For example, an image of the image group indicated by the display instruction received in step S118 is displayed. Alternatively, the images of the image group accumulated in the immediately preceding step S117 may be displayed.

  (Step S120) The image designation receiving unit 107 determines whether designation for one of the displayed images has been received. If accepted, the process proceeds to step S121. If not accepted, the process proceeds to step S122. Note that the image designation receiving unit 107 may further accept an operation for designating an image group including an image to be displayed, for example, an operation for designating a period during which the image group was captured.

  (Step S121) The corresponding image acquisition unit 108 acquires an image linked to the image specified in step S120 from the image storage unit 101. The corresponding image acquisition unit 108 may acquire all the linked images, or may acquire only a part thereof, for example, a newer image taken before the designated image. For example, when an operation for designating an image group including an image to be displayed is accepted in step S120, only a linked image of the designated image group may be read. If no link is set for the designated image, the process may proceed to step S124 without acquiring the image.

  (Step S122) The output unit 109 displays the image acquired in step S121 on a monitor or the like (not shown). Note that the image specified in step S120 may also be displayed. Then, the process proceeds to step S124.

  (Step S123) The output unit 109 determines whether an instruction to display an image of another image group has been received. If accepted, the process returns to step S119 to display an image of the instructed image group. If not accepted, the process proceeds to step S124.

  (Step S124) The output unit 109 determines whether or not to end the display of the images constituting the image group. For example, the display is ended when an instruction to end the display is received from the user via a reception unit (not shown) or the like. Then, the process returns to step S101.

  In the flowchart of FIG. 2, the process is terminated by powering off or a process termination interrupt.

  Hereinafter, a specific operation of the image management system 1000 in the present embodiment will be described.

  FIG. 3 is a perspective view seen from below showing an example of a drone 200 that is a moving body in which the camera 2 is installed. The drone 200 flies in the air, for example, by wireless remote control. The camera 2 is arranged below the drone 200 so that the optical axis faces the side. It is assumed that the direction of the optical axis of the camera 2 with respect to the drone 200, that is, the shooting direction is fixed.

  For example, in order to manage the state of the bridge “ABC Bridge”, the user is flying the drone 200 around the bridge with the lens of the camera 2 facing the bridge on June 5, 2015. Assume that a moving image is captured by the camera 2. It is assumed that a part of the bridge is captured in each frame image of the moving image. In addition, it is assumed that the drone 200 is arranged at a predesignated position with a predesignated direction at the start of the photographing. As a result, the camera 2 is arranged with a pre-specified direction at a pre-specified position at the start of shooting, and the first frame image to be shot at the start of shooting is designated in advance. The image is an image of the area. In addition, the camera 2 is provided with a temperature sensor, a radio wave level sensor, and a radioactivity sensor (not shown), and each of the camera 2 captures the temperature and radio wave at predetermined time intervals during moving image shooting. It is assumed that the level and the radioactivity level are detected, and the detected value is accumulated in a storage unit (not shown) of the camera 2 or the drone 200 in association with the detection time. Here, for convenience of explanation, it is assumed that a time stamp is given to the value acquired by each sensor at the same timing as the time stamp given to the frame image taken by the camera 2.

  After the shooting is finished, an image management apparatus is configured to wirelessly communicate a moving image (for example, a moving image file) captured by the camera 2, information on the shooting date and time, and sensor information acquired by each sensor or the like by user operation 1, the image acquisition unit 102 receives the moving image, information on the shooting date and time, sensor information associated with the detection time, and the like. Further, when the user inputs a character string “ABC bridge” indicating an imaging target via an input device (not shown) or the like, the image acquisition unit 102 receives this character string as a target identifier.

  Since the image received from the camera 2 is a moving image, the image acquisition unit 102 sequentially extracts frame images from the moving image one by one every one or more frame images. For example, first, a first frame image that is a frame image at the start of shooting is acquired, and thereafter, one frame image is sequentially acquired every two frames specified by default. For example, the first frame image, the fourth frame image, the seventh frame image, the tenth frame image, and the like of the moving image are sequentially acquired.

  FIG. 4 is a diagram schematically showing a part of a frame image constituting the moving image, for explaining a process in which the image acquiring unit 102 acquires a frame image from the moving image. For example, when the nth frame image 401 of the moving image is acquired, the frame images 402 and 403 are skipped and the n + 3th frame image 404 is acquired. A plurality of frame images acquired here, for example, a frame image including the frame image 401 and the frame image 404 is a series of a plurality of images acquired by the image acquisition unit 102.

  Here, the frame images are acquired at the same frequency specified by default, but each time one frame image is acquired, the frame image is captured from the next two or more consecutive frame images. The moving speed may be estimated, and the next frame image may be acquired at a frequency according to this speed. For example, when the estimated moving speed is slower than the previously estimated speed by a threshold or more, the frame image acquisition frequency is lowered by a predetermined degree, for example, by one frame, and when the estimated moving speed is faster than the threshold The frequency of acquiring the frame image may be increased by a predetermined degree, for example, by one frame. For example, when the estimated moving speed is slower than the immediately estimated speed by a threshold or more, one frame image is acquired every three frame images, and the estimated moving speed is a threshold value higher than the immediately estimated speed. If it is earlier, one frame image may be acquired every other frame image. Here, it is assumed that an image identifier in which a number indicating a date, a number indicating a time stamp, and the like are arranged is assigned to each acquired frame image. As an example, an image identifier “20150605013107” in which a number “20150605” indicating a date of “June 5, 2015” and a number “013107” indicating a time stamp of “1 minute 31 seconds 7 frames” are arranged is given. Shall be.

  Next, the position related information acquisition unit 103 sequentially acquires position related information for each of a series of images acquired by the image acquisition unit 102.

First, the position related information acquisition unit 103 includes information indicating a reference shooting position prepared in advance for an image having the earliest shooting time among a plurality of images acquired by the image acquisition unit 102, and a shooting direction. Position-related information having information indicating. For example, position-related information having coordinates (x ₁ , y ₁ , z ₁ ) as information indicating the shooting position and direction vectors (X ₁ , Y ₁ , Z ₁ ) as information indicating the shooting direction is acquired. . Here, the positions and shooting directions of other images are relative positions and directions with respect to the reference position and shooting direction. The position related information acquisition unit 103 temporarily stores the acquired position related information in association with the first image in a memory or the like (not shown). Here, it is assumed that x ₁ , y ₁ , z ₁ , X ₁ , Y ₁ , Z _1, etc. are arbitrary numerical values. The same applies to the following.

Next, the position related information acquisition unit 103 acquires an image with the earliest shooting time and an image with the second earliest from the series of images acquired by the image acquisition unit 102. Then, a plurality of feature points are detected for each image, and the feature points are matched. Then, using the matching result, information indicating the shooting position and shooting direction relative to the shooting position and shooting direction of the first image is acquired for the second image by a five-point algorithm or the like. It is assumed that the coordinates indicating the acquired shooting position are (x ₂ , y ₂ , z ₂ ), and the direction vector is (X ₂ , Y ₂ , Z ₂ ). The coordinates and direction vector indicating the shooting position are temporarily stored in a memory or the like (not shown) in association with the second image as position related information corresponding to the second image.

Next, the link setting unit 106 is another group of images already stored in the image storage unit 101, and the target identifier “ABC bridge” associated with the image received by the image acquisition unit 102 above. For each of one or more image groups associated with the same object identifier, an angle formed with respect to the direction indicated by the direction vector (X ₂ , Y ₂ , Z ₂ ) included in the position-related information acquired above is designated in advance. An image associated with position-related information having a direction vector that is equal to or less than the threshold and having coordinates closest to the coordinates (x ₂ , y ₂ , z ₂ ) of the position-related information acquired above. Are detected respectively. Although the threshold value here varies depending on the angle of view of the camera 2, it is preferably at least 180 degrees or less, and when the camera 2 is at the same position, the threshold is 90 degrees or less so that the shooting regions overlap each other by more than half. More preferably.

  FIG. 5 is an image management table for managing images stored in the image storage unit 101. The image management table has attributes of “target ID”, “image ID”, “position related information”, and “image”. The position related information further has attributes of “position” and “direction”. “Target ID” is a target identifier, and images having the same target identifier indicate that the images are taken of the same target. “Image ID” is an image identifier, and as described above, the front 8 digits indicate the date (year, month, day), and the rear 6 digits indicate the time stamp (minute, second, frame). Show. Here, it is assumed that the images associated with the “image ID” whose front 8 digits match are images constituting the same image group. “Position” is a coordinate indicating the shooting position. “Direction” is a direction vector indicating the photographing direction. “Image” is image data.

Here, for example, as an image group in which “target ID” is already associated with “ABC bridge” in the image storage unit 101 and each image is associated with position-related information, FIG. As shown in FIG. 5, two images are stored: an image group in which the shooting date is “June 2, 2014” and an image group in which the shooting date is “June 8, 2013”. For example, the link setting unit 106 has a direction vector whose angle formed with respect to the direction indicated by the direction vector (X ₂ , Y ₂ , Z ₂ ) is equal to or less than a predetermined threshold value for each image group, and detecting the association was image and location-related information distance to the acquired position-related information has a coordinate _{(x 2, y 2, z} 2) has the coordinates closest above. Here, in the image management table shown in FIG. 5, an image group whose date is “June 2, 2014”, that is, “Image ID” in which the first 8 digits of “Image ID” are “20140602”. The angle formed by the direction vector indicated by the “direction” attribute value and the direction indicated by the direction vector (X ₂ , Y ₂ , Z ₂ ) is equal to or less than a threshold value from the records (rows) having The record having the closest distance between the coordinates indicated by the “position” attribute value and the coordinates (x ₂ , y ₂ , z ₂ ) is detected, and the attribute value of “image ID” is acquired from the detected record. . It is assumed that the acquired attribute value is “20140602000010”. Similarly, in the image management table shown in FIG. 5, an image group whose date is “June 8, 2013”, that is, “image ID” whose first 8 digits are “20130608”. From the records (rows) having “ID”, the record whose “image ID” is “20130608000007” is detected, and the “image ID” of this record is acquired. Then, link information having the image identifiers “20140602000010” and “20130608000007” of each detected image and the image identifier of the second image acquired by the image acquisition unit 102 is acquired, and the acquired link information is stored in the image storage unit. 101. It is assumed that the image identifier of the second image acquired by the image acquisition unit 102 is “20150605000004”. Here, the link information stored immediately before is deleted when newly acquired link information is accumulated.

  FIG. 6 is a link information management table for managing link information stored in the image storage unit 101. Here, for convenience of explanation, a case where a plurality of pieces of link information are already stored will be described. The link information management table has an attribute indicating the date when the image group was taken. Here, it has attributes of “20130608”, “20140602”, and “20150605”. In the row corresponding to each attribute, the image identifier of the image corresponding to the date indicated by each attribute is arranged. The images indicated by the image identifiers included in each row of the link information management table are linked to each other. In the link information management table, for example, each time the image acquisition unit 102 acquires an image associated with a new date, a column indicating the corresponding date is added.

  The link setting unit 106 is an image group associated with the same target identifier as the image acquired by the image acquisition unit 102 and may set a lick only for an image included in the image group stored immediately before. Good. In this case, when a new column associated with the date of the image acquired by the image acquisition unit 102 is added to the link information management table and a link is set for the image stored immediately before, this What is necessary is just to arrange | position the image identifier of the image which the image acquisition part 102 in which the link was set to the newly added row | line | column of the record which has the image identifier corresponding to the image accumulate | stored immediately before. Further, since the first image of each image group is taken at the same shooting position and shooting direction, it may be considered that the first image is automatically linked without being managed by link information. Alternatively, similarly to the above, the first image may be linked to acquire link information.

  When the link setting by the link setting unit 106 is finished, the position related information acquisition unit 103 performs the same processing after the image having the second earliest shooting time among a plurality of images acquired by the image acquisition unit 102. The same processing is sequentially performed on the image. Specifically, using the image with the second earliest shooting time and the third earliest image, the position-related information is acquired and linked with respect to the third earliest image. Using the third earliest image and the fourth earliest image, position-related information is acquired for the fourth earliest image and linked. By sequentially performing such processing, the position-related information is acquired and linked for all of the series of images acquired by the image acquisition unit 102.

  Next, the feature amount acquisition unit 104 acquires a feature amount for each of a series of images acquired by the image acquisition unit 102. Here, for example, the average luminance value of each image is acquired. Note that the feature amount acquisition unit 104 may acquire a plurality of types of feature amounts. The feature amount acquisition unit 104 temporarily stores the acquired feature amount in association with the image identifier of each image in a memory (not shown) or the like.

  Next, the storage unit 105 acquires sensor information detected at the time of capturing each of the series of images acquired by the image acquisition unit 102 from the sensor information received by the image acquisition unit 102. For example, when each of the one or more pieces of sensor information received by the image acquisition unit 102 is associated with the acquisition time when the sensor information is acquired, the acquisition time closest to the acquisition time of each of the series of images Is acquired as sensor information corresponding to each image. For example, when the sensor information is associated with the time stamp of the moving image received by the image acquisition unit 102 and the synchronized acquisition time, the time indicated by the time stamp associated with each of the series of images is displayed. The sensor information associated with the closest time is acquired as sensor information corresponding to each image. The sensor information acquired here is assumed to be temperature, radio wave level, and radioactivity level. In the case where the sensor information is associated with the time stamp of the moving image received by the image acquisition unit 102 and the synchronized acquisition time, for example, the start of shooting by the camera 2 and the start of detection of the sensor are performed simultaneously. This is the case where the time stamp shooting start time and the sensor detection start time are set to the same time, for example, 0 minute 0 second.

  The storage unit 105 associates an image group composed of images temporarily stored in association with the position-related information by the position-related information acquisition unit 103 with the position-related information corresponding to each image, and stores the image storage unit 101. Further, when accumulating, the accumulating unit 105 further accumulates the target identifier “ABC bridge” received by the image acquiring unit 102 and the image identifier of each image in association with each image.

  FIG. 7 is a diagram illustrating an image management table in a state where the storage unit 105 stores a new image group in the image storage unit 101. Here, a record of a newly accumulated image is added to the image management table shown in FIG.

  FIG. 8 shows a set of coordinates indicating the shooting position and position vectors indicating the shooting direction of the position-related information associated with each image constituting the image group newly accumulated by the accumulation unit 105 as virtual 3. It is the schematic diagram arrange | positioned and shown in the dimensional space. In the figure, each black circle indicates a coordinate indicating a shooting position, and an arrow extending from the black circle indicates a direction vector. A black circle 80 indicates a shooting start position. An arrow 81 indicates the outline of the moving direction of the photographing position. This figure may be considered to represent the relationship between the movement trajectory of the camera 2 at the time of shooting and the shooting direction. However, FIG. 8 is a schematic diagram for explanation, and does not accurately represent the position related information corresponding to the image group newly accumulated by the accumulation unit 105.

  The accumulation unit 105 accumulates the feature amounts acquired by the feature amount acquisition unit 104 for each of a series of images in the image storage unit 101 in association with the images.

FIG. 9 is a feature amount management table for managing the feature amounts of images stored by the storage unit 105 in association with each image. The feature amount management table has attributes of “image ID” and “brightness value”. The “luminance value” is an average luminance value that is a feature amount of an image. Note that V _{1 and the} like here are arbitrary values indicating luminance.

  Further, as described above, the storage unit 105 stores the sensor information acquired for each of a series of multiple images in the image storage unit 101 in association with each image.

FIG. 10 is a sensor information management table in which the storage unit 105 manages sensor information corresponding to images stored in association with each image. The sensor information management table has attributes of “image ID”, “temperature”, “radio wave level”, and “radioactivity level”. “Temperature” is the temperature detected by the temperature sensor. The “radio wave level” is a radio wave level detected by a sensor that detects radio waves. The “radioactivity level” is a radioactivity level detected by a center that detects radioactivity. Note that T ₁ , F ₁ , R _{1 and the} like are arbitrary values. Thereby, it is possible to know the distribution state of the sensor information at the shooting position of each image.

  Here, when the user inputs an instruction to output an image taken in 2015 on the ABC bridge to the image management apparatus 1, the output unit 109 indicates that the value of “target ID” is “ABC bridge” and “image A record in which the first four digits of “ID” are “2015” is detected, an image indicated by “image ID” of the detected record is read from the image storage unit 101, and the read images are arranged in the order of shooting. Are arranged and displayed on the monitor 109a. Here, each image is reduced and displayed. Note that thumbnail images of images to be written may be created instead of reduced display, and corresponding thumbnail images may be displayed instead of each image.

  FIG. 11 is a diagram illustrating a display example of an image group composed of images taken in 2015 on the ABC bridge. In the figure, an image 111 is an image obtained by reducing and displaying one of the images constituting the image group. For example, when the user performs a predesignated operation such as clicking on the displayed image such as the image 111 via an input device such as a mouse (not shown), the designated image is displayed in an enlarged manner. The

  Next, when a predetermined operation for designating one or more images, for example, double-clicking one of the displayed images as shown in FIG. 11, the image designation receiving unit 107 designates the image. Accept. The corresponding image acquisition unit 108 acquires one or more image identifiers linked to the image that the image specification receiving unit 107 has received the specification from the link information management table illustrated in FIG. 6, and the output unit 109 acquires the acquired image The image indicated by each identifier is displayed together with the image specified above. For example, when the user visually recognizes an image having the image identifier “20150605000028”, an abnormality such as a crack is confirmed. When the user performs a predetermined operation as described above on the image, the corresponding image is displayed. The acquisition unit 108 detects a record having an attribute value that matches the image identifier “2015060600028” from the link information management table illustrated in FIG. 6, and an image identifier “20140602000101” that is another attribute value of the detected record. And “201306008000104” are acquired, and the images respectively indicated by the acquired image identifiers are acquired from the image storage unit 101. Then, the output unit 109 displays the image acquired by the corresponding image acquisition unit 108 on the monitor together with the designated image.

  FIG. 12 is a diagram illustrating a display example of an image by the output unit 109. In the figure, an image 121 is an image associated with the image identifier “2013060600028”, an image 122 is an image associated with the image identifier “20140602000101”, and an image 123 is an image associated with the image identifier “201306008000104”. It is.

  As shown in FIG. 12, it is easy to compare images by displaying images taken at different time zones side by side at the shooting positions and shooting directions corresponding to each other. It is possible to easily confirm whether or not the crack that has been present has existed before.

  Note that the output unit 109 receives an instruction to display an image taken in another year, for example, an instruction to display an image taken in 2014, while the display as shown in FIG. 11 is being performed. Then, the image associated with this year is read from the image storage unit 101, and the read images are arranged in the order in which they were photographed, and displayed on the monitor in place of the 2015 images described above.

  As described above, according to the present embodiment, a captured image can be appropriately managed by managing the captured image in association with the position related information regarding the position at the time of shooting. For example, according to the present embodiment, it is possible to easily acquire and output an image obtained by capturing substantially the same region in different time zones using position-related information, and a time-dependent one part of a subject Changes can be easily confirmed.

  In addition, according to the present embodiment, position information related to the position at the time of shooting can be acquired from two or more shot images. Therefore, the position cannot be specified by GPS or the like, or the position is accurate with GPS or the like. Position-related information can also be acquired for an image obtained by photographing a relatively small size structure or object that cannot be identified, or an image obtained by photographing an object extending in the height direction where position detection is difficult.

  In each of the above embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be.

  Further, although cases have been described with the above embodiments where the image management apparatus is a stand-alone, the image management apparatus may be a stand-alone apparatus or a server apparatus in a server / client system. In the latter case, the output unit or the reception unit receives an input or outputs a screen via a communication line.

  In each of the above embodiments, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component can be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. At the time of execution, the program execution unit may execute the program while accessing a storage unit (for example, a recording medium such as a hard disk or a memory).

  The software that realizes the image management apparatus in each of the above embodiments is the following program. In other words, this program is information associated with a plurality of image groups composed of a series of a plurality of images captured at different time zones, and one or more images constituting each image group, Designate one or more images in the image group stored in the image storage unit by using a computer accessible to the image storage unit in which position related information, which is information related to the position at the time of shooting for one or more images, is stored. An image designation accepting unit that accepts an image associated with position-related information to one or more images that the image designation accepting unit has accepted designation, and a designated image in the image group stored in the image storage unit This is a program for functioning as a corresponding image acquisition unit that is acquired from an image group different from the image group to be included, and an output unit that outputs an image acquired by the corresponding image acquisition unit.

  In the program, the functions realized by the program do not include functions that can be realized only by hardware. For example, a function that can be realized only by hardware such as a modem or an interface card in an acquisition unit that acquires information or an output unit that outputs information is not included in the function realized by the program.

  Further, the computer that executes this program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  FIG. 13 is a schematic diagram showing an example of the external appearance of a computer that executes the program and realizes the image management apparatus according to the embodiment. The above-described embodiment can be realized by computer hardware and a computer program executed on the computer hardware.

  In FIG. 13, the computer system 900 includes a computer 901 including a CD-ROM (Compact Disk Read Only Memory) drive 905, a keyboard 902, a mouse 903, and a monitor 904.

  FIG. 14 is a diagram showing an internal configuration of the computer system 900. In FIG. 14, in addition to the CD-ROM drive 905, a computer 901 is connected to an MPU (Micro Processing Unit) 911, a ROM 912 for storing a program such as a boot-up program, and the MPU 911. A RAM (Random Access Memory) 913 that temporarily stores and provides a temporary storage space, a hard disk 914 that stores application programs, system programs, and data, and a bus 915 that interconnects the MPU 911, ROM 912, and the like Prepare. The computer 901 may include a network card (not shown) that provides connection to the LAN.

  A program that causes the computer system 900 to execute the functions of the image management apparatus and the like according to the above-described embodiment may be stored in the CD-ROM 921, inserted into the CD-ROM drive 905, and transferred to the hard disk 914. Instead, the program may be transmitted to the computer 901 via a network (not shown) and stored in the hard disk 914. The program is loaded into the RAM 913 when executed. The program may be loaded directly from the CD-ROM 921 or the network.

  The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 901 to execute the functions of the image management apparatus according to the above-described embodiment. The program may include only a part of an instruction that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 900 operates is well known and will not be described in detail.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the image management device according to the present invention is suitable as a device for managing captured images, and is particularly useful as a device for managing a series of images taken in different time zones. .

DESCRIPTION OF SYMBOLS 1 Image management apparatus 2 Camera 101 Image storage part 102 Image acquisition part 103 Position related information acquisition part 104 Feature-value acquisition part 105 Accumulation part 106 Link setting part 107 Image designation reception part 108 Corresponding image acquisition part 109 Output part 200 Drone 1000 Image management system

Claims (12)

A plurality of image groups configured by a series of a plurality of images captured at different time zones, and information associated with one or more images constituting each image group, the one or more images An image storage unit storing position-related information that is information related to the position at the time of shooting,
An image designation accepting unit for accepting designation of one or more images in the image group stored in the image storage unit;
An image group including the designated image among the image groups stored in the image storage unit, and an image associated with the one or more images received by the image designation receiving unit by position related information; Is a corresponding image acquisition unit that acquires from different image groups;
An image management apparatus comprising: an output unit that outputs an image acquired by the corresponding image acquisition unit. The image management apparatus according to claim 1, wherein the position related information includes position information of a camera that has captured an image and information indicating a shooting direction. An image acquisition unit for acquiring a series of multiple images;
A position-related information acquisition unit that acquires position-related information for one or more images using two or more of a series of images acquired by the image acquisition unit;
An image group composed of a series of a plurality of images including the image from which the position related information acquisition unit has acquired the position related information, and the position related information acquired for each of one or more images constituting the image group. The image management apparatus according to claim 1, further comprising an accumulation unit that accumulates in the storage unit. The image management apparatus according to claim 3, wherein the position related information is information related to a relative position with respect to a position designated in advance. 5. The image acquisition unit according to claim 3, wherein the image acquisition unit acquires a plurality of series of images having the same shooting start position as a plurality of series of images acquired for one target. Image management device. The image acquisition unit acquires a series of a plurality of images by acquiring one frame image every one or more frame images from a plurality of continuous frame images constituting a moving image. The image management apparatus according to any one of claims 5 to 6. The image management device according to claim 6, wherein the image acquisition unit acquires a frame image one by one for every number of frame images corresponding to a moving speed of the camera when shooting a moving image. A link setting unit for linking images included in different image groups using position-related information;
The image management device according to claim 1, wherein the corresponding image acquisition unit acquires an image linked to each of the one or more images received by the image designation receiving unit. The image management apparatus according to claim 1, wherein one or more images constituting the image group are stored in an image storage unit in association with sensor information. The image management apparatus according to claim 1, wherein one or more images constituting the image group are stored in an image storage unit in association with a feature amount acquired from each image. A plurality of image groups configured by a series of a plurality of images captured at different time zones, and information associated with one or more images constituting each image group, the one or more images An image management method that is performed using an image storage unit in which position-related information that is information about a position at the time of shooting is stored, an image designation receiving unit, a corresponding image acquisition unit, and an output unit,
An image designation accepting step in which the image designation accepting unit accepts designation of one or more images of the image group stored in the image storing unit;
The corresponding image acquisition unit is configured to specify an image associated with the one or more images received in the image specification receiving step by the position related information, in the image group stored in the image storage unit. A corresponding image acquisition step of acquiring from an image group different from the image group including the image;
An image management method comprising: an output step in which the output unit outputs the image acquired in the corresponding image acquisition step. A plurality of image groups configured by a series of a plurality of images captured at different time zones, and information associated with one or more images constituting each image group, the one or more images A computer that can access an image storage unit in which position-related information that is information about the position at the time of shooting is stored,
An image designation accepting unit for accepting designation of one or more images in the image group stored in the image storage unit;
An image group including the designated image among the image groups stored in the image storage unit, wherein the image associated with the one or more images received by the image designation receiving unit is associated with the position-related information. A corresponding image acquisition unit for acquiring from different image groups;
A program for functioning as an output unit that outputs an image acquired by the corresponding image acquisition unit.

Images