JP2017102672A - Geographic position information specification system and geographic position information specification method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology for narrowing down images for collation before performing image collation processing and specifying the position of an inspection object captured in an object image accurately in a short time.SOLUTION: A geographic position information specification system comprises: a first database that includes images for collation for an object image capturing an inspection object, information on the positions of the images for collation, and information on the attributes of the images for collation; a searching part that searches for the images for collation in the first database by using a narrowing condition being the information on the attribute of the inspection object; and a similar image specification part that compares the object image with the searched image for collation and extracts, from the first database, an image for collation similar to the object image and the information on the position of the image for collation.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a geolocation information identification system and a geolocation information identification method.

  In recent years, many of the social infrastructure facilities such as roads, bridges, and tunnels that were established during the high growth period in the 1960s have exceeded the 50-year life expectancy, and maintenance and renewal work required for aging has increased. doing. Currently, in many operations, skilled workers are performing visual checks on site. For example, regarding transportation infrastructure facilities, workers must visually check the cracks on painted surfaces of road structures and bridges, exposure of rebars, wear on railway tracks, abnormalities in signals and overhead lines, etc., and damages and repairs are necessary. I have found a special place. In the visual check at the site, there are problems that it takes time for workers to move when the target equipment is scattered, and that there is danger when workers perform inspections from the railway line etc. . As the number of target facilities is large and it is difficult to secure financial resources and human resources, there is a high demand for operational improvements that reduce local work.

  In recent years, the use of image data taken by unmanned aerial vehicles (UAVs) and surveying systems using mobile measurement vehicles (Mobile Mapping Systems, hereinafter MMSs) is promising as an improvement initiative. For example, for solar panels installed with tens of thousands of panels, information such as temperature is automatically calculated by analyzing image data taken by UAV, and compared with normal values to detect abnormalities. Alternative methods for inspection have been proposed. In addition, by verifying the state of highway bridges using image data taken by UAV, it is verified that the work will be confirmed visually by a worker on a dedicated vehicle after performing conventional lane control. Has been. Such a facility inspection method using image data that reduces work on site can reduce work costs such as labor, time, and cost. In addition, there is an advantage that the safety of workers can be ensured, and the utility value is high.

  In the inspection method using the image data as described above, it is necessary to specify the location of the photographed equipment (hereinafter referred to as an inspection object). For that purpose, the geographical position information (hereinafter referred to as geographical position information) of the photographing point is acquired from the GPS function attached to the photographing device, or the moving distance is determined from the value detected by the acceleration sensor of the moving body equipped with the device. In many cases, the geographical position information of the shooting point is acquired by deriving the. For example, Patent Document 1 discloses a system for managing facility position information using the GPS function of a portable information terminal.

  However, depending on the situation, such as when shooting in places where tunnels or under elevated roads cannot receive radio waves and the GPS function cannot be used, or when using a shooting device that does not have a GPS / acceleration sensor, the geolocation information of the shooting location may vary. May not be able to get. Further, even if the geographical position information of the photographing point can be acquired, there is a possibility that the inspection object is specified at a position different from the place where the inspection object is actually located.

  As a method for identifying the geographical position information of the inspection object from such image data whose exact photographing point is unknown, the image data (target image) and a large number of image data whose geographical position information prepared in advance are known ( There is a method of calculating the similarity between the image and the matching image. For example, when the similarity between the target image and a certain image for comparison is high, it is determined that the same inspection object is captured, and the geographical position information of the image for comparison is acquired as the position of the inspection object.

JP2013-089211A

  In the inspection work of social infrastructure equipment using image data, a method of identifying the geographical position information of the inspection object existing in the image data is desired, but in order to put this method into practical use and introduce it to work, Even when there are a large number of facilities to be inspected, it is required to acquire the geographical position information of the facilities accurately and in a relatively short time as much as or more than conventional on-site inspection work.

  However, in the method of specifying the geolocation information by the image collation described above, if there are a large number of collation images, it takes a lot of processing time to calculate and determine the similarity. Further, as the number of pieces of collation image data increases, there is a possibility that the number of cases in which the inspection objects are actually different, but the degree of similarity is high, so that they are erroneously determined to be the same.

  Therefore, the present invention provides a technique for performing the image matching process after narrowing down the images for matching, and specifying the position of the inspection object in the target image in a short time and with high accuracy.

  For example, in order to solve the above-mentioned problem, the configuration described in the claims is adopted. The present application includes a plurality of means for solving the above-described problems. To give an example, a collation image with respect to a target image in which an inspection target is shown, position information of the collation image, and the collation image A first database that includes attribute information; a search unit that searches for the image for verification from the first database using a filtering condition that is attribute information of the inspection object; and the target image and the verification that has been searched A geo-location information specifying system comprising: a comparison image similar to the target image; and a similar image specifying unit that extracts the position information of the similar comparison image from the first database. Provided.

  According to another example, there is provided a geolocation information specifying method executed in an information processing system including a processor and a storage unit. The storage unit stores a first database that includes a collation image with respect to a target image in which an inspection object is captured, position information of the collation image, and attribute information of the collation image. In the geolocation information specifying method, the processor searches the first database for the matching image using a narrowing condition that is attribute information of the inspection object, and the processor Comparing the searched matching image and extracting the matching image similar to the target image and the position information of the similar matching image from the first database.

  According to the present invention, it is possible to specify the position of the inspection object in the target image in a short time and with high accuracy by performing the image matching process after narrowing down the matching image. Further features related to the present invention will become apparent from the description of the present specification and the accompanying drawings. Further, problems, configurations and effects other than those described above will be clarified by the description of the following examples.

1 is a system configuration diagram showing an embodiment of the present invention. It is an example of a structure of the image table for typical collation. It is an example of the equipment registered in the typical collation image table. It is an example of a structure of an equipment position information table. It is an example of distribution of the positional information stored in the equipment positional information table. It is an example of the image stored in the image database for collation. It is an example of a structure of the image information table for collation. FIG. 8 is an example in which position information registered in the collation image information table in FIG. 7 is pointed at a target area. It is an example of a structure of the image feature point table for collation. It is an example of the flowchart of the geolocation information specific process of the test object which concerns on this invention. It is an example of the display screen of the estimation result of the attribute information regarding a test subject. It is the image figure which narrowed down the space search object using the equipment position information table. It is an example of the process which extracts a similar image using a feature-value. It is an example of a display of the image data for collation of the installation estimated that it is the same as the test target object of a target image.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. The accompanying drawings illustrate specific embodiments consistent with the principles of the invention, but are for the purpose of understanding the invention and are not to be construed as limiting the invention in any way. . Moreover, the same reference number may be attached | subjected about the common structure in each figure.

  The following embodiments relate to a system for identifying the geographical position information of an inspection object whose geographical position information is unknown in image data.

  As a result of examining the above problems, the inventor of the present application described the following method for collating image data (target image) whose shooting location is unknown and image data (collation image) whose geographical position information is prepared in advance. I came to know.

  In the method of collating the target image with the collation image, it is useful not to calculate the similarity between all the collation images and the target image but to narrow down the collation image to some extent in advance. As a condition for narrowing down the matching image, it may be determined whether the attribute information attached to the matching image matches the attribute information of the target image. For example, when the target image is image data obtained by photographing facilities related to traffic infrastructure, inspection objects are often distributed along roads and railway lines. Therefore, by reducing the number of image data for verification under the condition that the image data is taken at a location close to equipment such as roads and railway lines, or where the equipment specifications of the inspection object are distributed, image similarity is reduced. Compared with specifying geolocation information only by the degree, the false detection rate is lower. Further, the processing time required for such spatial search and attribute search is shorter than the processing time required for determination using only the conventional similarity. In the following, a technique for narrowing down the image for verification on the condition of the location of a possible facility, the shooting date and time, etc. will be described.

  FIG. 1 is a system configuration diagram showing an example of an embodiment of the present invention. The geographic location information identification system is configured by an information processing system such as a computer, and includes, for example, a geographic location information identification server 100 and a user terminal 120.

  The geolocation information specifying server 100 includes a storage unit 101, a communication unit 102, a CPU (processor) 103, a memory 104, and databases 110 to 114.

  The storage unit 101 is a large-capacity nonvolatile storage device such as a magnetic storage device or a flash memory. The storage unit 101 stores a program for each module that provides the functions of the present invention. Specifically, the storage unit 101 stores an equipment type estimation unit 105, a position estimation unit 106, a feature amount calculation unit 107, a similar image specification unit 108, and a space search unit 109.

  The CPU 103 reads the program of each module into the memory 104 as necessary, and executes various processes. Therefore, in the following description, when the processing content is described with the module stored in the storage unit 101 of FIG. 1 as the subject, the description may be made with the processor as the subject. The communication unit 102 communicates with other devices via the Internet or the like.

  The program executed by the CPU 103 is provided to the geolocation information specifying server 100 via a non-volatile storage medium or a network. For this reason, the geolocation information specifying server 100 may include an interface for reading a storage medium (CD-ROM, flash memory, etc.).

  The equipment type estimation unit 105 estimates the equipment type and equipment standards of the equipment to be inspected in the target image data based on the representative collation image.

  The position estimation unit 106 specifies a region where a space search is performed based on the equipment type and equipment standard of the equipment to be inspected.

  The feature amount calculation unit 107 extracts feature points from the image and calculates feature amounts. Here, the feature point is a point where it is easy to take correspondence between image data. As an example, the feature point is a place where a change in shading is large in the image data. In addition, you may use the point which can be extracted with another well-known technique as a feature point. The feature amount is data representing the feature on the image of the area calculated from the area around the feature point. Here, the feature amount is represented by a vector amount derived from pixel values in a region around the feature point. A known technique can be used to calculate the feature amount.

  The similar image specifying unit 108 determines whether or not the inspection objects appearing in the two images are the same (similar) using the feature amount. The similar image specifying unit 108 detects corresponding points by comparing feature amounts between two pieces of image data, and determines the similarity of image data based on the number of corresponding points.

  The space search unit 109 specifies an image ID of a collation image having position information in the position information 704 in the collation image information table 700 that includes position information that intersects with the area where the space search specified by the position estimation unit 106 is performed.

  The user terminal 120 connected to the geolocation information specifying server 100 via a network is configured by a general computer, and includes an input unit 121, a display unit 122, a communication unit 123, a CPU 124, a memory 125, and a storage unit. 126.

  The input unit 121 is a user interface such as a keyboard and a mouse that receives input from the user. The display unit 122 is a user interface such as a display. The communication unit 123 communicates with other devices via the Internet or the like.

  The storage unit 126 is a large-capacity and nonvolatile storage device such as a magnetic storage device or a flash memory. The storage unit 126 stores various programs for exchanging with the geolocation information specifying server 100. The CPU 124 reads each program into the memory 125 as necessary, and executes various processes.

  Next, the databases 110 to 114 will be described. The databases 110 to 114 may be stored in the storage unit 101 or may be stored in a storage device outside the geolocation information specifying server 100.

  In the present embodiment, the information used by the system does not depend on the data structure and may be expressed in any data structure. In the following, information in a table format is shown as an example. For example, information can be stored using a data structure appropriately selected from a table, a list, a database, or a queue.

  The representative collation image database 110 stores the attribute information and the feature amount of the representative collation image symbolizing the facility for the equipment to be inspected in the field and business where the present invention is used. The equipment position information database 111 stores attribute information and geographical position information of equipment that can be inspected. The verification image database 112 stores various verification images obtained by photographing facilities that can be inspected. The collation image information database 113 stores the shooting date and time of each collation image registered in the collation image database 112 and the captured geolocation information. The matching image feature point database 114 stores the feature point information of the equipment that appears in each matching image registered in the matching image database 112. In addition, the information regarding the image for collation may be managed by the some databases 112-114 like this example, and may be integrated and managed in one database.

  FIG. 2 is an example of the representative collation image table 200 stored in the representative collation image database 110. In the representative collation image table 200, the feature quantity in the representative image (representative collation image) prepared for each field and field of use of the present invention and the type of equipment to be inspected in the business and the standard are registered.

  The representative collation image table 200 includes, as columns, a classification code 201 for uniquely identifying a record, an equipment type 202, an equipment standard 203, and a feature amount 204 of the equipment in the representative image of the target equipment. In this example, the representative collation image database 110 stores the feature amount of the representative collation image. However, the present invention is not limited to this, and the representative collation image itself may be stored. In this case, the feature amount may be calculated from the representative collation image each time.

  FIG. 3 is an example of equipment to be inspected. As illustrated in FIG. 3, when the facility type 202 is a power transmission tower, there are four types of standards for the power transmission tower. Since the shape of the transmission tower is different for each standard, the feature amount 204 is different. Therefore, in the representative collation image table 200, the characteristic amount of the transmission tower with the standard A, the characteristic amount of the transmission tower with the standard B, the characteristic amount of the transmission tower with the standard C, and the characteristic amount of the transmission tower with the standard D Are registered respectively. Similarly, for other types of equipment, feature quantities are registered in the representative collation image table 200 for each standard.

  FIG. 4 is an example of the equipment location information table 400 stored in the equipment location information database 111. The equipment position information table 400 includes a column of a reference code 401 for uniquely identifying a record, an inspection target name 402, an equipment type 403, an equipment standard 404, inspection target position information 405, and a format 406. Include as.

  In the equipment position information table 400, the inspection objects existing in the inspection target area are grouped by equipment type and equipment standard. In the position information 405, position information of all inspection objects belonging to each group is listed.

  The format 406 stores information on the shape of equipment distribution according to the characteristics of the equipment type. For example, when a certain area such as a distribution area of power transmission towers is designated, “area” is stored in the format 406. In addition, in the case of equipment connected in a line shape such as distribution of electric wires, trains, and road lines, “line” is stored in format 406, and “point” is stored in format 406 when facilities of a certain standard are scattered. Is done.

  By classifying in this way, when the format 406 is “region”, the space search unit 109 executes a space search using the region specified by the position information 405 and its periphery as a search target space. Further, when the format 406 is “line”, the space search unit 109 executes a space search using the line including the position information 405 and its surroundings as a search target space. In addition, when the format 406 is “point”, the space search unit 109 performs a space search using the vicinity of the point of the position information 405 as a search target.

  FIG. 5 shows a distribution example 500 of facility positions stored in the facility position information database 111. White circles in the distribution example 500 indicate the distribution of standard A power transmission towers.

  FIG. 6 is an example of an image stored in the collation image database 112. In the verification image database 112, a large number of verification images related to the inspection target existing in the inspection target area are registered. Although not shown in FIG. 6, each verification image is stored in the verification image database 112 in association with the image ID.

  FIG. 7 is an example of a collation image information table 700 stored in the collation image information database 113. The verification image information table 700 stores attribute information for each verification image registered in the verification image database 112. The collation image information table 700 includes, as columns, an image ID 701 that uniquely identifies an image, a shooting date 702 when the image was shot, a shooting time 703, and position information 704.

  FIG. 8 shows an example 800 in which the information in the matching image information table 700 shown in FIG. 7 is pointed within the target area.

  FIG. 9 is an example of a matching image feature point table 900 stored in the matching image feature point database 114. The matching image feature point table 900 includes, as columns, an image ID 901 that uniquely identifies an image and a feature quantity 902 derived from the matching image.

  The image ID 901 corresponds to the image ID of the collation image database 112 and the image ID 701 of the collation image information table 700. The feature quantity 902 is a vector quantity derived from pixel values in the area around the feature point for each feature point in the matching image. The number of feature points varies from image to image. An image 903 is an example of a feature point of a matching image of a certain power transmission tower.

  FIG. 10 is an example of a flowchart of the geolocation information specifying process of the inspection object according to the present invention.

  First, the user terminal 120 accepts selection of an image (hereinafter referred to as a target image) on which an inspection target whose geolocation information is to be estimated by a user is captured (S1001). The communication unit 123 of the user terminal 120 transmits the target image data to the geolocation information specifying server 100 (S1002).

  The communication unit 102 of the geolocation information specifying server 100 receives the target image from the user terminal 120 (S1003). The feature amount calculation unit 107 extracts feature points from the received target image, and calculates feature amounts for each feature point (S1004).

  The similar image specifying unit 108 calculates the similarity by comparing the calculated feature amount with the feature amount 204 of the representative collation image registered in the representative collation image table 200. The similar image specifying unit 108 extracts, for example, a record having the highest calculated similarity from the representative collation image table 200. The equipment type estimation unit 105 acquires the equipment type 202 and equipment standard 203 of the extracted record (S1005). The equipment type estimation unit 105 outputs the equipment type 202 and equipment standard 203 information of the extracted record as an estimation result of the equipment type and equipment standard of the inspection object shown in the received target image. The equipment type estimation unit 105 has a similarity between the calculated feature quantity and the feature quantity of a certain equipment type higher than a predetermined value, but the specific type in the representative collation image table 200 can be specified as the equipment standard can be specified. When the degree of similarity with the record is not high, only the type of equipment may be output, and the fact that the equipment standard could not be specified may be output.

  The communication unit 102 transmits the estimation result (equipment type and equipment standard) related to the inspection object shown in the target image to the user terminal 120 (S1006). The communication unit 123 of the user terminal 120 receives the estimation result regarding the inspection target (S1007), and displays the estimation result on the display unit 122 (S1008).

  FIG. 11 is an example of a display screen of the estimation result related to the inspection target in the user terminal 120. On the screen, the target image selected by the user and the location condition of the facility are displayed as the facility estimation result. The screen 1101 includes a target image setting unit 1102, a target image display unit 1103, an estimation result display unit 1104, and a condition setting unit 1105.

  The target image setting unit 1102 is an area where a file path of the target image is displayed, and the user can select the target image using, for example, a reference button. The target image display unit 1103 is an area where the target image is displayed. The estimation result display unit 1104 is an area in which an estimation result (equipment type and equipment standard) related to the inspection object is displayed. In the facility type estimation unit 105, the facility type can be specified. However, if the facility standard cannot be specified, the facility type may be displayed in the facility standard column so that it can be understood. The condition setting unit 1105 is an area for setting additional conditions for the narrowing-down conditions. In the example of FIG. 11, the shooting date and time of the verification image can be set as an additional condition for narrowing down. The user terminal 120 may transmit an additional condition to the geolocation information specifying server 100 in the process of transmitting a search continuation instruction. Thereafter, the geo-location information specifying server 100 may execute a search process for narrowing down the matching images according to the specified additional conditions.

  In addition, the screen 1101 may include an interface for correcting the contents of the estimation result display unit 1104. For example, when the results (equipment type, equipment standard) estimated by the geolocation information specifying server 100 are different, or the equipment type and equipment standard cannot be estimated by the geolocation information specifying server 100, the user However, the correct information may be selected from a list of equipment types and equipment standards. The user terminal 120 may transmit the result to the geo-location information specifying server 100, and the geo-location information specifying server 100 may continue the process based on the content corrected by the user. According to the above configuration, even if there is an error in the estimation result of the geolocation information specifying server 100 or no estimation result is obtained, the processing can be continued by correcting the attribute information to correct attribute information. Can be raised.

  The user confirms the content of the estimation result display unit 1104 on the screen 1101 and, if there is no problem, presses the search continuation button 1106 to confirm the content of the estimation result display unit 1104 as a narrowing condition. When the user terminal 120 receives information on pressing of the search continuation button 1106 (S1009), the communication unit 123 transmits a search continuation instruction to the geolocation information specifying server 100 (S1010).

  Note that the user presses a cancel button 1107 when interrupting the process. In this case, the communication unit 123 of the user terminal 120 transmits a process interruption instruction to the geolocation information identification server 100, and the process of the geolocation information identification server 100 ends.

  The communication unit 102 of the geolocation information specifying server 100 receives an instruction from the user terminal 120 (S1011). When the geolocation information specifying server 100 receives an instruction to continue the search, the position estimation unit 106 determines the equipment type, the equipment standard, the equipment type 403, and the equipment standard estimated by the equipment type estimation unit 105 from the equipment position information table 400. The record that 404 matches is specified, and the position information 405 and the format 406 of the record are acquired (S1012). For example, when the type of equipment estimated by the equipment type estimation unit 105 is “power transmission tower” and the standard of the equipment is “A”, the position estimation unit 106 uses the position information 405 of the record whose reference code 401 is “3”. And the format 406 is acquired.

  FIG. 12 is an image diagram in which the space search target is narrowed down using the equipment position information table 400. A plot diagram 1201 is a diagram in which the positions of the subjects of all the images for comparison before narrowing down the space search target are plotted. The plot diagram 1202 is a diagram in which the position information 405 acquired in S1012 is plotted when the type of equipment estimated by the equipment type estimation unit 105 is “power transmission tower” and the standard of equipment is “A”. As described above, according to the present embodiment, it is possible to narrow down the matching image by limiting the search target space using the attribute information.

  As a result of the estimation by the equipment type estimation unit 105, the type of equipment is “power transmission tower”. However, when the equipment standard cannot be estimated, the position estimation unit 106 records the record with the reference code “1”, That is, the position information 405 and the format 406 of the record in which the equipment type 403 is “power transmission tower” and the equipment standard is “All” are acquired.

  The space search unit 109 searches the matching image information table 700 based on the acquired position information 405 and data of the format 406, and specifies a matching image (S1013). Specifically, when the format 406 is “point”, the space search unit 109 sets a space around the coordinates set in the position information 405 as a target of space search, and enters the space from the matching image information table 700 in the space. The image ID 701 of the record having the existing position information 704 is acquired.

  For example, when the equipment type is “transmission line” and the equipment standard is “A”, the format 406 is “line”. Therefore, the space search unit 109 records, from the collation image information table 700, a record having position information 704 that exists in the space including the line connecting the position specified by the position information 405 acquired in S1012 and its surroundings. An image ID 701 is acquired.

  As another example, when the equipment type is “power transmission tower” and the equipment standard is “All”, the format 406 is “area”. Therefore, the space search unit 109 stores a record having position information 704 that exists in the space (for example, the area surrounded by the coordinates) specified by the position information 405 acquired in S1012 from the matching image information table 700. An image ID 701 is acquired. According to the above configuration, the position information of the inspection object can be estimated more effectively and accurately. The reason why the space search includes the periphery of the content specified in the position information 405 is to cope with a case where the facilities are moved a little due to the addition of roads or railway lines. Further, there is some error in the position information, and this error is allowed.

  In S1013, the space search unit 109 may further narrow down the acquired image IDs 701 by using the shooting date / time information specified in the condition setting unit 1105 of the screen 1101.

  The feature amount calculation unit 107 acquires the feature amount of the matching image specified by the image ID 701 acquired in S1013 from the feature amount 902 of the matching image feature point table 900. The similar image specifying unit 108 calculates the similarity by comparing the feature amount of the target image with the acquired feature amount 902. If the calculated similarity is higher than a predetermined threshold, the similar image specifying unit 108 estimates that the matching image is the same as the inspection object shown in the target image, and uses the matching image as the matching image database 112. (S1014). In addition, the similar image specifying unit 108 extracts the shooting date 702, the shooting time 703, and the position information 704 corresponding to the extracted matching image from the matching image information table 700.

  FIG. 13 is an example of processing for extracting a similar image using a feature amount in S1014. When the target image 1301 and the comparison image 1302 are compared, there are many corresponding feature points between the two images. For this reason, the similar image specifying unit 108 estimates that the inspection object of the target image 1301 and the equipment shown in the verification image 1302 are the same. On the other hand, when the target image 1303 and the comparison image 1304 are compared, there are few corresponding feature points between the two images. For this reason, the similar image specifying unit 108 estimates that the inspection object of the target image 1303 and the equipment shown in the verification image 1304 are different. The degree of corresponding feature points to be estimated to be the same is arbitrary, but since it is after the matching image has been narrowed down by equipment type and equipment standard, the similarity determination condition is higher than the first S1005. May be set strictly. In the method for determining the similarity of image data based on local feature points used here, the similarity can be determined even if there is a difference in scale, rotation, deformation, or color tone between the image data. Therefore, there are differences in the distance between the inspection object and the imaging device, the imaging angle of the image data, the solar altitude, the weather, etc. It doesn't matter.

  The communication unit 102 transmits the verification image extracted in S1014 to the user terminal 120 (S1015). At this time, the communication unit 102 also transmits information on the shooting date 702, the shooting time 703, and the position information 704 corresponding to the extracted comparison image to the user terminal 120. The communication unit 123 of the user terminal 120 receives an image for verification, attribute information, geographical position information, and the like (S1016). The display unit 122 displays the received verification image, attribute information, geo-location information, and the like on the screen as candidate equipment that is estimated to be an inspection object in the target image (S1017).

  FIG. 14 is an example of a screen displayed on the user terminal 120. The screen 1401 includes a target image setting unit 1402, a target image display unit 1403, and a candidate image display unit 1404. The target image setting unit 1402 and the target image display unit 1403 are the same as the target image setting unit 1102 and the target image display unit 1103 in FIG.

  The candidate image display unit 1404 is an area for displaying a candidate image 1405 that is estimated to contain the same inspection target object as the target image. The candidate image display unit 1404 also displays information 1406 (here, shooting date and position information) of each candidate image. Note that the plurality of candidate images displayed on the candidate image display unit 1404 may be displayed, for example, in descending order of similarity calculated on the geolocation information specifying server 100 side. The candidate image display unit 1404 includes a check button 1407. Using the check button 1407, the user can select an image (desired image) in which the same inspection object as the target image is shown from among a plurality of candidate images. The user selects an image using the check button 1407 and then presses the enter button 1408. Thereby, the information (for example, position information, equipment information, etc.) of the selected image can be registered in the system or application that uses the information as the information on the inspection object in the target image. If the user presses the cancel button 1409, the position information, facility information, etc. of the inspection object shown in the target image are not fixed, and the process ends.

  Note that a target image for which position information has been specified may be newly registered as a matching image. For example, when the user terminal 120 presses the decision button 1408, the user terminal 120 may transmit information indicating which matching image has been selected to the geolocation information specifying server 100. The geolocation information specifying server 100 may include a registration unit that receives the information and newly registers the target image as a matching image. The registration unit registers new data in the databases 112 to 114 that associates the target image with information on the selected image for comparison (for example, geolocation information, attribute information (equipment type, standard), etc.). The inspection object may change over time (eg, degradation, dirt, etc.), and the specific accuracy of similar images may decrease over time. According to the above configuration, by newly registering the latest target image as a matching image, the accuracy of specifying a similar image is not reduced, and as a result, the accuracy in specifying position information is maintained. Can do.

  As described above, the geolocation information specifying system of the present embodiment includes the host system (geolocation information specifying server 100) and the host terminal (user terminal 120) connected to the host system via the network. The host system includes a representative collation image database 110 that stores feature points of representative collation image data that symbolizes equipment types and equipment of a specific standard, and attribute information such as target equipment types and equipment standards. In the equipment position information database 111 storing the geographical position information related to the distribution of equipment, the collation image database 112 storing the image data obtained by photographing the inspection objects existing in various places, and the collation image data Existence exists in the collation image data database 113 and the collation image information database 113 including the attribute information such as the image data ID for uniquely identifying the geographical position information of the inspection object and the collation image data and the photographing date and time. And a matching image feature point database 114 storing feature point information of the inspection object. The host system has an attribute information estimation unit (equipment type estimation unit 105) that estimates the attribute information (type, standard, etc.) of equipment that is a narrowing condition for the target image specified by the host terminal, and the determined narrowing A position estimation unit (position estimation unit 106) that acquires geographical position information related to the conditions, and for the verification of the verification image database 112 according to the conditions of the geographical position information acquired from the position estimation unit and the shooting date and time input from the host terminal A search unit (spatial search unit 109) for searching for an image, a feature amount calculation unit (feature amount calculation unit 107) for extracting feature points of image data and calculating a feature amount, and collating with a target image using the feature amount A similar image specifying unit (similar image specifying unit 108) for calculating the similarity between the image and the image and determining the similarity. The host terminal displays the narrow-down condition automatically derived by the narrow-down condition specifying processing unit, the candidate image data determined to be similar to the target image, the position information of the candidate image, and the like.

The following effects can be expected by the system for specifying the geographical position information of the inspection object in the present embodiment.
(1) Geographical position information can be specified for an inspection object for which accurate geographical position information is unknown that exists in the image data.
(2) Compared with the conventional method by extracting image data with a high degree of similarity after narrowing down the image data for verification from the external information such as the target equipment and the location information of the equipment distributed in relation to the target equipment It is possible to specify the geographical position information of the inspection object existing in the image data with high accuracy and in a short time.

  By applying this example to image data taken by devices installed in UAV or MMS, for example, the position information of image data of the shooting device can be obtained with higher accuracy and in real time than the GPS function or acceleration sensor. It can be acquired in a short time as close to. According to this application example, it is possible to contribute to the practical use of UAV and MMS autonomous driving and remote driving operation. As the use of UAVs and autonomous vehicles is expected to expand in the future, the scope of application of this technology is wide. It can also be applied to the identification of position information of image data taken by an investigation organization such as the police.

  In the above example, as the conditions for narrowing down the inspection object, various types of information such as the type of equipment, equipment standard, equipment geo-location information, date of image for verification, time of image for verification are used. It is not limited to this. The narrowing-down condition for the verification image may be any attribute information for specifying the inspection object. In addition to the above example, attribute information such as a facility management organization, a name of a route of an infrastructure (railway, road, etc.), and a movement record of an imaging device can be used as a narrowing condition.

  In the above example, the facility location information database 111 stores the location information of each facility, but is not limited to such a format. The facility location information database 111 may store any information as long as the location of each facility can be limited. For example, it is assumed that the inspection object is a facility related to transportation infrastructure. In this case, the facility location information database 111 may store, for example, location information of the traffic infrastructure (roads or railway lines) instead of storing location information of each facility itself. In the case of facilities related to traffic infrastructure, inspection objects are distributed along roads and railway lines, so that the verification image can be narrowed down from the position information of the traffic infrastructure. In this case, the position estimation unit 106 acquires line-shaped position information indicating a road or a railway line. Then, the space search unit 109 may set a line-shaped road or railroad line and its surroundings as targets for space search, and extract a matching image from the matching image database 112 within the section.

  The present invention is not limited to the above-described embodiments, and includes various modifications. The above embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Also, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment. Moreover, the structure of another Example can also be added to the structure of a certain Example. Further, with respect to a part of the configuration of each embodiment, another configuration can be added, deleted, or replaced.

  Each of the above-described configurations, functions, and the like may be realized by software by the processor interpreting and executing a program that realizes each function. Information such as programs, tables, and files that realize each function can be stored in a recording device such as a memory, a hard disk, or an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD. Further, each of the above-described configurations, functions, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit.

  In the above-described embodiments, the control lines and information lines are those that are considered necessary for explanation, and not all the control lines and information lines on the product are necessarily shown. All the components may be connected to each other.

DESCRIPTION OF SYMBOLS 100 ... Geolocation information specific server 101 ... Storage part 102 ... Communication part 103 ... CPU
104 ... Memory 105 ... Equipment type estimation unit 106 ... Position estimation unit 107 ... Feature quantity calculation unit 108 ... Similar image specifying unit 109 ... Space search unit 110 ... Representative collation image database (second database)
111 ... Equipment location information database (third database)
112 ... Image database for verification (first database)
113 ... Image information database for verification (first database)
114 ... Image feature point database for verification (first database)
120 ... User terminal 121 ... Input unit 122 ... Display unit 123 ... Communication unit 124 ... CPU
125 ... Memory 126 ... Storage unit

Claims (11)

A first database including a collation image with respect to a target image in which an inspection object is captured, position information of the collation image, and attribute information of the collation image;
A search unit that searches for the image for matching from the first database using a narrowing condition that is attribute information of the inspection object;
A similar image specifying unit that compares the target image with the searched matching image, and extracts the matching image similar to the target image and the position information of the similar matching image from the first database; ,
Geolocation information identification system comprising: The geolocation information identification system according to claim 1,
A second database including attribute information of a representative collation image that is a representative image of the inspection object;
An attribute information estimation unit that acquires the attribute information of the representative collation image similar to the target image;
Further comprising
The search unit uses the attribute information of the representative collation image acquired by the attribute information estimation unit as the narrowing-down condition. In the geolocation information specifying system according to claim 2,
A display unit for displaying a screen for displaying the attribute information of the representative collation image acquired by the attribute information estimation unit as the narrowing-down condition;
The search unit executes a search process after receiving an input for determining the narrow-down condition for the screen. In the geolocation information specifying system according to claim 3,
The geolocation information specifying system, wherein the screen includes an interface for correcting the narrowing-down condition. The geolocation information identification system according to claim 1,
A third database including attribute information of the inspection object and position information of the inspection object;
A position estimation unit that acquires the position information of the inspection object from the third database using the narrowing-down condition;
Further comprising
The search unit uses the position information of the inspection object acquired by the position estimation unit as the narrowing-down condition. The geolocation information identification system according to claim 5,
The third database further includes information on the shape of the distribution of the inspection object,
The geolocation information specifying system, wherein the search unit searches the first database for the collation image existing in a space specified by the position information and the shape-related information of the inspection object. The geolocation information identification system according to claim 5,
The geolocation information specifying system, wherein the search unit further uses peripheral information of the position information of the inspection object acquired by the position estimation unit as the narrowing-down condition. The geolocation information identification system according to claim 1,
The geographical position information specifying system further comprising a display unit for displaying a screen for displaying the target image, the similar matching image, and the position information of the similar matching image. The geolocation information identification system according to claim 8,
The geolocation information specifying system according to claim 1, wherein the screen includes an interface capable of selecting a desired matching image from the similar matching images. The geolocation information identification system according to claim 9,
The image processing apparatus further includes a registration unit that associates and registers the target image, the position information of the selected collation image, and the attribute information of the selected collation image in the first database. Geolocation information identification system. A geolocation information specifying method executed in an information processing system including a processor and a storage unit, wherein the storage unit is a collation image with respect to a target image in which an inspection object is captured, position information of the collation image, Storing a first database including attribute information of the matching image;
The processor searches the image for verification from the first database using a narrowing condition that is attribute information of the inspection object;
The processor compares the target image with the searched matching image, and extracts from the first database the matching image similar to the target image and the position information of the similar matching image. When,
Geolocation information identification method including

Images