JP2016206825A - Image registration device and method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently associate a photographic image, a photography place and a photography object.SOLUTION: An image registration device 100 comprises: a template database 107 for storing template images in which inspection object appliances are photographed, in advance along a first inspection route of the inspection object appliances; an image input part 101 for inputting inspection images in which the inspection object appliances are photographed, and storing the inspection images in an inspection image database in an order of photography time of the inspection images; a similarity calculation part 102 for calculating similarity between the template images and the inspection images; a required time difference evaluation part 103 for calculating a time difference between a time difference between the photography times of different inspection images, and a predetermined time required for moving between the inspection object appliances along the first inspection route by an operator; and a template image selection part 104 for, based on the time difference and the similarity, estimating a second inspection route of the inspection object appliances where the inspection images are photographed, and based on the second inspection route, selecting the template image corresponding to the inspection image from the template database.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image registration apparatus and method for registering an image taken by an inspection operator in an operation of inspecting equipment such as a plant facility.

  In the inspection of equipment in the plant facility, an inspection operator patrols the equipment installed in the plant facility one by one to check the state of the equipment. In addition, in order to leave an inspection record of equipment, an inspection operator takes an image of the equipment to be inspected as an object to be photographed using a camera.

  In Patent Document 1, as a method for preventing mistakes during periodic outdoor inspection, the position information of an inspection worker measured by GPS is within a predetermined range of the position information of an inspection object recorded in advance in an information terminal. A method of confirming whether or not is disclosed.

  Patent Document 2 describes a method of starting a vehicle when a vehicle equipped with a position detection unit such as GPS is driven and the GPS positioning result reaches a predetermined inspection position, or an image captured at a predetermined inspection location. And a method of comparing the images taken in advance at the inspection place, correcting the position detection result by GPS, and checking the inspection position more accurately.

JP 2007-280282 A JP 2000-32436 A

  In order to utilize an image obtained by photographing an inspection target device that is a photographing target, it is necessary to sort the images. Here, the sorting operation is an operation for specifying which device has captured the captured image at which location in the plant facility. If the photographed image is associated with the device to be photographed and the installation location of the device, sorting of the photographed image is facilitated. If the sorting work becomes easy, it is possible to easily find an image necessary for creating the inspection report, and it becomes easy to utilize the image. Also, if the images are organized for each device, the images of the same device can be browsed in time series, so that changes in the devices over time can be noticed, leading to efficient inspection work.

  However, since the number of inspection target devices in the plant facility is enormous, the number of captured images is large, and there are many similar inspection target devices, it takes a long time to sort the captured images. In addition, since the inspection route in the plant facility may change depending on the inspection date, mistakes are likely to occur in the sorting operation of the captured images.

  The techniques of Patent Document 1 and Patent Document 2 cannot be used in a plant facility in which a metal material that shields GPS radio waves is used in buildings and structures.

  Therefore, in a plant facility where position information using GPS cannot be used, as a sorting operation of the captured images, the captured image (inspection image), the imaging location (location of the inspection target device), the imaging target (inspection target device), Need to be associated efficiently.

  The disclosed image registration apparatus includes a template database in which an inspection target device, which is template data, is imaged, a template database stored in advance along the first inspection route of the inspection target device, and an inspection image in which the inspection target device is imaged. An image input unit that inputs and stores the inspection image in the inspection image database in order of the shooting time of the inspection image, a similarity calculation unit that calculates the similarity between the inspection image and the template image, a time difference between the shooting times of the different inspection images, The required time difference evaluation part that calculates the time difference with the predetermined time required for the operator to move between the inspection target devices along the inspection route of the inspection, and the inspection image was taken based on the similarity and the time difference, A second inspection route of the inspection target device is estimated, and a template image corresponding to the inspection image is obtained from the template database based on the second inspection route. Having a template image selecting section for al selection.

  According to the disclosed image registration apparatus, a captured image, a shooting location, and a shooting target can be efficiently associated with each other as a sorting operation of the shot images.

It is a block diagram which shows the structure of an image registration system. It is an example of the hardware constitutions of an image registration apparatus. It is a process flowchart of an image registration apparatus. It is a structural example of an inspection image database. It is a process flowchart of a similarity calculation part. It is a structural example of a template database. It is a process flowchart of a required time difference evaluation part. It is a structural example of the database for movement time calculation. It is a process flowchart of a template image selection part. It is a schematic diagram explaining operation | movement of a template image selection part. It is a schematic diagram explaining operation | movement of a template image selection part. It is an example of a structure of an output database. It is a block diagram which shows the structure of the image registration system of Example 2. FIG. FIG. 10 is a block diagram illustrating a configuration of an image registration system according to a third embodiment. It is an example of a screen display of a processing result by the image registration device. It is an example of display on a layout diagram of a processing result by an image registration device. It is a structural example of the template database of Example 2. FIG. It is an example of a structure of the output database of Example 2. FIG. It is an example of a work support screen. It is an example of the display screen of work content. FIG. 10 is a block diagram illustrating a configuration of an image registration system according to a fourth embodiment. It is a structural example of a standard inspection route database. 14 is a process flowchart of a template image selection unit according to the fourth embodiment.

  Hereinafter, embodiments will be described using some examples.

  FIG. 1 is a block diagram showing the configuration of the image registration system of this embodiment. When an image obtained by photographing an object using the photographing apparatus 111 is input, this system identifies the photographing object of the input image and outputs information about the object such as the position of the identified photographing object as a processing result. In this system, the operator can check the shooting omission of the object and the shooting contents. An example of a procedure for specifying an inspection object using this image registration system and giving information about the object will be described below.

  The image registration system includes an imaging device 111 and an image registration device 100 for an inspection operator (hereinafter referred to as an operator) such as a digital camera or a tablet terminal to image an inspection target device. The image registration apparatus 100 includes an image input unit 101, a similarity calculation unit 102, a required time difference evaluation unit 103, a template image selection unit 104, a processing unit of the image information registration unit 105, an inspection image database 106, and a template database 107. , A travel time calculation database 108, and an output database 109. In the drawings, the database is represented as DB.

  The image input unit 101 inputs an inspection image obtained by imaging the inspection target device by the imaging device 111 and stores the inspection image in the inspection image database 106. The similarity calculation unit 102 calculates the similarity between the inspection image in the inspection image database 106 and the template image in the template database 107. The required time difference evaluation unit 103 evaluates the difference (time difference) between the difference time between the shooting times of different inspection images and the time required for the inspection worker to move between devices calculated from the movement time calculation database 108. The template image selection unit 104 selects a template image corresponding to the inspection image based on the similarity calculated by the similarity calculation unit 102 and the time difference calculated by the required time difference evaluation unit 103. Further, the template image selection unit 104 outputs a template image that has no corresponding inspection image and is not selected as an inspection omission target. The image information registration unit 105 registers the installation position information of the inspection target device associated with the selected template image in advance in the output database 109 in association with the inspection image.

  FIG. 2 is an example of a hardware configuration of the image registration apparatus 100. The image registration apparatus 100 is realized by a computer in which a processor 201, a main memory 202, an input device 203, an output device 204, an interface (I / F) 205, and a storage device 206 are connected to one another. The processor 201 executes a program stored in the main memory 202. The main memory 202 is a semiconductor memory, for example, and stores a program executed by the processor 201 and data referred to by the processor 201. Specifically, at least a part of the program and data stored in the storage device 206 is loaded into the main memory 202 as necessary. The input device 203 receives input from a user such as an operator or a system administrator. The input device 203 is, for example, a keyboard or a mouse. The output device 204 outputs information to a user or a system administrator. The output device 204 is an image display device such as a liquid crystal display. An I / F 205 is an interface for inputting an image from the imaging apparatus 111. The storage device 206 is a non-volatile storage device such as a hard disk device (HDD) or a flash memory. The storage device 206 of this embodiment includes at least an image input unit 101, a similarity calculation unit 102, a required time difference evaluation unit 103, a template image selection unit 104, an image information registration unit 105, an inspection image database 106, and a template database 107. The travel time calculation database 108 and the output database 109 are stored. The image input unit 101, similarity calculation unit 102, required time difference evaluation unit 103, template image selection unit 104, and image information registration unit 105 are processing units (programs) executed by the processor 201.

  FIG. 3 is a process flowchart of the image registration apparatus 100. This processing flowchart represents a schematic processing flow of the image registration apparatus 100, and details of each processing will be described later.

  When an operator patrols the plant facility and completes photographing of the inspection target device and connects the photographing device 111 to the image registration device 100, the image input unit 101 executes processing (step 301). The image input unit 101 inputs an inspection image in the photographing apparatus 111 and the photographing time information thereof obtained by photographing the inspection target device, and stores them in the inspection image database 106.

  FIG. 4 is a configuration example of the inspection image database 106. The inspection image database 106 stores the inspection image 401 and the photographing time 402 when the inspection image 401 was photographed in association with each other. The inspection image database 106 stores the inspection images 401 in order of the photographing times 402.

  FIG. 6 is a configuration example of the template database 107. The template database 107 stores template data created in advance by a skilled worker or the like along an inspection route that is easy to understand and time-efficient for each inspection route. The template data stored in the template database 107 includes a template image 601 obtained by photographing the inspection target device, a device ID 602 that is a number (identifier) for uniquely identifying the inspection target device, and an inspection target on a predetermined layout drawing of the plant facility. A coordinate value 603 of the installation position of the device and a standard inspection time 604 of the inspection target device are included. The inspection time 604 is ST (Standard Time) in process management.

  When the process of the image input unit 101 ends, the similarity calculation unit 102 executes the process (step 302).

  FIG. 5 is a process flowchart of the similarity calculation unit 102. The similarity calculation unit 102 acquires the inspection image 401 from the inspection image database 106 and the template image 601 from the template database 107 one by one (steps 501 and 502), and the similarity between the acquired inspection image 401 and the template image 601. The degree is calculated (step 503). The similarity is larger as the similarity is higher. The similarity calculation unit 102 calculates the similarity for all combinations of the inspection image 401 and the template image 601 (steps 504 and 505). Any method may be used for calculating the similarity as long as the method can quantify the similarity between two images. For example, by using SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features) which are feature quantity extraction methods, the feature quantities of the inspection image 401 and the template image 601 are extracted, and the extracted feature quantities The distance is calculated, and the reciprocal of the calculated distance is used as the similarity.

  When the process of the similarity calculation unit 102 ends, the required time difference evaluation unit 103 executes the process (step 303).

  FIG. 7 is a process flowchart of the required time difference evaluation unit 103. The required time difference evaluation unit 103 acquires the photographing time 402 of two inspection images 401 whose photographing order is continuous from the inspection image database 106, and calculates the time difference (steps 701 and 702). The required time difference evaluation unit 103 acquires travel time data from the travel time calculation database 108 (step 703).

  FIG. 8 is a configuration example of the travel time calculation database 108. The movement time calculation database 108 includes a movement start position 801, a device ID (hereinafter referred to as a start device ID) 802, a movement end position 803, and a device ID (hereinafter referred to as a movement end position 803) as movement required time data. End device ID) 804, and the required movement time 805 required for movement from the movement start position to the movement end position are stored in association with each other. The correspondence between the movement start position 801 and the movement end position 803 (or the start device ID 802 and the end device ID 804) may be covered for all devices to be inspected along the inspection route, but the devices are in a distant position relationship. It is only necessary to cover between devices on the expected inspection route. For example, if the inspection path of device A → device B → device C → device D is acceptable for the inspection path of device A → device C → device B → device D, then device A → device B, device B → In addition to device C, device C → device D, it is necessary to cover the movement of device A → device C, device C → device B, device B → device D. The required travel time 805 is a result of actually walking and measuring between inspection devices in advance. Alternatively, the actual walking time can be omitted by calculating the distance between two points from the position 603 in the template database 107 and estimating the moving time using the walking speed of a general worker. When estimating the travel time 805, more accurate travel time can be estimated by taking into account the difference in the speed of going up and down the stairs.

  The required time difference evaluation unit 103 evaluates the difference between the time difference obtained in step 702 (hereinafter, time difference) and the required movement time 805 acquired in step 703 (step 704). The time difference includes the inspection time of the inspection target device. Therefore, the required time difference evaluation unit 103 refers to the template database 107 and (movement time) = (time difference) − {(inspection time 604 of the start device ID 802) + (inspection time 604 of the end device ID 804)} / 2 to estimate the travel time.

  The required time difference evaluation unit 103 uses an evaluation method of a difference between the travel time and the required travel time 805 (hereinafter referred to as a time difference) by simply using the time difference or correcting the time difference by a function such as a normal distribution. And a method using a time ratio can be used. The required time difference evaluation unit 103 executes processing in the combination of all the inspection images 401 along the inspection route and all the movement required time data along the inspection route of the movement time calculation database (steps 705 and 706). .

  The moving time obtained by correcting the time difference between the photographing times calculated in step 702 described above corresponds to the time required for the operator to move from the nth inspection target location to the n + 1th inspection target location. Therefore, if the movement start position and movement end position of the movement required time data acquired in step 703 coincide with the installation positions of the two images selected in step 701, the time difference calculated in step 704 becomes small. If they do not match, the time difference becomes large. By utilizing this property, the template image selection unit 104 described later evaluates the validity of the operator's inspection route.

  When the process of the required time difference evaluation unit 103 ends, the template image selection unit 104 executes the process (step 304).

  The template image selection unit 104 estimates the inspection route traveled by the worker based on the similarity calculated by the similarity calculation unit 102 and the time difference calculated by the required time difference evaluation unit 103, thereby generating an inspection image 401. This is processing for selecting corresponding template data.

  FIG. 9 is a process flowchart of the template image selection unit 104. A case where there is no inspection omission of the inspection target device will be described first, and a case where there is an inspection omission (insufficient inspection image) will be described later.

  As described above, even if there are inspection routes for the same plurality of devices as inspection targets, there may be a plurality of inspection routes. Therefore, the template image selection unit 104 uses an unevaluated inspection route as an evaluation target route. One is selected (step 901). The template image selection unit 104 calculates the sum of the similarities along the selected inspection route (step 902).

FIG. 10A is a schematic diagram for explaining the operation of the template image selection unit 104, and is a diagram for explaining the sum of similarities along the selected inspection route. The table in FIG. 10A represents the similarity (S _ij , i is a table row, and j is a table column) calculated in step 302. The row numbers (A to E) represent the device ID 602, and the column numbers (1 to 5) represent the photographing order of the inspection image 401. Each cell in the table stores a similarity S _ij between the template image 601 of the device ID 602 and the inspection image 401.

The inspection route corresponds to a combination in which one cell is selected from each column in the order of S _B1 → S _D2 → S _E3 → S _C4 → S _A5 in FIG. 10A. In other words, one inspection path corresponds to the result of selecting the device IDs 602 of the inspection target devices one by one for all the inspection images 401 photographed by the worker. The sum of the similarities in this table along the inspection route means the validity of the route, and the route with the larger sum of similarities among multiple routes is likely to be the route that the operator inspected. I can say that. Note that, as described above, if an inefficient route that continuously inspects devices in a distant positional relationship is excluded from the evaluation target, the number of routes to be evaluated can be reduced. The time required can be shortened.

  Therefore, the template image selection unit 104 calculates the sum of the similarities of the selected evaluation target route, and uses the sum of the similarities as a result of evaluating the validity of the route. Similarly, the template image selection unit 104 calculates the sum of time differences as an index of time differences along the selected evaluation target route (step 903). Thereby, the validity of the route can be evaluated from the viewpoint of time difference. As described above, if the evaluation target route is the same as the inspection route through which the operator has passed, the time difference on each arrow of the route in FIG. Smaller than other routes. When re-inspecting a certain device as described above, it can be expanded by adding the re-inspection device ID 602 to the row number and adding the inspection image at the time of re-inspection to the column number in FIG. 10A.

  The template image selection unit 104 calculates a difference (total difference) between the sum of the similarities and the sum of the time differences (step 904). The higher the similarity (the larger the value) and the smaller the time difference (the smaller the value), the more appropriate the evaluation target route is as the inspection route. Therefore, the template image selection unit 104 expresses the calculated sum difference as the route validity. Score. If the score evaluation of all the evaluation target routes is completed (step 905), the route having the highest score is estimated to be the operator inspection route (step 906). As described above, the template image selection unit 104 selects an appropriate template image 601 for each inspection image 401 according to the estimated inspection route.

  When the process of the template image selection unit 104 is completed, the image information registration unit 105 executes the process (step 305). Based on the processing result of the template image selection unit 104, the image information registration unit 105 registers the imaging target device information in the output database 109 in association with the inspection image 401.

  The sum difference is calculated as the difference between the sum of the similarities and the sum of the time differences. However, depending on the similarity calculation method used by the similarity calculating unit 102, the values of the sums may be orders of magnitude. . In such a case, using the coefficient k, the sum difference may be corrected by (sum difference) = (sum of similarities) −k (sum of time differences). Using the coefficient k means using a value proportional to the sum of the time differences.

  The processing of the template image selection unit 104 when there is an inspection omission (insufficient inspection image) of the inspection target device will be described. In addition, in the process flowchart of FIG. 9, although the process when there exists an inspection omission of an inspection object apparatus is abbreviate | omitted, it uses for the following description.

  FIG. 10B is a schematic diagram for explaining the operation of the template image selection unit 104. As shown in FIG. 10B, the template image selection unit 104 compares the number of inspection target devices in the evaluation target route selected in step 901 (the number of template images 601 included in the template database 107 for each inspection route or the device ID 602). If the number of inspection images is insufficient, it is determined that there is an inspection omission. The template image selection unit 104 stores the determination result (inspection omission) in the work area of the main memory 202 of the image registration apparatus 100. The template image selection unit 104 identifies a template image 601 along the inspection route that does not have a corresponding inspection image 401.

  In response to the determination result stored in the work area that there is an inspection omission, the template image selection unit 104 is in the processing loop of the process (steps 901 to 905) related to the evaluation target path or the inspection path of the operator In any of the steps after the estimation (step 906), a process of specifying the template image 601 along the inspection route that does not have the corresponding inspection image 401 is executed. In the example of FIG. 10B, since the inspection target device whose device ID is “C” is incomplete inspection (there is no corresponding inspection image), the template image selection unit 104 has the inspection target device whose device ID is “C”. Is displayed as a message on the output device 204. In response to the display of this message, the operator inspects the inspection target equipment that has been inspected and takes an inspection image.

  When the travel time by the required time difference evaluation unit 103 is obtained using the photographing time 402 of the inspection image 401 additionally photographed in response to the inspection omission, the photographing time is different from the previous inspection. Therefore, the inspection image 401 additionally photographed corresponding to the inspection omission is inserted into the previously estimated inspection route of the operator. In other words, in the inspection image database 106, the inspection image 401 of the inspection target device for inspection omission should be stored, and the additional image was taken corresponding to the inspection omission at the position in the order along the estimated inspection route of the operator. An inspection image 401 is inserted.

  As described above, the inspection omission of the inspection target device can be detected, and the additionally captured inspection image 401 can be additionally stored in the inspection image database 106. In response to the additional storage of the additionally imaged inspection image 401 in the inspection image database 106, the image information registration unit 105 registers the imaging target device information corresponding to the inspection image 401 additionally stored in the output database 109.

  The template image selection unit 104 estimates the inspection route of the worker using the sum difference between the sum of the similarities of the inspection images 401 and the sum of the time differences, and the inspection image along the estimated inspection route. Although it has been described that the template image 601 corresponding to 401 is selected, the inspection image is not estimated by using the difference between the similarity between each inspection image 401 and the template image 601 and the time difference in movement time. A template image 601 corresponding to 401 may be selected. This corresponds to the case where there is one inspection target device in the method of estimating the inspection route, and the second inspection image is obtained using the result of selecting the template image 601 corresponding to the first inspection image 401 in the inspection route. In this method, the template images 601 are sequentially selected so that the template image 601 corresponding to 401 is selected.

  FIG. 11 is a configuration example of the output database 109. The output of the template image selection unit 104 is a series of combinations of the device ID 602, the position 603, and the inspection image 401 of the inspection target device. The image information registration unit 105 acquires, from the inspection image database 106, the inspection image 1101 of the output database 109 that corresponds to the inspection image shooting order included in the output of the template image selection unit 104 and stores it. The image information registration unit 105 obtains the device ID 602 and the position 603 of the selected template data from the template database 107 and stores the device ID 1102 and the position 1103 of the output database 109 corresponding to the inspection image 401. To do. The image information registration unit 105 stores data in the output database 109 for all series included in the output of the template image selection unit 104. As described above, the inspection image is associated with the device ID to be imaged and the installation position of the device.

  In the present embodiment, a method of using two types of information of “similarity” and “time difference” has been shown in order to improve the accuracy of association between the inspection image and the template image. In a plant facility, there are cases where a plurality of devices having similar appearances are arranged in different places, so that a method using only the similarity of images detects a plurality of template images having a high similarity, and an inspection image. The problem is that the correspondence between the template image and the template image cannot be narrowed down uniquely. Therefore, in this embodiment, the validity of the inspection route is evaluated while considering the travel time in addition to the similarity between the images, and the inspection image and the template image are associated with each other. Thereby, even if it is apparatuses with similar external appearance, if the movement time to the position of those apparatuses differs, apparatus can be distinguished. Therefore, the accuracy of associating the inspection image with the template image is improved. In addition, when the route at the time of inspection is known in advance, the accuracy of associating the inspection image with the template image is further increased by using the information on the inspection route, or the processing time is shortened as described above. It is possible.

  The image registration system of the present embodiment displays the processing result by the image registration device 100 on the output device 204 such as a liquid crystal display. FIG. 14 is a display example on the output device 204. In the display example of FIG. 14, a list of device IDs, installation positions, and inspection images is displayed for each device on the screen of the output device 204. The displayed device ID is a device ID 1102 stored in the output database 109. The displayed position is obtained by calculating a position on a predetermined layout based on the position 1103 of the output database 109 and displaying a double circle at the calculated position. The inspection images may be displayed side by side in the order of inspection date and time for each device as shown in FIG. By displaying the list in this way, the worker and its manager can easily grasp the position of the device and the state of the device in the plant facility. Further, as in the example of the output screen of FIG. 15, the inspection results of a plurality of devices may be displayed together on one layout diagram.

  In the above, the template image 601 stored in the template database 107 is one for one device. However, even a plurality of images having different shooting conditions such as a shooting position, a shooting date, a shooting time, and a camera used for shooting can be used. good. By using a plurality of template images 601 and selecting the corresponding template image 601 based on the inspection image, it becomes possible to absorb the variation of the inspection image caused by the operator, the imaging environment, and the imaging device, and the template image selection unit 104 can be absorbed. The accuracy of associating the inspection image with the template image can be improved. For example, the template image 601 photographed in a bright room on a sunny day and the inspection image 401 photographed in a dark room on a rainy day have greatly different brightness values, so that even images captured by the same device may be used. The similarity is low, and the template image selection unit 104 may select an incorrect template image. As a solution to this problem, it is effective to store a plurality of template images 601 taken on days with different weather in the template database 107.

  When a plurality of template images 601 are stored in the template database 107, the similarity calculation unit 102 processes step 502 and step 503 described above as follows. The similarity calculation unit 102 acquires a plurality of template images 601 associated with one device, and calculates the similarity between the inspection image 401 and each of the plurality of template images 601. The similarity calculation unit 102 outputs the similarity having the largest value among the calculated plurality of similarities in association with the template image 601 indicating the similarity. With the above processing, even when the brightness of the captured image changes due to the weather on the inspection day, it is possible to select a correct template image 601 having a high similarity among the plurality of template images. Here, a change in weather is taken as an example, but it is desirable to photograph a plurality of template images 601 under different photographing conditions such as a photographing position and a photographing device, and to store them in the template database 107. It is also possible to use an image sorted by the image registration apparatus 100 after the inspection work as a template image for subsequent sorting. The accuracy of image sorting can be improved by collecting images for inspection and increasing the number of template images each time the inspection image is sorted using the image registration apparatus 100.

  According to the present embodiment described above, as an operation for sorting the captured images, it is possible to efficiently associate the captured image with the shooting location and the shooting target. In addition, it is possible to prevent shooting omission of the shooting object.

  Further, according to the present embodiment, since template data corresponding to the inspection image can be selected, the efficiency of sorting inspection images can be improved.

  The present embodiment is a mobile image registration apparatus as an image registration system in which the photographing apparatus 111 and the image registration apparatus 100 according to the first embodiment are integrated. FIG. 12 is a block diagram illustrating a configuration of the mobile image registration apparatus 120. The mobile image registration apparatus 120 uses a portable small computer such as a smartphone. In the first embodiment, the image input unit 101 executes the process when the photographing apparatus 111 is connected to the image registration apparatus 100, whereas in the present embodiment, the image input unit 101 executes the process immediately after the photographing. Other configurations and operations are the same as those in the first embodiment.

  In this embodiment, an inspection image is input immediately after shooting, and the position of the inspection target device can be specified by executing Step 301 to Step 305 of Embodiment 1, so that the shooting position of the shooting operator is specified in real time. it can.

  FIG. 16 is a configuration example of the template database 107 of this embodiment. In the present embodiment, in addition to the items of the template database 107 of the first embodiment, the template database 107 includes the name 1601 of the inspection work and the work content 1602 in each inspection target device. The name 1601 of the inspection work is a name given to the inspection work and is expressed by a character string. The work content 1602 is expressed by a character string, and is a content to be confirmed by the operator, for example, “confirm that the numerical value of the meter is less than or equal to XX”. Alternatively, the work content 1602 may include an image or a movie that describes the work content.

  These data stored in the template database 107 are stored in the output database 109 of the mobile image registration apparatus 120 as in the first embodiment.

  FIG. 17 is a configuration example of the output database 109 of this embodiment. The output database 109 stores the inspection work name 1701 and work content 1702 acquired from the template database 107 in addition to the inspection image 1101, the device ID 1102, and the position 1103.

  The mobile image registration apparatus 120 according to the present exemplary embodiment includes a work support screen generation unit 1201 for issuing work instructions to workers. Since the work support screen generation unit 1201 refers to the output database 109, the current position information of the worker can be displayed, or the inspected device and the uninspected device can be distinguished and displayed. FIG. 18A is an example of a work support screen that displays the current position of the worker and displays the inspected and uninspected devices separately. This can prevent the operator from missing inspection.

  FIG. 18B is an example of a work content display screen. The work content display screen displays the work content 1702 acquired from the output database 109. Immediately after the worker takes an image of the inspection device, the necessary work content at that location is displayed on the mobile image registration device 120, so the worker can work at an appropriate timing.

  FIG. 13 is a block diagram showing the configuration of the image registration system of this embodiment. The image registration system of the present embodiment has a configuration in which a work support terminal 1301 including an image device 111 and a work support screen generation unit 1201 and a server 1302 on which the image registration device of the first embodiment is mounted are connected via a network. . In general, the processing capability of the work support terminal 1301 is lower than the processing capability of the server 1302, and if the number of devices to be inspected increases, the processing at the work support terminal 1301 is delayed and real-time processing cannot be performed. On the other hand, in the present embodiment, the processing that increases as the number of inspection points increases is mounted on the server 1302 having a high processing capacity, so that the operator's work can be supported without impairing real-time performance.

  FIG. 19 is a block diagram showing the configuration of the image registration system of this embodiment. The image registration system of the present embodiment adds a standard inspection route database 1901 to the image registration system of the first embodiment. By using the standard inspection route database 1901, the template image selection unit 104 can more accurately evaluate the validity of the inspection route.

  FIG. 20 is a configuration example of the standard inspection route database 1901. The standard inspection route database 1901 stores an inspection route 2001, a device ID 2002 corresponding to the inspection route, and a device installation position 2003. In the example of FIG. 20, it is determined as a standard route that inspection is performed in the order of device A → device B →. The standard route is a route that is selected by a skilled worker or the like before the inspection work so that all devices in the plant facility can be visited most efficiently.

  Here, a template image selection unit 104 in which processing of the present embodiment is different from that of the first embodiment will be described. FIG. 21 is a processing flowchart of the template image selection unit 104 of the present embodiment. Steps 2101 and 2102 are different from the template image selection unit 104 of the first embodiment. The template image selection unit 104 selects one route from the unevaluated cyclic routes (step 901). The template image selection unit 104 calculates the sum of the similarities and the sum of the time differences along the selected route (steps 902 and 903).

  The template image selection unit 104 evaluates the difference between the selected evaluation target route and the standard inspection route (step 2101). This evaluation uses a determination function that increases in value as the evaluation target route and the standard inspection route differ. As a result, the inspection route traveled by the worker is generally along the standard route, and the difference from the standard route should be small, that is, the evaluation target route having the smallest result value using the judgment function is determined by the worker. It is possible to reflect the assumption that the route is an inspection route, and it is possible to estimate the position with higher accuracy. As a specific calculation method, the distance between the position of the inspection target device on the standard route and the position of the corresponding inspection target device on the evaluation target route is calculated, and the sum is defined as the difference between the routes. For example, when the standard route is device A → device B → device C and the evaluation target route is device C → device A → device B, (distance between device A and device C) + (distance between device B and device A) ) + (Distance between device C and device B), the sum of the distances between the Nth inspection devices on the route is defined as the difference between the two routes. In addition, by using a DP (dynamic programming) matching algorithm, it is possible to define a determination function that avoids the problem that the difference becomes unduly large when one inspection device is skipped for inspection. Any determination function may be used as long as it can evaluate the difference between the two paths.

  The template image selection unit 104 calculates the score of the evaluation target route by integrating the sum of the similarities, the sum of the time differences, and the result of the route difference (step 2102). As for the calculation of the score of the evaluation target route, a route having a high similarity to the template image, a small time difference, and a small difference in the route obtained in this example is appropriate as the route of the inspector. Therefore, the calculated value is used as a score expressing the validity of the route. The template image selection unit 104 can estimate the operator's inspection route by finishing the score evaluation of all routes and selecting the route having the highest score (step 906). This processing result is the result of selecting the most appropriate template image for the inspection image.

  According to the present embodiment, template data corresponding to the inspection image can be selected more correctly, and the inspection image sorting work can be supported.

  According to the embodiment described above, as an operation for sorting captured images, it is possible to efficiently associate a captured image with a shooting location and a shooting target.

  DESCRIPTION OF SYMBOLS 100: Image registration apparatus 101: Image input part 102: Similarity calculation part 103: Required time difference evaluation part 104: Template image selection part 105: Image information registration part 106: Inspection image database 107: Template Database: 108: travel time calculation database, 109: output database, 111: photographing apparatus.

Claims (15)

A template database in which a template image obtained by photographing an inspection target device that is template data is stored in advance along the first inspection route of the inspection target device,
An image input unit that inputs an inspection image in which the inspection target device is photographed, and stores the inspection image in the inspection image database in order of photographing time of the inspection image;
A similarity calculator that calculates the similarity between the inspection image and the template image;
Time difference evaluation for calculating a time difference between the time difference between the photographing times of the different inspection images and a predetermined time required for an operator to move between the inspection target devices along the first inspection route. Part, and
Based on the similarity and the time difference, a second inspection path of the inspection target device that has taken the inspection image is estimated, and the template image corresponding to the inspection image is based on the second inspection path. An image registration apparatus comprising a template image selection unit that selects a template from the template database.   The image registration apparatus according to claim 1, wherein the similarity is a reciprocal of a distance between a feature amount extracted from the inspection image and a feature amount extracted from the template image.   The image registration apparatus according to claim 1, wherein the time difference is a value proportional to the time difference.   The template database stores the inspection time of the inspection target device in association with the template image as the template data, and the time when the operator moves between the inspection target devices is taken for the different inspection images. The image registration apparatus according to claim 1, wherein the inspection time is subtracted from the time difference between the times.   The template image selection unit selects the template image corresponding to the inspection image from the template database according to a value obtained by subtracting the time difference obtained by multiplying the similarity by a predetermined coefficient. The image registration apparatus according to 1.   The template image selection unit estimates the second inspection path from which the inspection image is acquired according to a score obtained by subtracting the total of the time differences obtained by multiplying a total of the similarity degrees related to the inspection image by a predetermined coefficient. The image registration apparatus according to claim 1, wherein the template image corresponding to the inspection image is selected from the template database along the second inspection route.   The template image selection unit uses a determination function whose value increases as the first inspection path and the second inspection path differ, and when the value of the result using the determination function is the smallest, The image registration device according to claim 6, wherein the image registration device is estimated as two inspection routes.   When the inspection image corresponding to the template image is not in the inspection image database based on the second inspection route, the template image selection unit has the inspection target device without the inspection image being an inspection omission. The image registration apparatus according to claim 1, wherein the message is output. An image registration method in an image registration apparatus having a template database in which a template image obtained by photographing an inspection target device, which is template data, is stored in advance along a first inspection route of the inspection target device. Is
Input an inspection image in which the inspection target device is photographed, and store it in the inspection image database in order of photographing time of the inspection image,
Calculating the similarity between the inspection image and the template image;
Calculating a time difference between a time difference between the shooting times of the different inspection images and a predetermined time required for an operator to move between the inspection target devices along the first inspection path;
Based on the similarity and the time difference, a second inspection path of the inspection target device that has taken the inspection image is estimated, and the template image corresponding to the inspection image is based on the second inspection path. A method for registering an image, comprising: selecting from the template database.   The image registration method according to claim 9, wherein the similarity is a reciprocal of a distance between a feature amount extracted from the inspection image and a feature amount extracted from the template image.   The image registration method according to claim 9, wherein the time difference is a value proportional to the time difference.   The template database stores the inspection time of the inspection target device in association with the template image as the template data, and the time when the operator moves between the inspection target devices is taken for the different inspection images. The image registration method according to claim 9, wherein the inspection time is subtracted from the time difference between the times.   The image registration apparatus selects the template image corresponding to the inspection image from the template database according to a value obtained by subtracting the time difference obtained by multiplying the similarity by a predetermined coefficient. The image registration method described in 1.   The image registration device estimates a second inspection path obtained from the inspection image according to a score obtained by subtracting a total of the time differences multiplied by a predetermined coefficient from the total similarity of the inspection image, and The image registration method according to claim 9, wherein the template image corresponding to the inspection image is selected from the template database along a second inspection route.   The image registration device uses a determination function whose value increases as the first inspection route and the second inspection route differ, and when the value obtained as a result of using the determination function is the smallest, The image registration method according to claim 14, wherein the image registration method is estimated as an inspection route.

Images