JP2020159969A - Auxiliary facility state evaluation device, auxiliary facility state evaluation method, and auxiliary facility state evaluation program - Google Patents

Abstract

To obtain an auxiliary facility state evaluation device capable of efficiently supporting the inspection of an auxiliary facility.SOLUTION: An auxiliary facility state evaluation device 1 includes an information acquisition part 22, an auxiliary facility image extraction part 23, a state evaluation part 25, and a display information generation part 26. The information acquisition part 22 acquires the information of a captured image including the image of the auxiliary facility of a structure and the image of the structure. The auxiliary facility image extraction part 23 extracts the information of the image of the auxiliary facility from the information of the captured image acquired by the information acquisition part 22. The state evaluation part 25 determines the state-change level of each of a plurality of areas segmented in the shape of mesh in the auxiliary facility on the basis of the information of the image of the auxiliary facility extracted by the auxiliary facility image extraction part 23. The display information generation part 26 generates display information in which the information of the state-change level of each of the plurality of areas is associated with the information of the image of the auxiliary facility.SELECTED DRAWING: Figure 1

Description

The present invention relates to an ancillary equipment condition evaluation device for evaluating the condition of ancillary equipment which is an ancillary equipment to a structure, an ancillary equipment condition evaluation method, and an ancillary equipment condition evaluation program.

Conventionally, lighting equipment or ancillary equipment such as signboards attached to structures such as tunnels or bridges are regularly inspected. In the inspection of ancillary equipment, work is performed such as whether the bolts or screws for attaching the ancillary equipment to the structure are rusted or loose, or whether the ancillary equipment itself has problems such as cracks. Since such work is performed by the operator visually or visually confirming all the incidental equipment attached to the structure one by one, it takes a lot of time and effort.

Patent Document 1 discloses a technique for evaluating the soundness of a bolt for attaching ancillary equipment to a structure, and evaluates the soundness of the bolt by such a technique without depending on the distinction of sound by an operator. be able to.

Japanese Unexamined Patent Publication No. 2017-90057

However, in the technique described in Patent Document 1, it is necessary to hit each of the bolts for attaching the ancillary equipment to the structure, as in the case of distinguishing the sound by the operator, and the inspection is efficient. Is difficult to do.

The present invention has been made in view of the above, and an object of the present invention is to obtain an ancillary equipment condition evaluation device capable of efficiently supporting inspection of ancillary equipment.

In order to solve the above-mentioned problems and achieve the object, the incidental equipment state evaluation device of the present invention includes an information acquisition unit, an incidental equipment image extraction unit, a state evaluation unit, and a display information generation unit. The information acquisition unit acquires information on a captured image including an image of ancillary equipment of the structure and an image of the structure. The incidental equipment image extraction unit extracts the image information of the incidental equipment from the information of the captured image acquired by the information acquisition unit. The state evaluation unit determines the deformation level of each of a plurality of regions divided in a mesh shape in the incidental equipment based on the information of the image of the incidental equipment extracted by the incidental equipment image extraction unit. The display information generation unit generates display information in which the information on the deformation level of each of the plurality of areas is associated with the information on the image of the incidental equipment.

According to the present invention, there is an effect that the inspection of ancillary equipment can be efficiently supported.

The figure which shows the structural example of the incidental equipment state evaluation apparatus which concerns on Embodiment 1 of this invention. The figure for demonstrating the processing method by the incidental equipment state evaluation apparatus which concerns on Embodiment 1. The figure which shows the specific configuration example of the incidental equipment state evaluation apparatus which concerns on Embodiment 1. The figure which shows an example of the captured image information table which concerns on Embodiment 1. The figure which shows an example of the related information table which concerns on Embodiment 1. The figure which shows an example of the acquisition method of the structure information by the traveling type measuring apparatus which concerns on Embodiment 1. The figure which shows the relationship between the traveling position of the traveling type measuring apparatus which concerns on Embodiment 1 and each partial image capture image. The figure which shows an example of the incidental equipment information table which concerns on Embodiment 1. The figure which shows an example of the image of the ancillary equipment divided into a plurality of areas which concerns on Embodiment 1. The figure which shows an example of the image of ancillary equipment including the abnormality part determined by the abnormality determination part which concerns on Embodiment 1. The figure which shows an example of the relationship between the abnormality part determined by the abnormality determination part which concerns on Embodiment 1 and a plurality of areas used by a level determination part. The figure which shows an example of the display image which is displayed on the display part by the display processing part based on the display information which concerns on Embodiment 1. A flowchart showing an example of processing of the control unit of the incidental equipment state evaluation device according to the first embodiment. The figure which shows an example of the hardware configuration of the incidental equipment state evaluation apparatus which concerns on Embodiment 1.

Hereinafter, the incidental equipment condition evaluation device, the incidental equipment condition evaluation method, and the incidental equipment condition evaluation program according to the embodiment of the present invention will be described in detail with reference to the drawings. The present invention is not limited to this embodiment.

Embodiment 1.
FIG. 1 is a diagram showing a configuration example of an incidental equipment state evaluation device according to the first embodiment of the present invention. As shown in FIG. 1, the incidental equipment state evaluation device 1 according to the first embodiment includes a storage unit 10 and a control unit 20.

The storage unit 10 stores the captured image information which is the information of the captured image obtained by imaging the structure and the incidental equipment which is the equipment attached to the structure. The structure whose image is included in the captured image is, for example, a structure such as a tunnel, a bridge, a road, or a sewer. Ancillary equipment in which the captured image includes an image is, for example, a lighting device, a blower, or a signboard. Hereinafter, a tunnel will be described as an example of the structure, but the structure is not limited to the tunnel, and the structure may be a bridge, a sewer, or another structure.

The control unit 20 includes an information acquisition unit 22, an ancillary equipment image extraction unit 23, a state evaluation unit 25, a display information generation unit 26, and a display processing unit 27. The information acquisition unit 22 acquires the captured image information generated by the traveling type measuring device 2 from the traveling type measuring device 2, and stores the acquired captured image information in the storage unit 10. Further, the information acquisition unit 22 acquires the captured image information stored in the storage unit 10 from the storage unit 10.

The traveling type measuring device 2 has an imaging unit, and generates captured image information including information of the captured image obtained by the imaging unit while traveling. For example, when the structure is a tunnel, the traveling type measuring device 2 generates the captured image information from the captured image information obtained by imaging the inner wall of the tunnel by the imaging unit while traveling in the tunnel.

The incidental equipment image extraction unit 23 extracts the information of the image of the incidental equipment from the information of the captured image acquired by the information acquisition unit 22. The state evaluation unit 25 determines the deformation level of each of the plurality of regions divided in a mesh shape in the incidental equipment based on the information of the image of the incidental equipment extracted by the incidental equipment image extraction unit 23.

The display information generation unit 26 generates display information in which the deformation level information of each region determined by the state evaluation unit 25 is associated with the image information of the incidental equipment. Based on the display information generated by the display information generation unit 26, the display processing unit 27 causes the display unit 3 to display an image in which the deformation level information of each area is associated with the image information of the incidental equipment.

FIG. 2 is a diagram for explaining a processing method by the incidental equipment state evaluation device according to the first embodiment. In the example shown in FIG. 2, a lighting device is shown as ancillary equipment, but the ancillary equipment may include a blower, a signboard, or the like.

In step S1 shown in FIG. 2, the information acquisition unit 22 acquires the information of the captured image generated by the traveling type measuring device 2. Then, in step S2 shown in FIG. 2, the incidental equipment image extraction unit 23 extracts the information of the image of the incidental equipment from the information of the captured image. In the example shown in FIG. 2, in addition to the image of the inner wall of the tunnel, the captured image includes images of four lighting devices as images of ancillary equipment, and the ancillary equipment image extraction unit 23 is an image of each lighting device. Information is extracted from the information of the captured image.

Next, in step S3 shown in FIG. 2, the state evaluation unit 25 performs a state evaluation process for evaluating the state of the ancillary equipment based on the information of the image of the ancillary equipment extracted in step S2. Specifically, the state evaluation unit 25 changes each of the plurality of regions R1 to R9 divided in a mesh shape in the incidental equipment based on the information of the image of the incidental equipment extracted by the incidental equipment image extraction unit 23. Determine the condition level.

In the example shown in FIG. 2, the deformation level of the region R1 is determined to be "4", the deformation level of the regions R7 and R9 is determined to be "3", and the deformation level of the region R3 is determined to be "2". , The deformation level of the region R8 is determined to be "1". For example, in the state evaluation process, the deformation level is determined based on a state such as looseness or rust of a fixture for fixing the luminaire to a tunnel, and a state such as a crack or breakage that occurs in the luminaire. The fixture includes, for example, a fastener such as a screw or a bolt, as well as a mounting bracket and the like.

Next, in step S4 shown in FIG. 2, the display information generation unit 26 generates display information in which the information on the deformation level of each of the plurality of regions is associated with the information on the image of the incidental equipment. The display processing unit 27 causes the display unit 3 to display an image based on the display information generated by the display information generation unit 26. As a result, as shown in FIG. 2, an image in which the information of the deformation level of each of the plurality of regions is associated with the image of the incidental equipment is displayed on the display unit 3. In the example shown in FIG. 2, the information on the deformation level of each of the plurality of areas is superimposed and displayed on the image of the lighting equipment which is ancillary equipment.

As described above, the incidental equipment state evaluation device 1 according to the first embodiment extracts the information of the image of the incidental equipment from the information of the captured image including the image of the structure and the image of the incidental equipment, and determines the state of the incidental equipment. To do. Therefore, the inspector can efficiently inspect the ancillary equipment without directly touching the ancillary equipment.

Further, the incidental equipment state evaluation device 1 displays the deformation level of each of the plurality of regions R1 to R9 divided in a mesh shape in the incidental equipment in association with the information of the image of the incidental equipment. Therefore, the inspector of the ancillary equipment can easily grasp which part of the ancillary equipment has a problem, and the inspector can efficiently inspect the ancillary equipment.

Hereinafter, the configuration and operation of the incidental equipment state evaluation device 1 according to the first embodiment will be described in more detail. FIG. 3 is a diagram showing a specific configuration example of the incidental equipment state evaluation device according to the first embodiment.

As shown in FIG. 3, the incidental equipment state evaluation device 1 according to the first embodiment is communicably connected to the traveling type measuring device 2 via the network 5, and the structure information is transmitted from the traveling type measuring device 2. To get. Such structure information includes information on a captured image obtained by imaging a structure and ancillary equipment and related information on the captured image. The network 5 is, for example, a network such as a LAN (Local Area Network) or a WAN (Wide Area Network).

As shown in FIG. 3, the incidental equipment state evaluation device 1 according to the first embodiment includes a storage unit 10, a control unit 20, and a communication unit 30. The communication unit 30 is connected to the network 5. The communication unit 30 receives the information transmitted from the traveling type measuring device 2 to the incidental equipment state evaluation device 1 via the network 5, and outputs the received information to the control unit 20. Further, the control unit 20 can transmit display information from the communication unit 30 to a device (not shown) connected to the network 5.

The traveling type measuring device 2 records structure information on a portable recording medium such as a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc), or a USB (Universal Serial Bus) memory. You can also do it. The incidental equipment state evaluation device 1 can also acquire the structure information by reading the structure information from the portable recording medium on which the structure information is recorded.

The storage unit 10 includes a captured image information storage unit 11, a related information storage unit 12, and ancillary equipment information storage unit 13. The captured image information storage unit 11 stores the captured image information table 41. FIG. 4 is a diagram showing an example of the captured image information table according to the first embodiment.

The captured image information table 41 shown in FIG. 4 includes a plurality of information including “structure ID (Identifier)”, “image ID”, “imaging time”, and “image information”. The "structure ID" is the identification information of the structure. The "image ID" is identification information of the partially captured image obtained by the traveling type measuring device 2. The information of the partially captured image is the information of a part of the captured image information. The “imaging time” is the time when the partially captured image is obtained by the traveling type measuring device 2. The "image information" is information on a partially captured image obtained by the traveling type measuring device 2.

In the example shown in FIG. 4, the image information of the structure with the structure ID “C001” is shown. Specifically, in the example shown in FIG. 4, the image information of the partially captured image of the image ID “I001” is “I001.bmp”, and the imaging time of the partially captured image is “t _{d1 to} t _d2 ”. .. Further, in the example shown in FIG. 4, the image information of the partially captured image of the image ID “I002” is “I002.bmp”, and the imaging time of the partially captured image is “t _{d2 to} t _d3 ”.

Further, in the example shown in FIG. 4, the image information of the partially captured image of the image ID “I003” is “I003.bmp”, and the imaging time of the partially captured image is “t _{d3 to} t _d4 ”. Further, in the example shown in FIG. 4, the image information of the partially captured image of the image ID “I004” is “I004.bp”, and the imaging time of the partially captured image is “t _{d4 to} t _d5 ”.

The related information storage unit 12 stores the related information table 42. FIG. 5 is a diagram showing an example of a related information table according to the first embodiment. The related information table 42 shown in FIG. 5 includes a plurality of information including "structure ID", "imaging place", "imaging order", "time", and "position information". The "structure ID" is the identification information of the structure. The “imaging location” is information indicating the imaging location by the traveling type measuring device 2 and the name of the structure. In the example shown in FIG. 5, the “imaging place” is a tunnel A, a road B, or the like.

The “imaging sequence” is information indicating the imaging order by the traveling type measuring device 2. In the example shown in FIG. 5, it is shown that the imaging order at the imaging location “tunnel A” is from east to west, and the imaging order at the imaging location “road B” is from south to north. ..

The "time" is information indicating the time. The "position information" is information indicating the traveling position of the traveling type measuring device 2, and is, for example, information indicating latitude, longitude, and altitude. The "position information" is not limited to the information indicating the absolute position, and may be the information indicating the relative position with respect to the reference position. In the example shown in FIG. 5, the travel position of the traveling-type measuring device 2 at time "t _1" is "x _1, y _1, z _1", the time running position of the traveling-type measuring device 2 in the "t _2" is It is "x ₂ , y ₂ , z ₂ ". Further, in the example shown in FIG. 5, the travel position of the traveling-type measurement device 2 at the time "t _8" is "x _8, y _8, z ₈ ', traveling of the traveling-type measurement device 2 at the time" t ₁₅ " The position is "x ₁₅ , y ₁₅ , z ₁₅ ".

The information included in the captured image information table 41 and the related information table 42 described above is notified from the traveling type measuring device 2 to the incidental equipment state evaluation device 1. FIG. 6 is a diagram showing an example of a method for acquiring structure information by the traveling type measuring device according to the first embodiment.

As shown in FIG. 6, the traveling type measuring device 2 is equipped with an imaging unit 51. The traveling type measuring device 2 repeatedly images the surroundings of the traveling type measuring device 2 by the imaging unit 51 while traveling in the structure 6 which is a tunnel, and includes an image of the inner wall of the structure 6 and an image of the incidental equipment 7. Generates information on a captured image including a plurality of partially captured images. The imaging unit 51 is, for example, a line camera, and a plurality of pixels are arranged along a direction orthogonal to or intersecting the traveling direction of the traveling type measuring device 2. The traveling type measuring device 2 generates information on a captured image including a plurality of partially captured images based on pixel information repeatedly output from the line camera. The imaging unit 51 may be an area camera in which a plurality of pixels are arranged in a matrix.

FIG. 7 is a diagram showing the relationship between the traveling position of the traveling type measuring device according to the first embodiment and each partially captured image. As shown in FIG. 7, the traveling type measuring device 2 generates related information in which the traveling position and the time of the traveling type measuring device 2 are associated with each other.

In the example shown in FIG. 7, for example, the running position of the running type measuring apparatus 2 at time "t _1" is "x _1, y _1, z _1", traveling of the traveling-type measuring device 2 at time "t _2" The position is "x ₂ , y ₂ , z ₂ ". Further, in the example shown in FIG. 7, the running position of the running type measurement device 2 at the time "t _8" is "x _8, y _8, z ₈ ', traveling of the traveling-type measurement device 2 at the time" t ₁₅ " The position is "x ₁₅ , y ₁₅ , z ₁₅ ".

Further, in the example shown in FIG. 7, the time running position of the traveling-type measuring device 2 in the _{"t 22"} is _{"x 22,} y _{22, z} 22", traveling of the traveling-type measurement device 2 at the time _{"t 29"} The position is "x ₂₉ , y ₂₉ , z ₂₉ ". The captured image shown in FIG. 7 includes images of the ancillary equipment 7 ₁ , 7 ₂ , 7 ₃ , 7 _{4 as} an image of the ancillary equipment 7, and each ancillary equipment 7 ₁ , 7 ₂ , 7 ₃ , 7 ₄ are included in different portions captured images with one another.

The incidental equipment information storage unit 13 stores the incidental equipment information table 43. FIG. 8 is a diagram showing an example of an incidental equipment information table according to the first embodiment. The incidental equipment information table 43 shown in FIG. 8 shows the “structure ID”, “incidental equipment ID”, “equipment type”, “location information”, “image information”, “deformation level”, “comprehensive evaluation value”, And contains a plurality of information including "deformation rate".

The "structure ID" is the identification information of the structure 6. The "incidental equipment ID" is identification information of the incidental equipment 7. The "equipment type" is information indicating the type of the ancillary equipment 7, and is, for example, a lighting device, a blower, a signboard, or the like. "Location information" is information indicating the position of the ancillary equipment 7. The "image information" is the image information of the incidental equipment 7.

The “deformation level” is information indicating the deformation level of each of the plurality of regions divided in a mesh shape in the ancillary equipment 7. In the example shown in FIG. 8, the plurality of mesh-shaped regions in the ancillary equipment 7 are regions R1, R2, ..., Rm. m is an integer of 3 or more, and in the present embodiment, it is assumed that m = 9. Hereinafter, when each of the regions R1, R2, ..., Rm is shown without distinction, it may be described as the region R.

The "comprehensive evaluation value" is information indicating the degree of deformation of 7 units of ancillary equipment. The “deformation ratio” is information indicating the ratio of the deformation to the entire ancillary equipment 7. The comprehensive evaluation value and the deformation ratio are calculated by the control unit 20 based on the deformation levels of the regions R1, R2, ..., Rm.

In the example shown in FIG. 8, information regarding the ancillary equipment 7 of the structure 6 having the structure ID “C001” is shown. Specifically, in the example shown in FIG. 8, the incidental equipment 7 having the incidental equipment ID “F001” has the equipment type “lighting equipment” and the position information “x _f1 , y _f1 , z _f1 ”. The image information is "F001.bmp", and the deformation levels of the regions R1, R2, ..., Rm are "4,0, ..., 3". Further, in the incidental equipment 7 having the incidental equipment ID "F002", the equipment type is "lighting equipment", the position information is "x _f2 , y _f2 , z _f2 ", and the image information is "F002.bmp". , Regions R1, R2, ..., Rm has a deformation level of "2,0, ..., 2".

Further, in the example shown in FIG. 8, the ancillary equipment 7 having the ancillary equipment ID “F001” has a comprehensive evaluation value of “4”, a deformation rate of “0.33”, and an ancillary equipment ID “F002”. The ancillary equipment 7 has a comprehensive evaluation value of "2" and a deformation rate of "0.11".

Returning to FIG. 3, the control unit 20 will be described. The control unit 20 of the incidental equipment state evaluation device 1 includes an input reception unit 21, an information acquisition unit 22, an auxiliary equipment image extraction unit 23, a position identification unit 24, a state evaluation unit 25, and a display information generation unit 26. , A display processing unit 27 is provided.

The input receiving unit 21 receives a user operation input to the input unit 4. The input unit 4 is, for example, a mouse or a keyboard. The user operation is an operation input to the input unit 4 by the user, and includes, for example, a mouse movement operation and a mouse click operation. Further, the input unit 4 may be a touch panel arranged on the display unit 3. Hereinafter, the user operation input to the input unit 4 may be simply described as a user operation.

The information acquisition unit 22 acquires the captured image information from the traveling type measuring device 2 via the network 5 and the communication unit 30, and adds the acquired captured image information to the captured image information storage unit 11. Further, the information acquisition unit 22 acquires related information from the traveling type measuring device 2 via the network 5 and the communication unit 30, and stores the acquired related information in the related information storage unit 12. The information acquisition unit 22 can also acquire the captured image information and the related information by reading the structure information from the portable recording medium on which the captured image information and the structure information including the related information are recorded. It should be noted that the captured image information and the related information may be recorded on a different portable recording medium.

Further, the information acquisition unit 22 acquires captured image information and related information images from the storage unit 10 when the user operation received by the input reception unit 21 is a display request for incidental equipment status information. The information acquisition unit 22 notifies the incidental equipment image extraction unit 23 and the position identification unit 24 of the captured image information and the captured image information acquired from the storage unit 10.

The incidental equipment image extraction unit 23 extracts the image information of the plurality of incidental equipment 7 from the captured image information included in the captured image information acquired by the information acquisition unit 22. The ancillary equipment image extraction unit 23 has, for example, a learned model generated by inputting an image of the ancillary equipment 7 as a detection target by a machine learning process in advance. The incidental equipment image extraction unit 23 extracts the image information in the range in which the incidental equipment 7 is captured by inputting the information of the captured image acquired by the information acquisition unit 22 into the trained model.

Ancillary equipment image extraction unit 23 has, for example, a trained model for each type of ancillary equipment 7. The incidental equipment image extraction unit 23 inputs the information of the captured image acquired by the information acquisition unit 22 into each trained model, and the output score is equal to or higher than the threshold value and corresponds to the trained model having the highest score. It can be determined that the incidental equipment 7 is included in the captured image.

The incidental equipment image extraction unit 23 can also extract the image of the incidental equipment 7 by pattern matching processing instead of the trained model. For example, the incidental equipment image extraction unit 23 has information on the reference image for the type of the incidental equipment 7, and the information of the captured image acquired by the information acquisition unit 22 has a similarity with the reference image of the threshold value or more. Determine if the information is included. When the incidental equipment image extraction unit 23 determines that the information of the captured image acquired by the information acquisition unit 22 includes the information of the image whose similarity with the reference image is equal to or higher than the threshold value, the similarity with the reference image is the threshold value. The above image information is extracted as an image of the ancillary equipment 7.

The position specifying unit 24 specifies the position of the incidental equipment 7 from which the image information has been extracted by the incidental equipment image extraction unit 23. For example, the position specifying unit 24 specifies the position of the ancillary equipment 7 based on the information of the captured image obtained by extracting the image information of the ancillary equipment 7 by the ancillary equipment image extracting unit 23 and the related information of the captured image. be able to.

The position specifying unit 24 can specify the position of the ancillary equipment 7 by executing the position specifying process specified by the user operation among the first position specifying process and the second position specifying process. The first position specifying process is a process of assigning identification information to each ancillary equipment 7 as information indicating the position of each ancillary equipment 7 according to the imaging order by the imaging unit 51 of the traveling type measuring device 2.

First, the first position specifying process will be described. The first position specifying process is a process of identifying the position of the ancillary equipment 7 based on the time information associated with the image information of the ancillary equipment 7 by the ancillary equipment image extraction unit 23.

In the first position identification process, the position identification unit 24 gives the structure 6 a plurality of incidental equipment IDs that are continuous in the forward or reverse order of the time associated with the image information of the incidental equipment 7 by the incidental equipment image extraction unit 23. Allocate to a plurality of incidental facilities 7. The plurality of incidental equipment IDs that are continuous in the forward order of the time are, for example, a plurality of incidental equipment IDs that are assigned so that the number included increases as the time is newer. Further, the plurality of incidental equipment IDs that are continuous in the reverse order of the time are, for example, a plurality of incidental equipment IDs that are assigned so that the number included decreases as the time is newer.

Here, the captured image information table 41 is in the state shown in FIG. 4, the captured image information is in the state shown in FIG. 7, and a plurality of incidentals consecutive in the positive order of the time associated with the image information of the incidental equipment 7. It is assumed that the equipment ID is assigned to a plurality of incidental equipments 7 attached to the structure 6. In this case, the position specifying unit 24, the information of the auxiliary equipment ₇₁ of the image assigning "F001" ancillary equipment ID, assigned ancillary equipment ID to "F002" refers to the information of the auxiliary equipment _{7 2} images. The position specifying unit 24 assigns the ancillary equipment ID "F003" refers to the information of the auxiliary equipment _{7 3} images, assigning the ancillary equipment ID "F004" refers to the information of the auxiliary equipment _{7 4} images.

Position specifying unit 24, if it contains more than one ancillary equipment 7 to the information of one partial captured image, continuous in the ascending order of X _d coordinate position in the X _d Y _d coordinate system shown in among 7 of such partial captured image A plurality of incidental equipment IDs can be assigned to a plurality of incidental equipment 7 incidental to the structure 6. For example, it is assumed that the partially captured image of the image ID "I004" includes the image information of the two incidental equipment 7. In this case, the position specifying unit 24 assigns the incidental equipment ID "I004" to the image of the incidental equipment 7 closer to the negative direction of the X _d axis among the images of the two incidental equipment 7, and X of the images of the two incidental equipment 7. Ancillary equipment ID "I005" is assigned to the image of the ancillary equipment 7 closer to the _d- axis positive direction.

Further, the captured image information table 41 is in the state shown in FIG. 4, the captured image information is in the state shown in FIG. 7, and a plurality of incidental facilities continuous in the reverse order of the time associated with the image information of the incidental equipment 7. It is assumed that the ID is assigned to a plurality of incidental facilities 7 attached to the structure 6. In this case, the position specifying unit 24 assigns the ancillary equipment ID "F001" refers to the information of the auxiliary equipment _{7 4} image, assigning the ancillary equipment ID "F002" refers to the information of the auxiliary equipment _{7 3} images. The position specifying unit 24 assigns the ancillary equipment ID "F003" refers to the information of the auxiliary equipment _{7 2} images, assigning the ancillary equipment ID "F004" refers to the information of the auxiliary equipment ₇₁ of the image.

Position specifying unit 24, if it contains more than one ancillary equipment 7 to the information of one partial captured image, for example, in descending order of X _d coordinate position in the X _d Y _d coordinate system shown in among 7 of such partial captured image A plurality of continuous incidental equipment IDs are assigned to the plurality of incidental equipment 7 attached to the structure 6. For example, it is assumed that the partially captured image of the image ID "I001" includes the image information of the two incidental equipment 7. In this case, the position specifying unit 24 assigns the incidental equipment ID "I004" to the image of the incidental equipment 7 closer to the X _d- axis positive direction among the images of the two incidental equipment 7, and X of the images of the two incidental equipment 7. Ancillary equipment ID "I005" is assigned to the image of the ancillary equipment 7 closer to the negative direction of the _d- axis.

In addition, the position specifying unit 24 assigns the incidental equipment ID to the image information of the incidental equipment 7 based on the serial number information included in the file name of the partially captured image instead of the information of the acquisition time of the partially captured image. You can also do it. For example, in the captured image information table 41 shown in FIG. 4, the file names of the plurality of partially captured images include "001", "002", "003", and "004" as serial number information. The position specifying unit 24 can assign a plurality of incidental equipment IDs that are continuous in ascending or descending order of the serial number information to the plurality of incidental equipment 7.

Next, the second position specifying process will be described. The second position identification process is based on the traveling position and traveling time of the traveling type measuring device 2 and the imaging time of the captured image obtained by extracting the image information of the incidental equipment 7 by the incidental equipment image extracting unit 23. This is a process for specifying the position of 7.

The position specifying unit 24 acquires information on the imaging time of the captured image from which the image information of the incidental equipment 7 is extracted by the incidental equipment image extraction unit 23 from the captured image information storage unit 11. Further, the position specifying unit 24 acquires information on the traveling position of the traveling type measuring device 2 corresponding to the information of the imaging time acquired from the captured image information storage unit 11 from the related information storage unit 12. The position specifying unit 24 determines the position of the image of the incidental equipment 7 based on the acquired information on the traveling position of the traveling type measuring device 2.

For example, assume that the captured image information table 41 is in the state shown in FIG. 4 and the related information table 42 is in the state shown in FIG. The relationship between the running position of the running type measuring device 2 and the respective partial captured image is in the state shown in FIG. 7, the target position identifying unit 24 identifies a position to be the incidental facilities 7 _1. In this case, the imaging time of the auxiliary equipment ₇₁ of the partial captured image data of the image has been extracted by the auxiliary equipment image extracting section 23 is a range of times t _d1 ~t _d2. Further, the traveling position of the traveling type measuring device 2 in the range of time t _{d1 to} t _{d2 is in} the range of "x ₁ , y ₁ , z ₁ " to "x ₈ , y ₈ , z ₈ ".

Position specifying unit 24, based on a range of ancillary facilities image extracting section 23 auxiliary equipment _{7 1} information of the image is extracted by one of the partial image captured by the image ID "I001", _{"x 1,} y 1, _{z 1} "~ computed as the position of the auxiliary equipment _{7 1} range corresponding to the auxiliary equipment ₇₁ of the range of" _{x 8,} y _{8, z} 8 '. Position specifying unit 24 adds the additional equipment information table 43 of the ancillary equipment information storing unit 13 the operation information of the position of the auxiliary equipment ₇₁ in relation to the auxiliary equipment ₇₁ of the information of the image.

The position specifying unit 24, based on the auxiliary equipment image extracting section 23 auxiliary equipment ₇₁ of the center position information of the image is extracted by one of the partial image captured by the image ID "I001", "x _1, y ₁ , _{z 1} "~" it may be calculated as the position of the auxiliary equipment ₇₁ positions corresponding to the center position of the auxiliary equipment ₇₁ of the range of _{x 8,} y _{8, z} 8 '.

In this case, the position specifying unit 24 determines the time corresponding to the center position of the auxiliary equipment ₇₁ of the time _t d1 _{~t d2} centered position and time. Position specifying unit 24 may determine the traveling position associated with the time closest to the center position and time of the time t ₁ ~t ₈ as the position of the auxiliary equipment 7 _1. Further, the position specifying unit 24 linearly interpolates the two traveling positions associated with the two times close to the central position time among the times t _{1 to} t ₈ by proportional distribution to obtain the traveling position corresponding to the central position time. it can be determined as the position of the auxiliary equipment 7 _1.

Further, in the example shown in FIG. 4, in the captured image information table 41, the captured time of the partially captured image indicates a time range, but the captured time of the partially captured image may be one time. In this case, the position specifying unit 24, by prorating imaging time of two partial captured image by linear interpolation, image ancillary equipment ₇₁ identifies the range of time taken. The position specifying unit 24 is specified by the traveling position associated with the time closest to the oldest time in the specified time range and the traveling position associated with the time closest to the newest time in the specified time range. insofar as it is possible to determine the position of the auxiliary equipment 7 _1.

For example, in FIG. 4, it is assumed that the imaging time of the image ID “I001” is “t _d1 ” and the imaging time of the image ID “I002” is “t _d2 ”. In this case, the position specifying unit 24 specifies the travel position corresponding to the range of auxiliary equipment ₇₁ of the image in the time when the linear interpolation by prorating range of times t _d1 ~t _d2 as the position of the auxiliary equipment 7 ₁ be able to. The position specifying unit 24 specifies the center position time by linearly interpolating the imaging times of the two partially captured images by proportional distribution, and attaches the traveling position associated with the time closest to the center position time. It may be determined as the position of the 7 _1.

Further, the incidental equipment state evaluation device 1 can also associate a three-dimensional position with each pixel of the partially captured image and store it in the storage unit 10. In this case, the storage unit 10 can determine the three-dimensional position of the ancillary equipment 7 as the position of the ancillary equipment 7 based on the three-dimensional position associated with each pixel of the partially captured image.

The association of the three-dimensional position with each pixel of the partially captured image can be performed by, for example, the traveling type measuring device 2. For example, the traveling type measuring device 2 has a laser scanner device (not shown), measures the surroundings of the traveling type measuring device 2 by the laser scanner device while traveling in the structure 6, and based on the measurement result, the structure. Information on the 3D point group including the 3D point group of 6 and the 3D point group of the incidental equipment 7 is generated. The information of the three-dimensional point cloud includes the information of the three-dimensional coordinates of each three-dimensional point. The traveling type measuring device 2 can, for example, associate the three-dimensional position with each pixel of the partially captured image from the position where each three-dimensional point is measured and the position where the partially captured image is captured.

In this way, the position specifying unit 24 can determine the position of each ancillary equipment 7, and associates the determined position information of the ancillary equipment 7 with the image information of the ancillary equipment 7, and the ancillary equipment information storage unit 13 It can be added to the incidental equipment information table 43.

The state evaluation unit 25 determines the deformation level of each of the plurality of regions R partitioned in a mesh shape in the incidental equipment 7 based on the information of the image of the incidental equipment 7 extracted by the incidental equipment image extraction unit 23. .. The state evaluation unit 25 includes a division unit 31, a region deformation level determination unit 32, and a comprehensive evaluation unit 33.

The state evaluation unit 25 divides the image of the ancillary equipment 7 into a plurality of regions R, and then determines the deformation level of each region R. It has a second determination mode for determining the deformation level. Hereinafter, the first determination mode and the second determination mode will be specifically described. The determination mode executed by the state evaluation unit 25 is set to either the first determination mode or the second determination mode based on the user operation or an external setting request.

First, the first determination mode will be described. When the determination mode is set to the first determination mode, the division unit 31 of the state evaluation unit 25 determines the image of the incidental equipment 7 based on the information of the image of the incidental equipment 7 extracted by the incidental equipment image extraction unit 23. Is divided into a plurality of regions R. FIG. 9 is a diagram showing an example of an image of ancillary equipment divided into a plurality of regions according to the first embodiment. In the example shown in FIG. 9, the image of the ancillary equipment 7 is divided into nine regions R1 to R9 in a mesh shape. The number of regions R is not limited to 9, and may be 8 or less, or 10 or more. Further, in the example shown in FIG. 9, each region R has the same size and shape as each other, but is not limited to such an example.

The region deformation level determination unit 32 of the state evaluation unit 25 determines the deformation level of the region R based on the information of the image of the region R divided by the division unit 31. The region deformation level determination unit 32 includes an abnormality determination unit 34 and a level determination unit 35.

The abnormality determination unit 34 inputs the image information of each region R divided by the division unit 31 into the evaluation model, and determines whether or not there is an abnormality in each region R. For example, the region deformation level determination unit 32 determines that there is an abnormality in the region R when the score output from the evaluation model is equal to or greater than the threshold value, and when the score output from the evaluation model is less than the threshold value. , It can be determined that there is no abnormality in the area R.

The evaluation model is a trained model generated by machine learning. Such an evaluation model is generated by machine learning using, for example, an image showing that the attached state of the ancillary equipment 7 or the state of the ancillary equipment 7 itself is abnormal. For example, the evaluation model has an abnormality such as an image in which the fixture for attaching the auxiliary equipment 7 to the structure 6 is loose, an image in which the fixture is rusted, or an image in which the auxiliary equipment 7 is cracked. Generated by machine learning as an image. The loosened fitting is, for example, a state in which the alignment mark or alignment line of the fastener such as a screw or a bolt is deviated from the alignment mark or alignment line of the member to which the fastener is attached. ..

The evaluation model may be an evaluation model for each type of incidental equipment 7, an evaluation model for each type of deformation, and an evaluation model for each type of incidental equipment 7 and each type of deformation. You may. As a result, the abnormality determination unit 34 can specify the type of the ancillary equipment 7 or the type of deformation. The deformation type includes, for example, a state in which the fixture is loose, a state in which the fixture is rusted, and a state in which the incidental equipment 7 is cracked. Further, the evaluation model may be an evaluation model for each area R, an evaluation model for each type of ancillary equipment 7 and an evaluation model for each area R, for each type of ancillary equipment 7, each type of deformation, and It may be an evaluation model for each region R.

The level determination unit 35 analyzes the feature amount of the image of the region R determined to be abnormal by the abnormality determination unit 34, and determines the deformation level of the region R determined to be abnormal by the abnormality determination unit 34. To do.

The level determination unit 35 calculates the deformation level based on the feature amount of the image of the region R obtained from the information of the image of the region R determined to be abnormal by the abnormality determination unit 34. For example, the level determination unit 35 may use one or two of the RGB correlation value, the RGB dispersion value, the brightness correlation value, the brightness dispersion value, and the edge component value in the image of the region R determined to be abnormal by the abnormality determination unit 34. Deformation levels can be calculated based on one or more combinations.

The deformation level is a positive integer in which the normal state without deformation is 0 and the value increases as the degree of deformation increases, but the normal state without deformation is a positive integer and the degree of deformation is The higher the value, the closer the value may be to 0. Further, the deformation level may be a value as long as the degree of deformation can be grasped. For example, a normal state without deformation is set to 0, and a negative integer whose value becomes smaller and larger as the degree of deformation becomes higher. May be good. The higher the degree of deformation, the higher the level of deformation.

Further, when the type of the ancillary equipment 7 is specified by the abnormality determination unit 34, the level determination unit 35 can determine the deformation level based on a different standard for each type of the ancillary equipment 7. Further, when the abnormality type is specified by the abnormality determination unit 34, the level determination unit 35 can determine the deformation level based on different criteria for each deformation type.

Further, the level determination unit 35 determines the deformation level of the region R by inputting the image of the region R determined to be abnormal by the abnormality determination unit 34 into the evaluation model generated by machine learning. You can also do it. Such an evaluation model is generated by machine learning for each deformation type using, for example, an image showing that the attached state of the ancillary equipment 7 or the state of the ancillary equipment 7 itself is abnormal.

The level determination unit 35 inputs the image information of the area R determined to be abnormal by the abnormality determination unit 34 into the evaluation model for each deformation type. The level determination unit 35 determines the deformation level of the region R determined to be abnormal by the abnormality determination unit 34 based on the score output from the evaluation model for each deformation type.

For example, the level determination unit 35 determines the deformation level for each deformation type based on the score output from the evaluation model for each deformation type. The level determination unit 35 increases the deformation level as the score increases. The level determination unit 35 can determine the highest deformation level among the determined deformation levels for each deformation type as the deformation level of the region R.

When the abnormality determination unit 34 identifies the deformation type of the region R determined to be abnormal by the abnormality determination unit 34, the level determination unit 35 evaluates the deformation type notified from the abnormality determination unit 34. The model can be used to determine the level of deformation of region R.

Further, the level determination unit 35 can also determine the deformation level of the region R by using the evaluation model for each deformation type and each deformation level. The evaluation model may be the same as the evaluation model used by the abnormality determination unit 34, and the level determination unit 35 increases the deformation level as the score output from the evaluation model increases.

Next, the second determination mode of the state evaluation unit 25 will be described. When the determination mode is set to the second determination mode, the abnormality determination unit 34 of the state evaluation unit 25 inputs the image information of the incidental equipment 7 extracted by the incidental equipment image extraction unit 23 into the evaluation model. Determine the abnormal part of the incidental equipment 7.

The evaluation model used in the second determination mode is an evaluation model that distinguishes between an abnormal part and a normal part in the image of the incidental equipment 7. Such an evaluation model includes, for example, an evaluation model for each location where an abnormality occurs. The evaluation model for each place where the abnormality occurs is generated by machine learning using the image of the state where the abnormality occurs at the corresponding place.

FIG. 10 is a diagram showing an example of an image of ancillary equipment including an abnormality portion determined by the abnormality determination unit according to the first embodiment. In the example shown in FIG. 10, it is shown that four of the images of the incidental equipment 7 are determined as abnormal parts by the abnormality determination unit 34.

The level determination unit 35 shown in FIG. 3 analyzes the feature amount of the image of the region R including the abnormality portion determined by the abnormality determination unit 34, and changes the region R including the abnormality portion determined by the abnormality determination unit 34. Determine the condition level. For example, the level determination unit 35 may use one or two of the RGB correlation value, the RGB dispersion value, the brightness correlation value, the brightness dispersion value, and the edge component value in the image of the region R including the abnormality portion determined by the abnormality determination unit 34. The deformation level of region R can be calculated based on one or more combinations.

The area R is the same as the area R divided in the first mode. FIG. 11 is a diagram showing an example of the relationship between the abnormality portion determined by the abnormality determination unit according to the first embodiment and a plurality of regions used in the level determination unit. In the example shown in FIG. 11, the regions R1, R3, R8, and R9 include the abnormality portion determined by the abnormality determination unit 34. In this case, the level determination unit 35 analyzes the feature amount of the image of each region R1, R3, R8, R9 to determine the deformation level of each region R1, R3, R8, R9.

The level determination unit 35 inputs an image of the region R including the abnormality portion determined by the abnormality determination unit 34 into the evaluation model generated by machine learning, and determines the deformation level of the region R. You can also. Such an evaluation model may be the same as or different from the evaluation model used in the first mode.

As the evaluation model and the trained model described above, for example, a neural network can be used. The neural network includes a convolutional neural network (CNN: Convolution Neural Network), a recurrent neural network (RNN: Recurrent Neural Network), and the like.

When the captured image includes a plurality of images of the incidental equipment 7 of the same type, the level determination unit 35 can calculate the average value of the deformation level for each of the plurality of regions R1 to R9. The type of the ancillary equipment 7 is, for example, a lighting device, a blower, a signboard, or the like, but may be a model of the ancillary equipment 7.

The comprehensive evaluation unit 33 of the state evaluation unit 25 determines the degree of deformation of 7 units of the ancillary equipment based on the deformation levels of the plurality of regions R determined by the region deformation level determination unit 32. The comprehensive evaluation unit 33 can add the determined degree of deformation of the incidental equipment 7 units to the incidental equipment information table 43 of the incidental equipment information storage unit 13 in association with the image information of the incidental equipment 7.

The comprehensive evaluation unit 33 can set the highest deformation level among the deformation levels of the plurality of regions R determined by the region deformation level determination unit 32 as the degree of deformation of the incidental equipment 7. For example, as shown in FIG. 2, the deformation level of the region R1 is "4", the deformation level of the region R3 is "2", the deformation level of the region R7 is "3", and the deformation level of the region R8 is "3". The deformation level of is "1", and the deformation level of the region R9 is "3". Further, it is assumed that the deformation level of the region R where there is no abnormality is “0”. In this case, the comprehensive evaluation unit 33 sets the degree of deformation of the incidental equipment 7 to “4”.

Further, the comprehensive evaluation unit 33 determines the region R equal to or higher than the threshold level among the deformation levels of the plurality of regions R determined by the region deformation level determination unit 32. Then, the comprehensive evaluation unit 33 can determine the degree of deformation of the ancillary equipment 7 based on the ratio occupied by the region R equal to or higher than the threshold level in the image of the ancillary equipment 7.

For example, it is assumed that the deformation level of each region R is the state shown in FIG. 2 and the threshold level is “3”. Further, it is assumed that the deformation level of the region R where there is no abnormality is 0. In this case, the regions R whose deformation level is equal to or higher than the threshold level are regions R1, R7, and R9. Therefore, the comprehensive evaluation unit 33 sets the degree of deformation of the incidental equipment 7 to "0.33". In this way, the comprehensive evaluation unit 33 can obtain the degree of deformation of the ancillary equipment 7 as an evaluation value different from the five-level deformation level.

Further, the comprehensive evaluation unit 33 divides the value obtained by dividing the total value of the deformation levels of the plurality of regions R determined by the region deformation level determination unit 32 by the total value of the maximum values of the deformation levels. It is also possible to adjust the degree of deformation of the ancillary equipment 7. For example, it is assumed that the deformation level of each region R is in the state shown in FIG. Further, it is assumed that the deformation level of the region R where there is no abnormality is 0. In this case, the comprehensive evaluation unit 33 sets the degree of deformation of the incidental equipment 7 to 12/45. The denominator is the sum of "5", which is the maximum value of the deformation level of each region R. In this way, the comprehensive evaluation unit 33 can obtain the degree of deformation of the ancillary equipment 7 as an evaluation value different from the five-level deformation level.

The comprehensive evaluation unit 33 can express the degree of deformation of the ancillary equipment 7 as a comprehensive evaluation value, a deformation ratio, or the like. For example, the comprehensive evaluation unit 33 sets the highest deformation level among the deformation levels of the plurality of regions R as the comprehensive evaluation value, and the ratio of the region R equal to or higher than the threshold level in the image of the incidental equipment 7 or the plurality of regions R. The value obtained by dividing the total value of the deformation levels of the above by the maximum value can be used as the deformation ratio of the incidental equipment 7. The deformation ratio is the ratio of the deformation in the image of the incidental equipment 7.

The display information generation unit 26 generates display information based on the information in the incidental equipment information table 43 stored in the incidental equipment information storage unit 13. For example, the display information generation unit 26 generates display information in which the deformation level information of each area R, the comprehensive evaluation level information, and the deformation degree information of the incidental equipment 7 are associated with the image information of the incidental equipment 7. be able to. Further, the display information generation unit 26 provides display information in which, in addition to the information on the deformation level of each region R, the information indicating the position of the ancillary equipment 7 determined by the position specifying unit 24 is associated with the image information of the ancillary equipment 7. Can be generated. The display information generation unit 26 can also transmit the generated display information from the control unit 20 to the device connected to the network 5.

The display processing unit 27 displays an image on the display unit 3 based on the display information generated by the display information generation unit 26. FIG. 12 is a diagram showing an example of a display image displayed on the display unit by the display processing unit based on the display information according to the first embodiment.

The display image 80 shown in FIG. 12 includes a display area 81 for displaying the name of the structure 6, a display area 82 for displaying the measurement direction by the traveling type measuring device 2, and a display area 83 for displaying the imaging surface. A display area 84 for displaying the name of the target incidental equipment 7 is included. The imaging surface is a region of the structure 6 that is imaged by the imaging unit 51.

In the example shown in FIG. 12, the characters "tunnel A" are displayed in the display area 81 as the name of the structure 6, and the characters "east → west" are displayed in the display area 82 as the measurement direction of the traveling type measuring device 2. ing. Further, in the example shown in FIG. 12, the characters "left side" are displayed in the display area 83, and the characters "lighting equipment" are displayed in the display area 84.

The display image 80 shown in FIG. 12 has a button 85 for displaying a list box for selecting one imaging surface from a plurality of imaging surfaces, and a type of ancillary equipment 7 to be displayed from among a plurality of types of ancillary equipment 7. Includes a button 86 that displays a list box for selecting. The user of the incidental equipment state evaluation device 1 can select the imaging surface by the user operation of the button 85, or select the type of the incidental equipment 7 to be displayed by the user operation of the button 86. The control unit 20 extracts the image information of the incidental equipment 7 of the type selected by the user operation from the image information of the image pickup surface selected by the user operation, and is based on the image information of the extracted incidental equipment 7. Therefore, the deformed state of the ancillary equipment 7 can be evaluated by the above-mentioned processing.

In the traveling type measuring device 2 shown in FIG. 6, only the imaging unit 51 that images the left side of the plurality of imaging surfaces is shown, but the traveling type measuring device 2 does not show the image of the upper surface of the structure 6. An imaging unit and an imaging unit (not shown) that images the right side surface of the structure 6 are mounted. The control unit 20 of the incidental equipment state evaluation device 1 processes an image captured by an imaging unit (not shown) that images the upper surface and an image captured by the imaging unit (not shown) that images the right side surface, similarly to the image captured by the imaging unit 51. Then, it is possible to determine the deformation level of the area R and the degree of deformation of the incidental equipment 7.

The display image 80 shown in FIG. 12 includes an incidental equipment status information display area 87 showing incidental equipment status information of the incidental equipment 7. In the incidental equipment status information display area 87, the incidental equipment ID of the incidental equipment 7, the information indicating the position of the incidental equipment 7, the information indicating the comprehensive evaluation value of the deformation of the incidental equipment 7, and the deformation of the image of the incidental equipment 7 are displayed. Information indicating the proportion and information indicating an image of the ancillary equipment 7 are included for each ancillary equipment 7.

For example, the incidental equipment 7 having the incidental equipment ID “I001” shown in FIG. 12 has a position of “x ₄ , y ₄ , z ₄ ” and a comprehensive evaluation value of the deformation of “4”, which is occupied by the deformation. The ratio is "0.33". Further, the incidental equipment 7 of the incidental equipment ID “I002” shown in FIG. 12 has a position of “x ₁₁ , y ₁₁ , z ₁₁ ”, and the overall evaluation value of the deformation is “2”, which is occupied by the deformation. The ratio is "0.11". As a result, the user of the incidental equipment state evaluation device 1 can easily grasp at which position of the structure 6 the incidental equipment 7 is in a deformed state. Further, in the example shown in FIG. 12, the incidental equipment IDs are assigned in serial numbers in the order of arrangement of the incidental equipment 7, and even by this, the incidental equipment 7 at which position of the structure 6 is in a deformed state. You can figure out if there is one.

Further, in the image of the ancillary equipment 7, a line for dividing the image of the ancillary equipment 7 into nine areas R and an area R among the nine divided areas R determined to have a deformation are included. Deformation level is shown. As a result, the user of the incidental equipment state evaluation device 1 can easily grasp, for example, which region R of the incidental equipment 7 is deformed and which region R is likely to fail.

In the image of the ancillary equipment 7 shown in FIG. 12, the deformation level of each area R is indicated by characters, but the display information generation unit 26 indicates the deformation level of each area R by a color, a symbol, or the like. You can also do it. For example, the display information generation unit 26 can generate display information in which the deformation level of each region R is represented by the density of the image in the region R or the color of the image in the region R. Specifically, the display information generation unit 26 can generate display information so that the region R having a lower deformation level is darker than the original image. Also from this, it is possible to easily grasp which area R of the incidental equipment 7 is deformed and which area R is likely to fail.

Further, the display information generation unit 26 generates display information so that the region R having a lower deformation level is thinner than the original image, or generates display information representing the deformation level of each region R with colors and characters. Can be done. Also from this, it is possible to easily grasp which area R of the incidental equipment 7 is deformed and which area R is likely to fail.

Further, the display information generation unit 26 can generate display information including information of characters, colors, or symbols indicating the average value of the deformation levels of each region R determined by the level determination unit 35. The display processing unit 27 can display information on characters, colors, or symbols indicating the average value of the deformation levels of each area R on the display unit 3. As a result, the user of the incidental equipment state evaluation device 1 can easily grasp which region R is likely to be deformed for each type of the incidental equipment 7.

Next, the operation of the incidental equipment state evaluation device 1 will be described with reference to a flowchart. FIG. 13 is a flowchart showing an example of processing of the control unit of the incidental equipment state evaluation device according to the first embodiment. Such processing is started, for example, when the user operation is a request for displaying incidental equipment status information.

As shown in FIG. 13, the control unit 20 of the incidental equipment state evaluation device 1 acquires the information of the captured image from the storage unit 10 (step S10), and obtains the image information of the incidental equipment 7 from the acquired captured image information. Extract (step S11). Further, the control unit 20 acquires the captured image information and the related information from the storage unit 10, and determines the position of the incidental equipment 7 based on the acquired captured image information and the related information (step S12).

Next, the control unit 20 determines whether or not the first determination mode is set (step S13). When the control unit 20 determines that the first determination mode is set (step S13: Yes), the image of the incidental equipment 7 is divided into a plurality of regions R (step S14), and each region R is abnormal. Whether or not it is determined (step S15). Then, the control unit 20 determines the deformation level of the abnormal region R (step S16).

When the control unit 20 determines that the first determination mode is not set (step S13: No), that is, when it determines that the second determination mode is set, the control unit 20 determines the abnormal portion of the image of the incidental equipment 7. The determination is made (step S17), and the deformation level of the region R including the abnormal portion is determined (step S18).

When the process of step S16 is completed or the process of step S18 is completed, the control unit 20 determines the degree of deformation of the ancillary equipment 7 (step S19). The control unit 20 generates display information including the incidental equipment status information (step S20), displays the incidental equipment status information on the display unit 3 based on the display information (step S21), and performs the process shown in FIG. To finish.

FIG. 14 is a diagram showing an example of the hardware configuration of the incidental equipment state evaluation device according to the first embodiment. As shown in FIG. 14, the incidental equipment state evaluation device 1 includes a computer including a processor 101, a memory 102, an input / output circuit 103, and a communication device 104.

The processor 101, the memory 102, the input / output circuit 103, and the communication device 104 can send and receive data to and from each other by, for example, the bus 105. The communication unit 30 is realized by the communication device 104. The storage unit 10 is realized by the memory 102. By reading and executing the program stored in the memory 102, the processor 101 reads and executes an input reception unit 21, an information acquisition unit 22, an incidental equipment image extraction unit 23, a position identification unit 24, a state evaluation unit 25, and a display information generation unit. 26 and the display processing unit 27 are executed. The processor 101 is, for example, an example of a processing circuit, and includes one or more of a CPU (Central Processing Unit), a DSP (Digital Signal Processer), and a system LSI (Large Scale Integration).

The memory 102 includes one or more of RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (registered trademark) (Electrically Erasable Programmable Read Only Memory). Including. The memory 102 also includes a recording medium on which a computer-readable program is recorded. Such recording media include one or more of non-volatile or volatile semiconductor memories, magnetic disks, flexible memories, optical disks, compact discs, and DVDs (Digital Versatile Discs). The incidental equipment state evaluation device 1 may include integrated circuits such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array).

As described above, the incidental equipment state evaluation device 1 according to the first embodiment includes an information acquisition unit 22, an incidental equipment image extraction unit 23, a state evaluation unit 25, and a display information generation unit 26. The information acquisition unit 22 acquires the information of the captured image including the image of the incidental equipment 7 of the structure 6 and the image of the structure 6. The incidental equipment image extraction unit 23 extracts the image information of the incidental equipment 7 from the information of the captured image acquired by the information acquisition unit 22. The state evaluation unit 25 determines the deformation level of each of the plurality of regions R partitioned in a mesh shape in the incidental equipment 7 based on the information of the image of the incidental equipment 7 extracted by the incidental equipment image extraction unit 23. .. The display information generation unit 26 generates display information in which the information on the deformation level of each of the plurality of regions R is associated with the information on the image of the incidental equipment 7. As a result, the inspector can efficiently inspect the ancillary equipment 7 without directly touching the ancillary equipment 7. Further, it is possible to display the deformation level of each of the plurality of regions R partitioned in the mesh shape in the incidental equipment 7 in association with the information of the image of the incidental equipment 7. Therefore, the inspector of the ancillary equipment 7 can easily grasp which part of the ancillary equipment 7 has a problem, and the inspector can efficiently inspect the ancillary equipment 7. it can.

Further, the state evaluation unit 25 includes a division unit 31 and a region deformation level determination unit 32. The dividing unit 31 divides the image of the ancillary equipment 7 into a plurality of regions R. The region deformation level determination unit 32 determines each deformation level of the plurality of regions R based on the information of each image of the plurality of regions R divided by the division unit 31. Since the amount of information in the image of each region R is smaller than that in the image of the incidental equipment 7, the deformation level of the region R can be easily determined.

Further, the region deformation level determination unit 32 includes an abnormality determination unit 34 and a level determination unit 35. The abnormality determination unit 34 determines whether or not there is an abnormality in each of the plurality of regions R based on the information of each image of the plurality of regions R divided by the division unit 31. The level determination unit 35 determines the deformation level of the region R determined to be abnormal by the abnormality determination unit 34. As a result, it is sufficient to determine the deformation level of only the region R determined to have an abnormality, so that the processing load on the incidental equipment state evaluation device 1 can be reduced.

Further, the abnormality determination unit 34 determines the abnormality portion of the incidental equipment 7 based on the information of the image of the incidental equipment 7. The level determination unit 35 determines the deformation level of the region R including the abnormal portion determined to be abnormal by the abnormality determination unit 34 among the plurality of regions R. As a result, the deformation level of the region R can be determined without performing the process of dividing the image of the incidental equipment 7.

Further, the level determination unit 35 calculates the average value of the deformation level for each of the plurality of regions R when a plurality of images of the incidental equipment 7 of the same type are included in the captured image. The display information generation unit 26 generates display information in which information on the average value of the deformation level is associated with the information on the image of the incidental equipment 7 in addition to the information on the deformation level of each of the plurality of regions R. As a result, it is possible to easily grasp which area R is likely to be deformed for each type of incidental equipment 7.

Further, the state evaluation unit 25 includes a comprehensive evaluation unit 33 that determines the degree of deformation of 7 units of incidental equipment based on the deformation level of each of the plurality of regions R. The display information generation unit 26 generates display information in which, in addition to the information on the deformation level of each of the plurality of regions R, the information on the degree of deformation of the incidental equipment 7 units is associated with the information on the image of the incidental equipment 7. As a result, the inspector can grasp the degree of deformation of the incidental equipment 7 units obtained from the deformation level of each area R in addition to the deformation level of each area R.

Further, the incidental equipment state evaluation device 1 includes a position specifying unit 24 for specifying the position of the incidental equipment 7 in the structure 6. The captured image is an image including a plurality of images of the incidental equipment 7 of the same type. The display information generation unit 26 generates display information in which the position information of the ancillary equipment 7 is associated with the image information of the ancillary equipment 7 in addition to the information of the deformation level of each of the plurality of regions R. Thereby, for example, it is possible to easily grasp which position of the incidental equipment 7 of the same type is deformed.

Further, the captured image is an image obtained by the imaging unit 51 mounted on the traveling type measuring device 2 repeatedly taking an image while the traveling type measuring device 2 is traveling. The position specifying unit 24 identifies the position of each of the plurality of incidental equipment 7 based on the time when the region R of the incidental equipment 7 in the captured image is captured by the imaging unit 51. As a result, the position of each of the plurality of incidental facilities 7 can be accurately specified.

Further, the captured image is an image obtained by the imaging unit 51 mounted on the traveling type measuring device 2 repeatedly taking an image while the traveling type measuring device 2 is traveling. The position specifying unit 24 identifies the position of each of the plurality of ancillary equipment 7 by adding identification information to the ancillary equipment 7 in the order of imaging by the imaging unit 51. Thereby, for example, it is possible to easily grasp which position of the incidental equipment 7 of the same type is generated outside the field.

The configuration shown in the above-described embodiment shows an example of the content of the present invention, can be combined with another known technique, and is one of the configurations without departing from the gist of the present invention. It is also possible to omit or change the part.

1 Ancillary equipment status evaluation device, 2 Traveling type measuring device, 3 Display unit, 4 Input unit, 5 Network, 6 Structure, 7, 7 ₁ , 7 ₂ , 7 ₃ , 7 ₄ Ancillary equipment, 10 Storage unit, 11 Imaging Image information storage unit, 12 Related information storage unit, 13 Ancillary equipment information storage unit, 20 Control unit, 21 Input reception unit, 22 Information acquisition unit, 23 Ancillary equipment image extraction unit, 24 Position identification unit, 25 Status evaluation unit, 26 Display information generation unit, 27 display processing unit, 30 communication unit, 31 division unit, 32 area deformation level determination unit, 33 comprehensive evaluation unit, 34 abnormality determination unit, 35 level determination unit, 41 captured image information table, 42 related information Table, 43 incidental equipment information table, 51 imaging unit, 80 display image, 81, 82, 83, 84 display area, 85, 86 buttons, 87 incidental equipment status information display area, R, R1 to R9 area.

Claims (11)

An information acquisition unit that acquires information on a captured image including an image of ancillary equipment of a structure and an image of the structure.
Ancillary equipment image extraction unit that extracts image information of the ancillary equipment from the information of the captured image acquired by the information acquisition unit, and
Based on the information of the image of the ancillary equipment extracted by the ancillary equipment image extraction unit, a state evaluation unit that determines the deformation level of each of a plurality of regions divided in a mesh shape in the ancillary equipment.
Ancillary equipment state evaluation device comprising: a display information generation unit which generates display information which associates the information of the deformation level of each of the plurality of regions with the information of the image of the ancillary equipment. The state evaluation unit
A division portion that divides the image of the ancillary equipment into the plurality of areas, and
A claim comprising a region deformation level determination unit for determining the deformation level of each of the plurality of regions based on the information of each image of the plurality of regions divided by the division unit. Item 2. Ancillary equipment condition evaluation device according to item 1. The area deformation level determination unit
An abnormality determination unit that determines whether or not there is an abnormality in each of the plurality of regions based on the information of each image of the plurality of regions divided by the division unit.
The incidental equipment state evaluation device according to claim 2, further comprising a level determination unit for determining a deformation level of a region determined to be abnormal by the abnormality determination unit. The state evaluation unit
An abnormality determination unit that determines an abnormal part of the ancillary equipment based on the information of the image of the ancillary equipment, and
The ancillary equipment according to claim 1, further comprising a level determination unit for determining the deformation level of the region including the abnormality portion determined to be abnormal by the abnormality determination unit among the plurality of regions. Condition evaluation device. The level determination unit
When the captured image contains a plurality of images of the incidental equipment of the same type, the average value of the deformation level is calculated for each of the plurality of regions.
The display information generation unit
According to claim 3 or 4, in addition to the information on the deformation level of each of the plurality of regions, the information on the average value of the deformation level is associated with the information on the image of the ancillary equipment to generate display information. The described incidental equipment condition evaluation device. The state evaluation unit
A comprehensive evaluation unit for determining the degree of deformation of the incidental equipment unit including the plurality of regions based on the deformation level of each of the plurality of regions is provided.
The display information generation unit
Claims 1 to 5 include generating display information in which information on the degree of deformation of the incidental equipment unit is associated with information on an image of the incidental equipment in addition to information on the deformation level of each of the plurality of regions. Ancillary equipment condition evaluation device according to any one of. A position specifying part for specifying the position of the ancillary equipment in the structure is provided.
The captured image is an image including a plurality of images of ancillary equipment of the same type.
The display information generation unit
The ancillary equipment according to claim 6, wherein in addition to the information on the deformation level of each of the plurality of regions, the information on the position of the ancillary equipment is associated with the information on the image of the ancillary equipment to generate display information. Condition evaluation device. The captured image is an image obtained by repeatedly capturing an image captured by an imaging unit mounted on the traveling type measuring device while the traveling type measuring device is traveling.
The position specifying part is
The incidental equipment state evaluation according to claim 7, wherein the position of each of the plurality of incidental equipments is specified based on the time when the region of the incidental equipment in the captured image is imaged by the imaging unit. apparatus. The captured image is an image obtained by repeatedly capturing an image captured by an imaging unit mounted on the traveling type measuring device while the traveling type measuring device is traveling.
The position specifying part is
The ancillary equipment state evaluation apparatus according to claim 7, wherein identification information is given to each of the plurality of ancillary equipment in the order of imaging by the imaging unit to specify the position of each of the plurality of ancillary equipment. .. Ancillary equipment condition evaluation method executed by a computer
An information acquisition step for acquiring information on a captured image including an image of ancillary equipment of a structure and an image of the structure, and
Ancillary equipment image extraction step for extracting image information of the ancillary equipment from the information of the captured image acquired by the information acquisition step, and
Based on the information of the image of the ancillary equipment extracted by the ancillary equipment image extraction step, a state evaluation step of determining the deformation level of each of a plurality of regions divided in a mesh shape in the ancillary equipment, and a state evaluation step.
A method for evaluating ancillary equipment state, which comprises a display information generation step of generating display information in which information on the deformation level of each of the plurality of regions is associated with image information of the ancillary equipment. An information acquisition step for acquiring information on a captured image including an image of ancillary equipment of a structure and an image of the structure, and
Ancillary equipment image extraction step for extracting image information of the ancillary equipment from the information of the captured image acquired by the information acquisition step, and
Based on the information of the image of the ancillary equipment extracted by the ancillary equipment image extraction step, a state evaluation step of determining the deformation level of each of a plurality of regions divided in a mesh shape in the ancillary equipment, and a state evaluation step.
An incidental equipment state evaluation program characterized by causing a computer to execute a display information generation step of generating display information in which information on the deformation level of each of the plurality of regions is associated with image information of the incidental equipment.

Images