JP2023010324A - Inspection device for equipment construction, inspection method for equipment construction, and inspection program for equipment construction - Google Patents

Abstract

To provide an inspection device for equipment construction, an inspection method for equipment construction, and an inspection program for equipment construction that execute an easy and highly accurate inspection for equipment construction.SOLUTION: A construction inspection server device for equipment construction includes: an acquisition section (API section) that acquires inspection item information indicating that an item to be inspected is a component of a fence, an inspection image obtained by imaging the components of the fence, and guide information on a guide image for imaging the components of the fence included in the inspection image, from a user terminal device 100; an inspection object recognition section that calculates a degree of certainty indicating certainty of the components of the fence on the basis of the inspection image, and recognizes whether the inspection image includes the components of the fence on the basis of the degree of certainty; an inspection section that inspects whether the number of components of the fence is the same as the number of recognized components of the fence based on the guide information; and a notification section that notifies a terminal device of a result of inspection executed by the inspection section.SELECTED DRAWING: Figure 1

Description

[1] [Press release] Date April 21, 2021 Meeting name, venue NTT COMWARE CORPORATION New business brand “SmartMaintech” Tech)” press conference NTT Shinagawa TWINS Annex Building 1F (1-9-1 Konan, Minato-ku, Tokyo) ParkLabo. & Online distribution <Documents> Press conference presentation materials [2] [Website] Date of website publication April 21, 2021 Website address https://www. nttcom. co. jp/news/pr21042101. html <Relevant public website address> https://built. it media. co. jp/bt/articles/2104/23/news021_3. html https://www. nikkei. com/article/DGXZQOUC217L70R20C21A4000000/https://news. my navi. jp/article/20210422-1876405/ https://www. kenset news. com/web-kan/564373 https://b2b-ch. infomart. co. jp/news/detail. page? IM NEWS1=2533109 https://www. sugitec. net/28532/https://bp-affairs. com/news/2021/04/20210423-10827. html <Reference> News release and service content public web page <Reference> Related public website page

The present invention relates to an inspection device for facility construction, an inspection method for facility construction, and an inspection program for facility construction.

Conventionally, techniques for inspecting construction conditions are known. For example, Patent Literatures 1 to 3 describe techniques for inspecting the state of construction using a plurality of members with high accuracy. Patent Literature 1 describes a technique for inspecting the construction state of branching hardware for utility poles, and Patent Literature 2 describes a technique for inspecting the construction state of utility pole support members. Furthermore, Patent Literature 3 describes displaying guide frames and masks of inspection objects according to a plurality of inspection items included in equipment construction.

Japanese Patent Application Laid-Open No. 2021-61709 Japanese Patent Application Laid-Open No. 2021-60887 Japanese Patent Application Laid-Open No. 2021-60888

In the inspection of the construction state described in the above-mentioned Patent Documents 1 to 3, inspection items and required accuracy for each inspection item are often specified for each inspection object. inspection is desired.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-described problems, and provides an inspection device for facility construction, an inspection method for facility construction, and an inspection apparatus for facility construction, which can inspect facility construction easily and with high accuracy. The purpose is to provide a program.

(1) In one aspect of the present invention, from a terminal device, inspection item information indicating that an inspection item is a component of a fence, an inspection image obtained by imaging the component of the fence, and the fence included in the inspection image. an acquisition unit for acquiring guide information related to a guide image for imaging the structure of the fence, calculating a confidence factor indicating the likelihood of the structure of the fence based on the inspection image, and based on the confidence factor, the inspection A recognition unit for recognizing whether or not the image includes the constituents of the fence, and inspecting whether the number of constituents of the fence based on the guide information is the same as the number of the recognized constituents of the fence. An inspection device for facility construction, comprising: an inspection unit; and a notification unit that notifies the terminal device of a result of the inspection performed by the inspection unit.

(2) An aspect of the present invention is the facility construction inspection device described above, wherein the constituent of the fence is a lattice-shaped object of the fence, and the guide image corresponds to each of a predetermined number of lattice-shaped objects. may have a region with

(3) An aspect of the present invention is the facility construction inspection device described above, wherein the constituents of the fence are fasteners of the fence, and the guide image is an area corresponding to each of a predetermined number of fasteners. may have

(4) An aspect of the present invention is the inspection apparatus for facility construction, wherein the guide image is one guide image selected by a builder from among a plurality of guide images having different numbers of the fasteners. good.

(5) An aspect of the present invention is the facility construction inspection apparatus described above, wherein the fastener of the fence includes an insulating portion, and the acquisition unit includes an inspection item indicating that the inspection item is the insulating portion. information, an inspection image obtained by imaging the insulating portion, and guide information related to a guide image for imaging the insulating portion included in the inspection image, and the recognition unit obtains a certainty factor indicating the likelihood of the insulating portion. and recognizes whether or not the insulating portion is included in the inspection image based on the certainty, and the inspection unit determines whether the insulating portion is included in the inspection image based on the guide information. You can check whether

(6) In one aspect of the present invention, from a terminal device, inspection item information indicating that an inspection item is a component of a fence, an inspection image obtained by imaging the component of the fence, and the fence included in the inspection image. a step of acquiring guide information related to a guide image for imaging the structure of the fence, calculating a certainty factor indicating the likelihood of the structure of the fence based on the inspection image, and based on the certainty factor, the inspection image and checking whether the number of fence structures based on the guide information is the same as the number of the recognized fence structures. and a step of notifying the terminal device of an inspection result.

(7) In one aspect of the present invention, from a terminal device to a computer, inspection item information indicating that an inspection item is a component of a fence, an inspection image obtained by imaging the component of the fence, and acquiring guide information related to a guide image for imaging the constituents of the fence, calculating a degree of certainty indicating the certainty of the constituents of the fence based on the inspection image; Recognizing whether or not the image includes the constituents of the fence, inspecting whether the number of constituents of the fence based on the guide information is the same as the number of the recognized constituents of the fence, and performing inspection. It is an inspection program for facility construction, which notifies the terminal device of the results.

ADVANTAGE OF THE INVENTION According to one aspect of the present invention, facility construction can be inspected easily and with high accuracy.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows one structural example of the equipment construction inspection system of embodiment. 4 is a block diagram showing an example of an inspection object recognition unit; FIG. FIG. 4 is a sequence diagram showing an example of inspection processing for facility construction in the facility construction inspection system; It is a flow chart which shows an example of inspection processing of equipment construction in a construction inspection server device. It is a figure which shows an example of the screen which selects an inspection target. It is a figure which shows an example of the screen for selecting the guide image for test|inspecting a plica tube. It is a figure which shows an example of the inspection image of a plica tube, and the recognition result of a plica tube. It is a figure which shows an example of the tag information of Plicatube. It is a figure which shows an example of the guide image of a plica tube. It is a figure which shows an example of the guide information about a plica tube. It is a figure which shows an example of the test|inspection process of a plica tube. It is a figure which shows another example of the test|inspection process of a plica tube. It is a figure which shows an example of the inspection-results screen that installation of a plica tube satisfies a predetermined standard. It is a figure which shows an example of the inspection-results screen which construction of a plica tube does not meet a predetermined standard. FIG. 10 is a diagram showing an example of a screen for selecting guide images for inspecting fences; FIG. 4 is a diagram showing an example of an inspection image and a guide image of a grid-shaped object on a fence; FIG. 10 is a diagram showing an example of recognition results of a lattice-shaped object; It is a figure which shows an example of the tag information of a fence. It is a figure which shows an example of the guide information of a lattice-shaped object. FIG. 10 is a diagram showing an example of an inspection result screen in which construction of a lattice-shaped object satisfies the criteria; It is a figure which shows an example of the shape of the joint part in a fence. FIG. 5 is a diagram showing an example of a guide image corresponding to the shape of joints in a fence; It is a figure which shows an example of the recognition result of a nut. FIG. 10 is a diagram showing an example of a screen in which a guide image is superimposed on an inspection image including joints in a fence; FIG. 10 is a diagram showing an example of guide information of joints; It is a figure which shows an example of the inspection-result screen which the construction of a fence joint part has satisfy|filled the reference|standard. It is a figure which shows an example of the inspection process in the joint part of a fence. FIG. 10 is a diagram showing an example of a screen for selecting a guide image for inspecting insulating bushings; 1 is an example of an inspection image and recognition results including insulating bushings; It is a figure which shows an example of the tag information of an insulating bushing. It is a figure which shows an example of the inspection process of an insulating bushing. It is a figure which shows an example of the guide information about an insulating bushing. FIG. 10 is a diagram showing an example of a screen for selecting guide images for inspecting a boundary plate; FIG. 4 is a diagram showing an example of an inspection image of a boundary plate and a base member; FIG. 10 is a diagram showing an example of a guide image of a boundary plate and a recognition result; FIG. 4 is a diagram showing an example of tag information for boundary plates and foundation members; It is a figure which shows an example of guide information. It is a figure which shows the relationship between a boundary line and a base member. FIG. 10 is a diagram showing another example of a guide image for boundary plates; FIG. 11 is a diagram showing an example of recognition results of a boundary plate and a base member; FIG. 10 is a diagram showing an example of inspection results for a boundary plate; FIG. 10 is a diagram showing an example of a screen for selecting the type of guide for empty paint cans. FIG. 10 is a diagram showing an example of a screen for setting the quantity of selected empty paint cans. FIG. 4 is a diagram showing an example of an inspection image including a paint can; FIG. 10 is a diagram showing an example of a recognition result of a paint can; It is a figure which shows an example of the inspection result of a paint can.

DESCRIPTION OF THE PREFERRED EMBODIMENTS An equipment work inspection device, an equipment work inspection method, and an equipment work inspection program to which the present invention is applied will be described below with reference to the drawings.

The equipment construction inspection system according to the embodiment uses an image captured by an on-site worker for equipment construction of a base station to inspect the equipment construction. The equipment construction inspection system of the embodiment determines whether or not the construction member to be inspected exists in the image captured by the site worker, and whether or not the construction member is properly constructed. The inspection can be performed easily and with high accuracy. In addition, the facility construction inspection system of the embodiment presents the determination result (the recognition result of the construction member and the inspection result) to the site worker, thereby reducing the time and effort of the site worker to take an image, and the facility construction inspector. It is possible to reduce the trouble of inspection and the trouble of re-construction when there is a defect in equipment construction.

FIG. 1 is a block diagram showing a configuration example of a facility construction inspection system according to an embodiment. The facility construction inspection system includes, for example, one or more user terminal devices 100, a construction inspection server device 200, and a management terminal 300. The user terminal device 100, the construction inspection server device 200, and the management terminal 300 are connected to the communication network NW, for example. Each device connected to the communication network NW has a communication interface such as a NIC (Network Interface Card) or a wireless communication module (not shown in FIG. 1). The communication network NW includes, for example, the Internet, WAN (Wide Area Network), LAN (Local Area Network), cellular network, and the like.

The user terminal device 100 is, for example, a mobile terminal device equipped with a camera device such as a smart phone or a tablet terminal. The user terminal device 100 starts a UA (User Agent) such as a browser or an application program. UA is, for example, an application for communicating with the construction inspection server device 200 . The user terminal device 100 uses a construction inspection application as a UA, and performs display processing, operation reception processing, and the like using the content received from the construction inspection server device 200 . In addition, the user terminal device 100 provides the construction inspection server device 200 with operation information based on the user's operation, captured images, and the like using the construction inspection application.

The construction inspection server device 200 is an information processing device that receives inspection requests for facility construction work and provides inspection results for the facility construction work. The construction inspection server device 200 includes, for example, a cooperative API (Application Programming Interface) section 210 and an inspection target recognition section 220 . Functional units such as the cooperative API unit 210 and the inspection target recognition unit 220 are realized by executing a program stored in a program memory by a processor such as a CPU (Central Processing Unit). Some or all of these functional units may be realized by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), or FPGA (Field-Programmable Gate Array), It may be realized by cooperation of software and hardware.

The cooperative API unit 210 includes an API unit 212, an image processing unit 214, an inspection unit 216, and a control unit 218, for example. The API unit 212 performs processing for receiving various types of information from the user terminal device 100, processing for providing content to the user terminal device 100, and the like. The image processing unit 214 performs predetermined image processing on image data acquired from the user terminal device 100 . An image used for inspection is hereinafter referred to as an inspection image. The inspection unit 216 uses the inspection image acquired by the API unit 212 and/or the inspection image processed by the image processing unit 214 to perform inspection processing for facility construction. The control unit 218 performs control such as activation of the image recognition engine in the inspection object recognition unit 220 .

The inspection target recognition unit 220 includes, for example, a plurality of image recognition engines 222A, 222B, . . . 222N (N is a natural number). In addition, when collectively referring to a plurality of image recognition engines, they are simply referred to as the image recognition engine 222 . Each image recognition engine 222 corresponds to each inspection item of the plurality of inspection items. The image recognition engine 222 performs recognition processing for recognizing construction members as inspection targets in inspection items.

FIG. 2 is a block diagram showing an example of an inspection object recognition unit. The inspection object recognition unit 220 includes, for example, a learning image database 230, a learning processing unit 232, a recognition processing unit 234, and a result output unit 236.

The learning image database 230 is a database that stores, for example, images acquired from the management terminal 300 as learning images. The learning image database 230 stores, for example, positive example images as learning images. A positive image is an image of a target object in a normal state. A positive image is, for example, an image of a construction member used in construction of a base station facility. Construction members are components of base station equipment such as plica tubes (piping), fences, insulating bushings, foundations, boundary markings, and paint cans, for example. A positive example image is associated with tag information including a construction member name. Information indicating, for example, a process name and an inspection item name may be associated with the positive example image. Note that the learning image database 230 may store not only positive images but also images of objects different from construction members (abnormal objects and defective products) as learning images.

A learning image is supplied from the learning image database 230 to the learning processing unit 232 . The learning processing unit 232 performs recognition processing using the learning image and obtains a determination result. The learning processing unit 232 learns the processing parameters of the recognition processing unit 234 (recognition model 234A) so that the result of determination is that the construction member is detected, and generates learning result data. The processing parameter of the recognition processing unit 234 (recognition model 234A) is, for example, a filter (also called weight or bias) included in the neural network. The learning result data is stored in the learning result data storage unit 2341 .

The recognition processing unit 234 has, for example, a learning result data storage unit 2341 and a determination unit 2342 . The learning result data storage unit 2341 accumulates processing parameters of the recognition processing unit 234 as learning results, and the processing parameters of the recognition processing unit 234 are updated by the learning processing unit 232 . When the inspection image is acquired, the recognition processing unit 234 extracts feature amounts using a recognition model 234A based on the processing parameters of the recognition processing unit 234 . The determination unit 2342 detects a construction member based on the extracted feature amount. The recognition processing unit 234 outputs the determination result by the determination unit 2342 as the recognition result.

FIG. 3 is a sequence diagram showing an example of inspection processing for facility construction in the facility construction inspection system. The user terminal device 100 transmits inspection item information to the construction inspection server device 200 based on the user's operation, and transmits guide information corresponding to inspection images and inspection items to the construction inspection server device 200 . Next, the construction inspection server device 200 transmits the recognition result of the inspection object included in the inspection image to the user terminal device 100 and further transmits the inspection result of the inspection object to the user terminal device 100 . As a result, for the inspection items and inspection objects specified by the user terminal device 100, it is possible to determine whether the construction inspection server device 200 has correctly recognized the inspection objects, and whether or not the construction of the inspection objects has been properly performed. has been checked or not.

FIG. 4 is a flowchart showing an example of inspection processing for facility construction in the construction inspection server device. First, the cooperative API unit 210 acquires inspection item names in facility construction based on information received from the user terminal device 100 (step S100). The cooperative API unit 210 acquires the activation list of the inspection target recognition unit 220 at the time when the inspection item name is acquired. The cooperative API unit 210 compares the inspection item name acquired in step S100 with the activation list (step S104), and determines whether or not the image recognition engine 222 corresponding to the acquired inspection item name is activated (step S106). If the image recognition engine 222 corresponding to the inspection item name has not been activated (step S106: NO), the cooperative API unit 210 controls to activate the image recognition engine 222 corresponding to the inspection item name (step S108).

If the image recognition engine 222 corresponding to the inspection item name is activated (step S106: YES), or after the image recognition engine 222 corresponding to the inspection item name is activated in step S108, the cooperative API unit 210 extracts the image file And an inspection information file is obtained (step S110). Next, the cooperative API unit 210 performs predetermined image processing on the inspection image included in the acquired image file (step S112). The predetermined image processing includes, for example, edging (sharpening) processing and brightness correction processing. The user terminal device 100 transfers the inspection image that has undergone predetermined image processing to the image recognition engine 222 corresponding to the inspection item name.

The image recognition engine 222 performs recognition processing for determining whether or not there is a construction member (inspection target) corresponding to the inspection item name based on the image acquired from the cooperative API unit 210 (step S114). At this time, the image recognition engine 222 calculates a certainty factor for each construction member. In addition, the image recognition engine 222 calculates the reliability of the construction member for each image processing. The image recognition engine 222 determines that the construction member exists when the reliability of the construction member is equal to or greater than a predetermined value. When the image recognition engine 222 recognizes that there is a construction member (step S116: YES), the process proceeds to step S120. When the image recognition engine 222 recognizes that there is no construction member (step S116: NO), the image recognition engine 222 acquires the inspection image again (step S118). When reacquiring the inspection image, the construction inspection server device 200 transmits a request to recapture the image to the user terminal device 100 .

The cooperative API unit 210 compares the reliability of the construction member calculated by the image recognition engine 222 with a predetermined value (step S120). The cooperative API unit 210 performs inspection item determination processing using inspection images in which construction members are recognized with a certainty higher than a predetermined value (step S122). Note that the inspection item determination process will be described later. The cooperative API unit 210 transmits the inspection result to the user terminal device 100 as a result of inspection item determination processing (step S124).

(Inspection of plica tube (piping))
The inspection of plica tubes (pipes) in base station equipment will be described below.
FIG. 5 is a diagram showing an example of a screen for selecting an inspection object. The user terminal device 100 displays the inspection object selection screen shown in FIG. 5 based on the user's operation, and recognizes which inspection object has been selected by accepting the operation on the selection screen. The user terminal device 100 displays the screen shown in FIG. 6 when the Plicatube is selected. FIG. 6 is a diagram showing an example of a screen for selecting a guide image for inspecting the plica tube. In the embodiment, for example, guide images corresponding to four patterns can be selected as the specifications of the plica tube. The specification of the plica tube is, for example, the diameter of the plica tube, and the user selects the diameter of the plica tube to be constructed. As a result, the equipment construction inspection system can inspect the bending radius corresponding to the diameter of the constructed plica tube.

FIG. 7 is a diagram showing an example of an inspection image of the plica tube and a recognition result of the plica tube, FIG. 8 is a diagram showing an example of tag information of the plica tube, and FIG. 9 is a guide of the plica tube. It is a figure which shows an example of an image, and FIG. 10 is a figure which shows an example of the guide information about a plica tube. The user terminal device 100 takes an inspection image shown in FIG. 7A and transmits it to the construction inspection server device 200 . The construction inspection server device 200 performs a plica tube recognition process on the inspection image, and gives tag information #1 to the area in the inspection image recognized as the plica tube shown in FIG. 7B. The tag information includes the tag name “u-tube” as the recognition result of the image recognition engine 222. FIG. In addition, the construction inspection server device 200 may generate a recognition result that recognition of the plica tubes has failed when a plurality of plica tubes are recognized in one inspection image.

The user terminal device 100 displays the guide image of the plica tube shown in FIG. 9, and transmits guide information to the construction inspection server device 200, for example, when an operation for determining the position of the guide image is received. The guide images are the inner diameter side guide (first image) arranged on the inner diameter side of the plica tube, the outer diameter side guide (second image) arranged on the outer diameter side of the plica tube, the inner diameter side guide and the outer diameter side and a pair of positioning guides (third image) that overlap the side guides. The inner diameter side guide and the outer diameter side guide have a shape corresponding to the bending radius (specification) of the plica tube selected by the user (constructor). The positioning guides are a pair of rectangular areas separated by the distance between the inner and outer diameters of the selected plica tube. When the user terminal device 100 receives an operation in which the guide image is positioned, the user terminal device 100 transmits transmission information indicating the center coordinates (x, y) of the inner diameter side guide (guide circle) and the radius (r) of the guide circle to the construction inspection server. Send to device 200 .

FIG. 11 is a diagram illustrating an example of inspection processing for a plica tube. The construction inspection server device 200 uses the center coordinates C (x, y) of the inner diameter guide (guide circle) and the radius r of the inner diameter guide received from the user terminal device 100 to reproduce the inner diameter guide in the inspection image. do. The construction inspection server device 200 determines that the construction of the Plicatube is properly performed when the reproduced inner diameter side guide does not overlap the area recognized as the Plicatube, and the reproduced inner diameter side guide is the Plicatube. If it overlaps with the area recognized as , it is determined that the plica tube is not properly constructed.

FIG. 12 is a diagram showing another example of the plica tube inspection process. The construction inspection server device 200 may perform inspection by excluding a part of the range in which the reproduced inner diameter side guide and the area recognized as the plica tube are adjacent to each other. The construction inspection server device 200 sets, for example, a range that is a predetermined distance below the center coordinate C of the inner diameter side guide as an exclusion range. The predetermined distance is, for example, a length obtained by multiplying the radius r by a predetermined set value (eg, 0.7). Outside the exclusion range, the construction inspection server device 200 determines that the plica tube is properly constructed when the reproduced inner diameter side guide does not overlap the area recognized as the plica tube, and reproduces it. If the inner diameter side guide overlaps the area recognized as the plica tube, it is determined that the plica tube is not properly constructed. Although the range below the distance obtained by multiplying the radius r by 0.7 from the central coordinate C of the inner diameter guide was set as the exclusion range, it is not limited to this. The exclusion range may be changed if it is possible to reduce the possibility of

FIG. 13 is a diagram showing an example of an inspection result screen in which construction of the Plicatube satisfies the predetermined standard, and FIG. 14 shows an example of an inspection result screen in which the construction of the Plicatube does not satisfy the predetermined standard. It is a diagram. The construction inspection server device 200 transmits the inspection result to the user terminal device 100, and the user terminal device 100 displays the inspection result screen shown in FIG. 13 or 14 based on the inspection result.

(Fence Inspection)
The inspection of fences in base station equipment will be described below.
The user terminal device 100 displays the screen shown in FIG. 15 when the fence is selected as the inspection object. FIG. 15 is a diagram showing an example of a screen for selecting guide images for inspecting fences. In the embodiment, it is possible to select a guide image corresponding to, for example, the location of a lattice-shaped object on a fence or the shape of a joint on a fence. As a result, the equipment construction inspection system can inspect the fence corresponding to the location of the constructed fence and the shape of the joint.

16 is a diagram showing an example of an inspection image and a guide image of a lattice-shaped object on a fence, FIG. 17 is a diagram showing an example of a recognition result of a lattice-shaped object, and FIG. 18 is a diagram showing tag information of a fence. FIG. 19 is a diagram showing an example, and FIG. 19 is a diagram showing an example of guide information of a grid-shaped object. The user terminal device 100 captures an inspection image including a lattice-shaped object as shown in FIG. 16 and transmits the inspection image to the construction inspection server device 200 . The construction inspection server device 200 performs grid-shaped object recognition processing on the inspection image, and adds tag information (#1 or #2) to the region in the inspection image recognized as the grid-shaped object shown in FIG. do. The tag information includes the tag name “chidori-gin” or “chidori-cha” as the recognition result of the image recognition engine 222 .

The user terminal device 100 displays the guide image of the lattice-shaped object shown in FIG. 16, and transmits guide information to the construction inspection server device 200 when, for example, an operation to determine the position of the guide image is received. The guide image includes a plurality of circular images arranged at predetermined intervals in one direction. Each circular image area corresponds to an inspection area that is inspected to see if it contains a grid-shaped object. When the user terminal device 100 receives an operation in which the guide image is positioned, the user terminal device 100 transmits transmission information indicating the center coordinates (x1, y1, x2, y2, . . . ) and the radius (r) of each circular image. Send to server device 200 .

The construction inspection server device 200 uses the guide information to inspect whether or not the number of recognized lattice-shaped objects is the same as the number of circular images (inspection areas). In the construction inspection server device 200, the number of guide images is four as shown in FIG. A test result is generated that the construction of the feature meets predetermined criteria.

FIG. 20 is a diagram showing an example of an inspection result screen in which construction of a grid-shaped object satisfies the criteria. The construction inspection server device 200 transmits the inspection result to the user terminal device 100, and the user terminal device 100 displays the inspection result screen shown in FIG. 20 based on the inspection result.

Next, inspection of joints of fences will be described.
FIG. 21 is a diagram showing an example of the shape of joints in the fence, and FIG. 22 is a diagram showing an example of a guide image corresponding to the shape of the joints in the fence. As shown in FIG. 21, there are a plurality of types of joint shapes in the fence, and one of them is selected based on the user's operation. The user terminal device 100 transmits information indicating the shape of the joint in the selected fence to the construction inspection server device 200 as inspection item information, and further captures an inspection image including the joint and sends it to the construction inspection server device 200. Send. The construction inspection server device 200 performs a nut (fastening) recognition process on the inspection image, and adds tag information (#3 or #4 in FIG. 18) to the area in the inspection image recognized as the nut. to give The tag information includes the tag name “nut-gin” or “nut-cha” as the recognition result of the image recognition engine 222 . FIG. 23 is a diagram illustrating an example of a nut recognition result.

FIG. 24 is a diagram showing an example of a screen in which a guide image is superimposed on an inspection image including joints in a fence, and FIG. 25 is a diagram showing an example of guide information on joints. The guide images include a post guide image having a shape corresponding to the post that supports the lattice-shaped object of the fence, and a nut guide image corresponding to the nut (fastener) that fastens the posts together. The user terminal device 100 transmits transmission information indicating the center coordinates (x1, y1, x2, y2, . Send to server device 200 .

The construction inspection server device 200 uses the guide information to inspect whether or not the number of nuts (fasteners) recognized from the inspection image is the same as the number of circular images (inspection areas). When the number of nut guide images is two as shown in FIG. produce test results that meet the criteria for The construction inspection server device 200 transmits the inspection result to the user terminal device 100, and the user terminal device 100 displays an inspection result screen based on the inspection result.

FIG. 26 is a diagram showing an example of an inspection result screen in which construction of fence joints satisfies the criteria. The construction inspection server device 200 transmits the inspection result to the user terminal device 100, and the user terminal device 100 displays the inspection result screen shown in FIG. 26 based on the inspection result.

FIG. 27 is a diagram illustrating an example of inspection processing at joints of fences. The construction inspection server device 200 reproduces the nut guide image in the inspection image using the center coordinates C (x, y) of the nut guide image and the radius r of the inner diameter side guide received from the user terminal device 100 . If the reproduced nut guide image contains the barycentric coordinates of the region recognized as one or more nuts, the construction inspection server device 200 determines that the nut in the reproduced nut guide image satisfies a predetermined standard. I judge. The construction inspection server device 200 generates an inspection result of the joint portion of the fence by performing determination for each reproduced nut guide image.

The inspection of insulating bushings will be described below. Insulating bushings are mounted, for example, with bolts at the joints. FIG. 28 is a diagram showing an example of a screen for selecting a guide image for inspecting insulating bushings, FIG. 29 is an example of an inspection image including insulating bushings and recognition results, and FIG. It is a figure which shows an example of the tag information of a bushing.

The user terminal device 100 displays the screen shown in FIG. 28 when the insulating bushing is selected as the inspection object. For example, when “insulated bolt side” or “insulated nut side” is selected, the user terminal device 100 displays an inspection image including an insulating bushing as shown in FIG. Send to server device 200 . The construction inspection server device 200 performs insulation bushing recognition processing on the inspection image, and adds tag information (#2) to the area in the inspection image recognized as the insulation bushing as shown in FIG. 29(B). do. The tag information includes the tag name “bush-black” or “bush-white” as the recognition result of the image recognition engine 222 . In addition, in the embodiment, when an item other than "bolt side with insulation" and "nut side with insulation" is selected, the user terminal device 100 does not have to transmit the inspection item information to the construction inspection server device 200. The recognition processing and inspection processing by the inspection server device 200 may not be performed.

FIG. 31 is a diagram showing an example of inspection processing for insulating bushings, and FIG. 32 is a diagram showing an example of guide information for insulating bushings. The user terminal device 100 generates information representing the coordinate range (x1, y1, x2, y2) where the insulating bushing exists in the inspection image based on the user's operation, for example, and transmits the information to the construction inspection server device 200 . The installation inspection server device 200 determines that the installation of the insulation bushing satisfies a predetermined standard when one or more barycentric coordinates of the region recognized as the insulation bushing exist in the coordinate range received from the user terminal device 100. Generate test results.

(Inspection of Boundary Markings)
The inspection of boundary markings in base station equipment will be described below.
The user terminal device 100 displays the inspection object selection screen shown in FIG. 5 based on the user's operation, and recognizes which inspection object has been selected by accepting the operation on the selection screen. The user terminal device 100 displays the screen shown in FIG. 33 when the boundary plate (boundary marker) is selected. FIG. 33 is a diagram showing an example of a screen for selecting a guide image for inspecting the boundary plate. In the embodiment, for example, guide images corresponding to three composition patterns can be selected as the relationship between the boundary plate and the base member. The composition patterns are called, for example, "horizontal,""perpendicular," and "reverse perpendicular," and the user selects the constructed composition pattern. As a result, the facility construction inspection system can inspect the boundary plate corresponding to the composition pattern.

34 is a diagram showing an example of inspection images of the boundary plate and the base member, FIG. 35 is a diagram showing an example of the guide image and the recognition result of the boundary plate, and FIG. 36 is a diagram showing the boundary plate and the base member. FIG. 37 is a diagram showing an example of tag information, and FIG. 37 is a diagram showing an example of guide information. The user terminal device 100 takes an inspection image shown in FIG. 34 and transmits it to the construction inspection server device 200 . The construction inspection server device 200 performs boundary plate and base member recognition processing on the inspection image, and adds tag information #1 to the area in the inspection image recognized as the boundary plate shown in FIG. The tag information includes the tag name “pura” as the recognition result of the image recognition engine 222 .

The user terminal device 100 displays the guide image of the boundary plate, and transmits guide information to the construction inspection server device 200, for example, when an operation for determining the direction of the guide image is received. The guide images include a guide image corresponding to the boundary plate and a guide image corresponding to two boundary lines extending from the boundary plate. The relationship between the guide image corresponding to the boundary plate and the guide image corresponding to the boundary line is determined by the direction selected by the user, and the direction information selected by the user is transmitted to the construction inspection server device 200 as guide information. be. For example, when the user selects "right angle" and left, the user terminal device 100 displays the guide image shown in FIG. For example, when the user selects "right angle" and "right", the user terminal device 100 displays the guide image shown in FIG. The boundary line parallel to the X direction in FIG. 35A is located at coordinate y1, which is 1/3 from the lower end in the Y direction, and the boundary line parallel to the Y direction is located at coordinate x1, which is 2/3 from the right end in the X direction. Located in The boundary line parallel to the X direction in FIG. 35B is located at coordinate y1, which is 1/3 from the lower end in the Y direction, and the boundary line parallel to the Y direction is located at coordinate x2, which is 1/3 from the right end in the X direction. Located in

When a right-angled or reverse-right-angled guide image is selected, the construction inspection server device 200 determines that the boundary plate exists at a predetermined location when the coordinates at which the boundary lines intersect overlap with the area recognized as the boundary plate. I judge. When the horizontal guide image is selected, the construction inspection server device 200 determines that the boundary plate exists at the predetermined location when the boundary line and the region recognized as the boundary plate intersect. When the construction inspection server device 200 determines that the boundary plate does not exist at the predetermined location, the construction inspection server device 200 notifies the user terminal device 100 that the boundary plate could not be detected.

FIG. 38 is a diagram showing the relationship between boundary lines and base members. The construction inspection server device 200 determines the positional relationship between the foundation member and the boundary line based on the selected composition pattern and orientation information. As shown in FIG. 38A, when "right angle" is selected as the composition pattern and the orientation is left, the construction inspection server device 200 determines that the maximum value x12 in the X direction of the foundation member is the boundary line parallel to the Y axis. is smaller than the X-direction position x11 of the base member, and the maximum value y12 in the Y-direction of the base member is smaller than the Y-direction position y11 of the boundary line parallel to the X-axis, the positional relationship between the boundary plate and the base member is considered appropriate. judge. When the construction inspection server device 200 selects "horizontal" as the composition pattern and the orientation is left as shown in FIG. is smaller than the Y-direction position y11, it is determined that the positional relationship between the boundary plate and the base member is appropriate. As shown in FIG. 38(C), the construction inspection server device 200 selects the "reverse right angle" as the composition pattern and the direction is left, the intersection of the two boundary lines and the maximum value of X and Y If the base member does not exist inside the rectangle with the point (lower right point in the image) as the diagonal line, it is determined that the positional relationship between the boundary plate and the base member is appropriate.

In the construction inspection server device 200, as shown in FIG. 38(D), when "right angle" is selected as the composition pattern and the direction is right, the minimum value x22 in the X direction of the foundation member is the boundary line parallel to the Y axis. is smaller than the X-direction position x21 of the base member, and the maximum value y22 in the Y-direction of the base member is smaller than the Y-direction position y21 of the boundary line parallel to the X-axis, the positional relationship between the boundary plate and the base member is considered appropriate. judge. In the construction inspection server device 200, as shown in FIG. 38(E), when "horizontal" is selected as the composition pattern and the orientation is right, the maximum value y22 in the Y direction of the foundation member is a boundary line parallel to the X axis. is smaller than the Y-direction position y21, it is determined that the positional relationship between the boundary plate and the base member is appropriate. As shown in FIG. 38(F), the construction inspection server device 200 selects the "reverse right angle" as the composition pattern and the orientation is right. If the base member does not exist within the rectangle with the diagonal line of the point (lower left point in the image), it is determined that the positional relationship between the boundary plate and the base member is appropriate.

39 is a diagram showing another example of the guide image for the boundary plate, FIG. 40 is a diagram showing an example of the recognition result of the boundary plate and the base member, and FIG. 41 is a diagram showing the inspection result of the boundary plate. It is a figure which shows an example. As described above, the user terminal device 100 selects a composition pattern, superimposes a guide image including a boundary line on the inspection image, and displays the direction information of the guide image, and transmits the direction information to the construction inspection server device 200 . You can The construction inspection server device 200 recognizes the area of the boundary plate and the area of the foundation member by recognizing the inspection image, and based on the composition pattern and orientation information, the construction inspection server device 200 determines the boundary near the boundary line photographed by the user according to the guide image. It can be determined that a plate exists and that no foundation member exists beyond the boundary line based on the boundary plate. As a result, the construction inspection server device 200 transmits the inspection result to the user terminal device 100, and the user terminal device 100 displays the inspection result screen shown in FIG. 41 based on the inspection result.

(Inspection of paint cans)
The inspection of paint cans in base station equipment will be described below. FIG. 42 is a diagram showing an example of a screen for selecting the type of guide for empty paint cans, FIG. 43 is a diagram showing an example of a screen for setting the quantity of the selected empty paint cans, and FIG. 45 is a diagram showing an example of a paint can recognition result, and FIG. 46 is a diagram showing an example of a paint can inspection result. The user terminal device 100 displays the screen of FIG. 42 to select the application of the paint cans used for constructing the base station, and displays the screen of FIG. 43 to select the required number of paint cans. The usage and required quantity of paint cans are transmitted to the construction inspection server device 200 as guide information. Further, the user terminal device 100 captures an inspection image and transmits it to the construction inspection server device 200 as shown in FIG. 44 . The application inspection server device 200 recognizes the paint can as shown in FIG. 45, and if the application using the paint can satisfies a predetermined standard based on the guide information and the recognition result, an inspection as shown in FIG. View results.

(Effect of Embodiment)
According to the embodiment described above, from the user terminal device 100, the inspection item information indicating that the inspection item is piping, the inspection image obtained by imaging the piping installed by bending, and the piping included in the inspection image. An API unit 212 that acquires guide information related to a guide image for performing the inspection, calculates a degree of certainty indicating the certainty of the pipe based on the inspection image, and recognizes whether or not the inspection image includes the pipe based on the degree of certainty. an inspection target recognizing unit 220 that performs inspection, an inspection unit 216 that inspects the facility construction based on the area recognized as piping and the guide information, and the result of the inspection performed by the inspection unit 216 is notified to the user terminal device 100 It is possible to realize an inspection device for facility construction, which includes the cooperative API unit 210 . According to the embodiment, it is possible to easily and highly accurately inspect equipment construction related to piping that has been installed by bending.

According to the embodiment, from the user terminal device 100, the inspection item information indicating that the inspection item is a component of the fence, the inspection image obtained by capturing the component of the fence, and the component of the fence included in the inspection image. An API unit 212 that acquires guide information related to a guide image for performing inspection, calculates a certainty factor that indicates the likelihood of a fence component based on the inspection image, and calculates the certainty factor that indicates whether the fence component is included in the inspection image based on the certainty factor. an inspection object recognizing unit 220 that recognizes whether or not the and an API unit 212 that notifies the user terminal device 100 of the results of the inspection that has been performed. According to the embodiment, it is possible to easily and highly accurately inspect facility work related to lattice-shaped objects, fasteners, and insulating bushings as components of a fence.

According to the embodiment, from the user terminal device 100, inspection item information indicating that the inspection item is a boundary marking object, an inspection image obtained by capturing the boundary marking object and a part of the equipment, and the boundary marking object included in the inspection image. and an API unit 212 for acquiring guide information related to a guide image for imaging a boundary marking, calculating a degree of certainty indicating the likelihood of the boundary marking based on the inspection image, and calculating the degree of certainty that the boundary marking is included in the inspection image based on the degree of certainty. an inspection target recognition unit 220 that recognizes whether or not the facility is included in the inspection image, calculates a certainty factor indicating the certainty of the facility based on the inspection image, and recognizes whether or not the facility is included in the inspection image based on the certainty factor; An inspection unit 216 that inspects facility construction based on the positional relationship between the boundary line extending from the area recognized as the boundary marker based on the orientation information of the inspection image and the area recognized as the facility, and an inspection unit. and an API unit 212 that notifies the user terminal device 100 of the results of the inspection performed by the API unit 216. According to the embodiment, it is possible to easily and highly accurately inspect facility work related to boundary markers and facilities.

Although each embodiment and modifications have been described, these are only examples and are not limited to these. A section may be combined with one or more other embodiments or one or more other modifications to realize one aspect of the present invention.

A program for executing each process of the user terminal device 100 and the construction inspection server device 200 in this embodiment is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read by the computer system. , the above-described various processes related to the user terminal device 100 and the construction inspection server device 200 may be performed.

Note that the “computer system” referred to here may include hardware such as an OS and peripheral devices. The "computer system" also includes the home page providing environment (or display environment) if the WWW system is used. In addition, "computer-readable recording medium" means writable non-volatile memory such as flexible disk, magneto-optical disk, ROM, flash memory, portable medium such as CD-ROM, hard disk built in computer system, etc. storage device.

Furthermore, "computer-readable recording medium" means a volatile memory (e.g., DRAM (Dynamic
Random Access Memory)), which holds a program for a certain period of time. Also, the program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium.

Here, the "transmission medium" for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the program may be for realizing part of the functions described above. Further, it may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

As described above, the embodiments of the present invention have been described in detail with reference to the drawings, but the specific configuration is not limited to these embodiments, and includes designs within the scope of the gist of the present invention. For example, as an embodiment of the present invention, the user terminal device 100 may be used alone.

100 User terminal device 200 Construction inspection server device 210 Linkage API unit 212 API unit 214 Image processing unit 216 Inspection unit 218 Control unit 220 Inspection object recognition unit 222 Image recognition engine

Claims (7)

From the terminal device, inspection item information indicating that the inspection item is a component of the fence, an inspection image obtained by imaging the component of the fence, and a guide image for capturing the component of the fence included in the inspection image. an acquisition unit that acquires guide information about
a recognition unit that calculates a certainty factor indicating the certainty of the structure of the fence based on the inspection image, and recognizes whether or not the inspection image includes the structure of the fence based on the certainty factor;
an inspection unit that inspects whether or not the number of constituents of the fence based on the guide information is the same as the number of the recognized constituents of the fence;
a notification unit that notifies the terminal device of the result of the inspection performed by the inspection unit;
Equipment construction inspection device. The constituent of the fence is a lattice-shaped object of the fence, and the guide image has an area corresponding to each of a predetermined number of lattice-shaped objects,
The facility construction inspection device according to claim 1, comprising: The constituents of the fence are fasteners of the fence, and the guide image has an area corresponding to each of a predetermined number of fasteners.
The facility construction inspection device according to claim 1, comprising: 4. The facility construction inspection device according to claim 3, wherein said guide image is one guide image selected by a builder from among a plurality of guide images with different numbers of said fasteners. the fence fastener includes an insulating portion;
The acquisition unit acquires inspection item information indicating that the inspection item is the insulating portion, an inspection image obtained by imaging the insulating portion, and guide information related to a guide image for imaging the insulating portion included in the inspection image. Acquired,
The recognition unit calculates a certainty factor indicating the likelihood of the insulating part, and recognizes whether or not the inspection image includes the insulating part based on the certainty factor,
The inspection unit inspects whether or not the insulating portion is included in the inspection image based on the guide information.
The equipment construction inspection device according to claim 1 . From the terminal device, inspection item information indicating that the inspection item is a component of the fence, an inspection image obtained by imaging the component of the fence, and a guide image for capturing the component of the fence included in the inspection image. obtaining guide information about
a step of calculating a certainty factor indicating the certainty of the constituent of the fence based on the inspection image, and recognizing whether or not the constituent of the fence is included in the inspection image based on the certainty factor;
checking whether the number of the fence components based on the guide information and the recognized number of the fence components are the same;
a step of notifying the terminal device of the result of the inspection;
A method for inspecting equipment construction. to the computer,
From the terminal device, inspection item information indicating that the inspection item is a component of the fence, an inspection image obtained by imaging the component of the fence, and a guide image for capturing the component of the fence included in the inspection image. Get guide information about
calculating a certainty factor indicating the certainty of the constituents of the fence based on the inspection image, and recognizing whether or not the inspection image includes the constituents of the fence based on the certainty factor;
inspecting whether or not the number of constituents of the fence based on the guide information and the number of the recognized constituents of the fence are the same;
Notifying the terminal device of the result of the inspection,
A program for inspection of facility construction.

Images