JP7237026B2 - Building inspection method, information processing device, terminal device, program, and building inspection system - Google Patents

Description

[1] Date May 23, 2019 Name of meeting “1st Shimizu Corporation 'Reinforcement Joint' AI Image Analysis Status Report Meeting” <Reference> 2019 May 23 AI image analysis status report meeting Verification result report material

Application for application of Article 30, Paragraph 2 of the Patent Law has been filed [2] Date: October 2, 2019 Meeting name: “2nd Shimizu Corporation 'Reinforcement Joint' AI Image Analysis Status Report Meeting” <Reference> 2019 October 2 AI image analysis status report meeting Verification result report material

Patent Act Article 30, Paragraph 2 application filed [3] Date of publication of website November 21, 2019 Website address https://www. nttcom. co. jp/news/pr19112101. html https://cloud. watch. impress. co. jp/docs/news/1219967. html https://built. it media. co. jp/bt/articles/1911/22/news054. html https://news. my navi. jp/article/20191121-926766/ https://www. sankeibiz. jp/business/news/191122/bsj1911220500002-n1. http https://www. sankei. com/economy/news/191121/ecn1911210018-n1. html https://www. Decn. co. jp/? p=111077 https://active. nikkeibp. co. jp/atcl/r/19/RSP523850_21112019/https://xtech. nikkei. com/atcl/nxt/column/18/00154/00737/https://www. nikkei. com/article/DGXLRSP523850_R21C19A1000000/https://release. nikkei. co. jp/attach_file/0523850_01. pdf https://it. impress bm. co. jp/articles/-/18883 https://ieiri-lab. jp/it/2019/11/rebar-joint-inspection-by-ai. html https://www. sugitec. net/23680/https://feedclass. com/posts/KWmfjBkH <Reference> Press release web page printout <Reference> Press release web page printout

[4] Date November 21-22, 2019 Meeting name “NTT COMWARE’S DAY 2019” <Reference> NTT COMWARE’S DAY 2019 presentation

Application for application of Article 30, Paragraph 2 of the Patent Act has been filed [5] Experiment date January 2020 to March 2020 Experiment name Verification by field trial demonstration experiment of "rebar joint AI inspection"

The present invention relates to a building inspection method, an information processing device, a terminal device, a program, and a building inspection system.

2. Description of the Related Art Conventionally, a reinforcing bar inspection support device capable of supporting the inspection of reinforcing bars has been known, as disclosed in Patent Literature 1 below. This reinforcing bar inspection support device acquires a reinforcing bar image, which is an image of a reinforcing bar at a construction site, and based on the acquired reinforcing bar image and a pre-generated model for outputting inspection information of the reinforcing bar shown in the image. Output the inspection information of the image.

JP 2019-114146 A

However, since the above-described reinforcing bar inspection support apparatus uses a trained model (a trained model for inspection) for inspection, there is a possibility that the amount of learning of the trained model for inspection may reduce the accuracy of inspection. be. In addition, since the above-described reinforcing bar inspection support device determines the position in the image where the pressure hump of the reinforcing bar exists by the learned model (learned model for determination), there is a possibility that the position of the pressure hump cannot be specified accurately. There is Furthermore, in the above-described reinforcing bar inspection support apparatus, when the position in the image where the pressure hump of the reinforcing bar exists is determined by the learned model for determination, if the resolution of the partial image where the pressure hump exists is low, the inspection can be performed with high accuracy. may not be able to

The present invention has been made in view of the above problems, and provides a building inspection method, an information processing device, a terminal device, a program, and a building inspection system that can inspect buildings with high accuracy. intended to provide.

(1) In one aspect of the present invention, an image capturing step of capturing an image of a building, and calculating a first certainty factor indicating the likelihood of the entire building included in a captured image that is an image of the building. a step of determining whether or not the entire building is included based on the first certainty; calculating the degree of confidence, and determining whether or not each component part of the building is included based on each of the second degrees of certainty; and and an inspection step of performing an inspection based on an image showing the component when it is determined that the component of the building is included.

(2) An aspect of the present invention is the above-described building inspection method, wherein the imaging step may display a guide image based on the shape of the building by superimposing it on the captured image.

(3) An aspect of the present invention is the above-described building inspection method, wherein the determination step uses a recognition model for the entire building and the plurality of component parts to make a determination, and is a recognition model trained using learning images including the building, and processing parameters for recognizing that the entire building and the plurality of constituent parts exist when the photographed image is input may be learned.

(4) An aspect of the present invention is the above-described building inspection method, wherein the image representing the component is an image representing an area recognized as the component in the photographed image by the recognition model. It's okay.

(5) An aspect of the present invention is the above-described building inspection method, wherein the plurality of components include a first reinforcing bar, a second reinforcing bar, and a connection between the first reinforcing bar and the second reinforcing bar. and a connected portion.

(6) An aspect of the present invention is the above-described building inspection method, wherein the inspection step uses an image showing the component part as parameters for inspection, and sets the first reinforcing bar and the second reinforcing bar as parameters for inspection. thickness of the reinforcing bar, thickness of the connecting portion, length of the connecting portion, eccentricity of the first reinforcing bar and the second reinforcing bar, bending angle of the first reinforcing bar and the second reinforcing bar, swelling of the connecting portion may be calculated at least one of the widths of , and an inspection using the calculated parameters may be performed.

(7) One aspect of the present invention is the above-described building inspection method, which includes a display step of displaying inspection results obtained by the inspection step, wherein the display step displays the one or more parameters for each of the displaying a plurality of icon images corresponding to each of the corresponding inspection items and the overall inspection result of the one or more inspection items; and receiving an operation of selecting one of the plurality of icon images. an image showing the parameters of the inspection item corresponding to the selected icon image, an image showing inspection details using the parameters of the inspection item corresponding to the selected icon image, and an inspection corresponding to the selected icon image. and displaying the inspection results of the item.

(8) An aspect of the present invention is the above-described building inspection method, wherein the display step displays an icon image corresponding to an inspection item determined to be abnormal by the inspection step among the plurality of inspection items. A display form and a display form of an icon image corresponding to an inspection item determined to be normal in the inspection step among the plurality of inspection items may be displayed in different display forms.

(9) In one aspect of the present invention, a first certainty factor indicating the likelihood of the entire building included in a captured image that is an image of a building is calculated, and based on the first certainty factor, the a first determination unit for determining whether or not the entire building is included; a second determination unit that determines whether each component part of the building is included based on each of the second degrees of certainty; and the entire building and the and an inspection unit that performs an inspection based on an image showing the constituent parts when it is determined that the plurality of constituent parts are included.

(10) In one aspect of the present invention, the computer of the information processing device calculates a first certainty factor indicating the likelihood of the entire building included in an inspection image that is an image of the building, and a step of determining whether or not the entire building is included based on one degree of certainty; and determining whether or not each component part of the building is included based on each of the second degrees of confidence; and determining that the entire building and the plurality of component parts are included. and performing an inspection based on an image showing the component when the component is detected.

(11) One aspect of the present invention includes a display unit that displays a guide image based on a shape of a building, and an imaging unit that captures an image of the building while the guide image is displayed on the display unit. ,
The display unit displays a plurality of icon images corresponding to inspection items respectively corresponding to one or more parameters calculated based on inspection images captured by the imaging unit, and the one or more inspection items. When an operation to select one of the plurality of icon images is accepted, an image showing parameters of an inspection item corresponding to the selected icon image and the selected icon image and an inspection result of the inspection item corresponding to the selected icon image.

(12) One aspect of the present invention includes a display step of displaying a guide image based on the shape of a building on a computer of a terminal device; wherein the display step includes displaying a plurality of icons corresponding to inspection items respectively corresponding to one or more parameters calculated based on an inspection image captured by an imaging unit. When an image and an overall inspection result of the one or more inspection items are displayed, and an operation of selecting one of the plurality of icon images is accepted, parameters of the inspection item corresponding to the selected icon image. , an image showing inspection details using the parameter of the inspection item corresponding to the selected icon image, and the inspection result of the inspection item corresponding to the selected icon image.

(13) One aspect of the present invention is a terminal device that includes an imaging unit that captures an image of a building; an image that captures the building is received from the terminal device; a first determination unit that calculates a first degree of certainty indicating the likelihood of the entire building being included, and determines whether or not the entire building is included based on the first degree of certainty; A second degree of certainty indicating the likelihood of each of the plurality of structural parts of the building included in the image is calculated, and whether each of the structural parts of the building is included based on each of the second degrees of certainty. and a second determination unit that determines whether or not an image showing the components is included when the first determination unit and the second determination unit determine that the entire building and the plurality of component parts are included. and an information processing device that transmits the inspection result of the inspection unit to the terminal device.

According to one aspect of the present invention, a building can be inspected with high accuracy.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows one structural example of the building inspection system of embodiment. 4 is a block diagram showing an example of an inspection object recognition unit; FIG. It is a figure which shows an example of the processing procedure of the building inspection system in embodiment. FIG. 4 is a diagram showing an example of a guide image; FIG. FIG. 4 is a diagram showing an image in which an image (inspection image) recognized by an image recognition engine is superimposed on a photographed image; FIG. 4 is a diagram showing an example of tag information recognized by an image recognition engine; FIG. 4 is a diagram for explaining the process of calculating the thickness d of the upper reinforcing bar and the lower reinforcing bar; (A) is a diagram showing the center line of the upper reinforcing bar and the lower reinforcing bar; FIG. 10 is a diagram for explaining processing for calculating height; FIG. 4 is a diagram for explaining a process of calculating a diameter D of a pressure contact portion; FIG. FIG. 10 is a diagram for explaining a process of calculating length L (distance in Y direction) of a pressure contact portion; It is a figure for demonstrating the process which calculates the eccentricity amount e of an upper reinforcement and a lower reinforcement. It is a figure for demonstrating the process which calculates the bending angle (theta) of an upper reinforcement and a lower reinforcement. FIG. 10 is a diagram for explaining a process of calculating a width Δh of the one-sided bulge in the pressure contact portion; It is a figure which shows an example of the screen which presents the inspection result about the whole reinforcing bar. FIG. 10 is a diagram showing an example of an inspection result screen of the diameter D of the pressure contact portion; FIG. 10 is a diagram showing an example of an inspection result screen of the length L of the pressure contact portion; FIG. 10 is a diagram showing another example of an inspection result image of the length L of the pressure contact portion; FIG. 10 is a diagram showing an example of an inspection result screen of the width Δh of the one-sided bulge in the pressure contact portion; It is a figure which shows an example of the inspection result screen of the eccentricity amount e of an upper reinforcement and a lower reinforcement. It is a figure which shows an example of the inspection result screen of the bending angle (theta) of upper reinforcement and lower reinforcement. It is a figure which shows an example of the calculation result screen of the thickness d of a reinforcing bar. It is a figure which shows an example of an inspection result screen when there is abnormality in an inspection result. FIG. 10 is a diagram showing another example of an inspection result screen of the diameter D of the pressure contact portion; FIG. 10 is a diagram showing another example of an inspection result screen of the length L of the pressure contact portion;

Hereinafter, a building inspection method, an information processing device, a terminal device, a program, and a building inspection system to which the present invention is applied will be described with reference to the drawings.

The building inspection system of the embodiment inspects the building using, for example, an image captured by a building site worker. The building inspection system of the embodiment determines whether or not a building to be inspected exists in a photographed image and whether or not the building is properly constructed, thereby inspecting the building with high accuracy. It is to be inspected. In addition, the building inspection system of the embodiment presents the inspection results, etc. to the site worker, thereby reducing the time and effort of the site worker to take images, the time and effort of the inspector to perform the inspection, and the time and effort of reporting the inspection. be able to.

In the embodiment, a building in which two reinforcing bars are pressed together is targeted for inspection, but the present invention is not limited to inspection of a building in which two reinforcing bars are pressed together. For example, the present invention can be applied to both a building and an inspection object in which a plurality of components are combined. For example, the present invention can be applied to inspect a building in which two components are connected by a connecting member, and to inspect the positional relationship and dimensions between the two components and the connecting member.

<Configuration of building inspection system>
FIG. 1 is a block diagram showing a configuration example of a building inspection system according to an embodiment. The building inspection system includes, for example, one or more user terminal devices 100 , an inspection server device 200 and a management terminal 300 . The user terminal device 100, the inspection server device 200, and the management terminal 300 are connected to, for example, a communication network NW. Each device connected to the communication network NW has a communication interface such as a NIC (Network Interface Card) or a wireless communication module (not shown in FIG. 1). The communication network NW includes, for example, the Internet, WAN (Wide Area Network), LAN (Local Area Network), cellular network, and the like.

The user terminal device 100 includes, for example, an imaging unit 102, a display unit 104, and a control unit 106. The user terminal device 100 is, for example, a mobile terminal device such as a smart phone or a tablet terminal equipped with a camera device (imaging unit 102). The user terminal device 100 activates a UA (User Agent) such as a browser or an application program. The UA is, for example, an application (control unit 106) for communicating with the inspection server device 200 and the like. The user terminal device 100 utilizes an inspection application as a UA, and uses the content received from the inspection server device 200 to perform display processing by the display unit 104, reception processing of touch panel operation integrated with the display unit 104, and the like. I do. In addition, the user terminal device 100 uses the inspection application to provide the inspection server device 200 with operation information based on the user's operation, captured images, and the like.

The inspection server device 200 is an information processing device that receives a building inspection request and provides building inspection results. The inspection server device 200 includes, for example, a cooperative API (Application Programming Interface) unit 210 and an inspection target recognition unit 220 . Functional units such as the cooperative API unit 210 and the inspection target recognition unit 220 are realized by executing a program stored in a program memory by a processor such as a CPU (Central Processing Unit). Some or all of these functional units may be realized by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), or FPGA (Field-Programmable Gate Array), It may be realized by cooperation of software and hardware.

The cooperative API unit 210 includes, for example, an API unit 212, a parameter calculation unit 214, an inspection unit 216, and a control unit 218. The API unit 212 performs processing for receiving various types of information from the user terminal device 100, processing for providing content to the user terminal device 100, and the like. The parameter calculator 214 calculates parameters necessary for inspection processing. The inspection unit 216 performs a building inspection process using the parameters. The control unit 218 performs control such as activation of the image recognition engine in the inspection object recognition unit 220 . The inspection object recognition unit 220 includes an image recognition engine 222, for example. The image recognition engine 222 performs recognition processing for recognizing a building as an inspection target. An image used for parameter calculation and inspection is referred to as an inspection image.

FIG. 2 is a block diagram showing an example of an inspection object recognition unit. The inspection object recognition unit 220 includes, for example, a learning image database 230 , a learning processing unit 232 and a recognition processing unit 234 .

The learning image database 230 is a database that stores, for example, images acquired from the management terminal 300 as learning images. The learning image database 230 stores, for example, positive example images as learning images. A positive image is an image of a building in a normal state. The positive images are, for example, an image of the entire reinforcing bar, an image of the upper reinforcing bar, an image of the lower reinforcing bar, and an image of the pressure contact part, which is an image of the upper reinforcing bar, the lower reinforcing bar, and the pressure contact portion formed by pressing the upper reinforcing bar and the lower reinforcing bar together. is an image of The positive image is registered in the learning image database 230 with any of tag information corresponding to the entire reinforcing bar, tag information corresponding to the upper reinforcing bar, tag information corresponding to the lower reinforcing bar, and tag information corresponding to the press contact portion. be. Note that the learning image database 230 may store not only positive images but also images of objects different from buildings (abnormal objects and defective products) as learning images. A building composed of an upper reinforcing bar, a lower reinforcing bar, and a pressure welded portion is an example of a building as a whole. Each of the upper reinforcing bars, the lower reinforcing bars, and the pressure welding section is an example of a structural part of a building. A pressure contact is an example of a connection formed by connecting two components.

A learning image is supplied from the learning image database 230 to the learning processing unit 232 . The learning processing unit 232 performs recognition processing using the learning image and obtains a determination result. The learning processing unit 232 learns the processing parameters of the recognition processing unit 234 (recognition model 234A) so that when a positive example image is input, the determination result is that the entire building or the component parts of the building are present. , to generate learning result data. The learning processing unit 232 sets the processing parameters of the recognition processing unit 234 (recognition model 234A) so that when an image different from the building is input, the determination result is that the entire building and the component parts of the building do not exist. and generate learning result data. The processing parameter of the recognition processing unit 234 (recognition model 234A) is, for example, a filter (also called weight or bias) included in the neural network. The learning result data is stored in the learning result data storage unit 2341 .

The recognition processing unit 234 is a component corresponding to the image recognition engine 222 . The recognition processing unit 234 has, for example, a learning result data storage unit 2341 and a determination unit 2342 . The learning result data storage unit 2341 accumulates processing parameters of the recognition processing unit 234 as learning results, and the processing parameters of the recognition processing unit 234 are updated by the learning processing unit 232 . When the captured image is acquired from the user terminal device 100 , the recognition processing unit 234 extracts the feature amount using the recognition model 234A based on the processing parameters of the recognition processing unit 234 . The determination unit 2342 determines whether the entire building and the component parts of the building exist based on the extracted feature amount. The recognition processing unit 234 outputs the determination result by the determination unit 2342 as the recognition result.

The recognition processing unit 234 also generates an image (mask image) representing an area in the captured image recognized as the entire building, and an image (mask image) representing an area in the captured image recognized as each constituent part of the building. image showing the configuration part) is generated and transferred to the cooperative API unit 210 .

<Overall processing>
FIG. 3 is a diagram illustrating an example of a processing procedure of the building inspection system according to the embodiment; The flow of inspection processing in the building inspection system described above will be described below.

First, the user terminal device 100 activates an inspection application based on a user's operation (step S100). The user terminal device 100 activates the imaging unit 102 in response to activation of the inspection application, starts imaging, and causes the display unit 104 to display the captured image. In addition, the user terminal device 100 superimposes a guide image on the captured image and displays it (step S102). As a result, the user terminal device 100 obtains a photographed image by capturing an image of the entire reinforcing bar with the guide image displayed on the display unit 104, and transmits the obtained photographed image to the inspection server device 200 (step S104).

The image recognition engine 222 of the inspection server device 200 acquires the captured image transmitted from the user terminal device 100, and determines whether or not the entire reinforcing bar, the upper reinforcing bar, the lower reinforcing bar, and the press-contact portion are present in the captured image. Recognition processing is performed (step S106). At this time, the image recognition engine 222 calculates a certainty factor for each of the entire reinforcing bar, the upper reinforcing bar, the lower reinforcing bar, and the press contact portion. The image recognition engine 222 determines that each of the entire reinforcing bar, the upper reinforcing bar, the lower reinforcing bar, and the pressure contact part exists when the certainty for each of the entire reinforcing bar, the upper reinforcing bar, the lower reinforcing bar, and the pressure contact part is equal to or greater than a threshold value. . When the image recognition engine 222 recognizes that all of the reinforcing bars, the upper reinforcing bars, the lower reinforcing bars, and the pressure contact portion are present (step S108: YES), the process proceeds to step S112. If the image recognition engine 222 finds that none of the reinforcing bars, the upper reinforcing bar, the lower reinforcing bar, and the press-contact portion are present (step S108: NO), the image recognition engine 222 notifies the user terminal device 100 of re-imaging (step S110). ), and the photographed image is acquired again (step S106).

Note that the image recognition engine 222 determines whether or not the certainty factors of the entire reinforcing bar, the upper reinforcing bar, the lower reinforcing bar, and the press contact portion have exceeded the thresholds, but the present invention is not limited to this. A recognition result that all of the entire reinforcing bar, the upper reinforcing bar, the lower reinforcing bar, and the pressure weld are present may be generated when the sum of the certainty values of the reinforcing bar and the pressure weld exceeds a threshold value. In addition, the image recognition engine 222 may change the confidence factor threshold for each of the entire reinforcing bar, the upper reinforcing bar, the lower reinforcing bar, and the press contact portion. As a result, the image recognition engine 222 can recognize each of the entire reinforcing bar, the upper reinforcing bar, the lower reinforcing bar, and the press-contact portion with high accuracy.

In step S112, the parameter calculator 214 calculates parameters. At this time, the parameter calculation unit 214 calculates the parameters using the image showing the component acquired from the image recognition engine 222 . The image representing the component part is an inspection image, and is an image representing an area recognized as the component part in the captured image by the image recognition engine 222 (recognition model). Next, the inspection unit 216 performs inspection using the calculated parameters (step S114). The details of parameter calculation and inspection will be described later.

Next, the user terminal device 100 transmits information about the inspection result to the user terminal device 100 (step S116). The information about the inspection result may be, for example, information indicating the inspection content and the result corresponding to the inspection item, and may be information that causes the user terminal device 100 to display content including the inspection content and the inspection result. Display data including results and the like may be used.

Next, the user terminal device 100 displays the inspection content and the inspection result using the information on the inspection result received from the inspection server device 200 (step S118). The details of the display of inspection contents and inspection results will be described later.

<Shooting process>
FIG. 4 is a diagram showing an example of a guide image. The guide image is an image that assists the user terminal device 100 to photograph the building such that it is included in a predetermined range in the photographed image. The guide images include, for example, an upper guide image, a press contact portion guide image, and a lower guide image. In addition, the user terminal device 100 displays a message image "Please put a reinforcing bar in the guide and shoot" on the upper part of the screen. As a result, the user terminal device 100 aligns the upper guide image with the upper reinforcing bar, aligns the pressure contact portion guide image with the pressure contact portion, and aligns the lower guide image with the lower reinforcing bar. , the entire rebar, the upper rebar, the lower rebar, and the welded portion.

<Recognition processing>
FIG. 5 is a diagram showing an image in which the image (inspection image) recognized by the image recognition engine 222 is superimposed on the captured image. The inspection images include an upper rebar mask image, a pressure contact mask image, a lower rebar mask image, and an overall rebar mask image. The mask image is used in the sense of an image superimposed on the entire reinforcing bar, the upper reinforcing bar, the lower reinforcing bar, and the pressure contact portion included in the photographed image, but the inspection image is not limited to the mask image.

FIG. 6 is a diagram showing an example of tag information recognized by the image recognition engine 222. As shown in FIG. For example, the tag information corresponding to the upper reinforcing bar is indicated by the item number "1", the tag information corresponding to the lower reinforcing bar is indicated by the item number "2", and the tag information corresponding to the pressure welding part is indicated by the item number " 3”, and the item number of the tag information corresponding to the entire reinforcing bar is represented by “4”. The image recognition engine 222 assigns the tag information item number “1” to the upper reinforcing bar mask image, the tag information item number “2” to the lower reinforcing bar mask image, and the item number “3” to the pressure contact portion mask image. tag information is given, and the tag information of the item number "4" is given to the entire reinforcing bar mask image.

For example, when a plurality of upper reinforcing bars are detected in one captured image, the image recognition engine 222 determines the upper reinforcing bar mask image included in the entire reinforcing bar mask image as the recognition result. In addition, when a plurality of upper reinforcing bar mask images are included in one reinforcing bar mask image, the image recognition engine 222 recognizes the upper reinforcing bar mask image with a wide range included in the reinforcing bar mask image as the reinforcing bar mask image. Determine as the corresponding upper reinforcing bar mask image.

<Parameter calculation processing>
Processing for calculating the parameters will be described below. It should be noted that the process of calculating the parameters is performed using the mask image to which the tag information as shown in FIGS. 5 and 6 is added.

FIG. 7 is a diagram for explaining the process of calculating the thickness d of the upper and lower reinforcing bars, (A) is a diagram showing the center lines of the upper and lower reinforcing bars, and (B) is a mask image. FIG. 10 is a diagram for explaining a process of calculating thickness using .

The parameter calculation unit 214 calculates 0.25, 0.4, 0.5, and 0.6 when the length in the Y direction (height direction) is 1 for each of the upper reinforcing bar mask image and the lower reinforcing bar mask image. , 0.75. In addition, the lines extending in the X direction in FIG. 7(B) are the lines whose Y-direction positions are 0.25, 0.4, 0.5, 0.6, and 0.75.

Next, the parameter calculation unit 214 calculates the upper reinforcing bar mask image and the lower reinforcing bar mask image using the least squares method based on the obtained X-direction maximum value and X-direction minimum value for each of the upper reinforcing bar mask image and the lower reinforcing bar mask image. Approximate a line through the center of the mask image. The parameter calculator 214 rotates each of the upper reinforcing bar mask image and the lower reinforcing bar mask image so that the line passing through the center is vertical. Thereby, the parameter calculation unit 214 calculates the center line as shown in FIG. 7A for each of the upper reinforcing bar mask image and the lower reinforcing bar mask image. The parameter calculation unit 214 calculates the X-direction distance (thickness) of the upper reinforcing bar mask image and the mask of the lower reinforcing bar for each of the Y-direction positions of 0.25, 0.4, 0.5, 0.6, and 0.75. Calculate the X-direction distance (thickness) of the image.

The parameter calculation unit 214 calculates the median value among the thicknesses of the upper reinforcing bar mask image and the mask image of the lower reinforcing bar at Y-direction positions of 0.25, 0.4, 0.5, 0.6, and 0.75. Determined as the thickness d of the rebar and the bottom rebar.

Note that the parameter calculation unit 214 may determine the thickness d of each of the upper reinforcing bar and the lower reinforcing bar as an inspection parameter, but is not limited to this. It may be determined as a parameter. Further, when rotating the mask image, the parameter calculation unit 214 rotates the mask image of the thicker one of the upper reinforcing bar and the lower reinforcing bar, sets the line passing through the center vertically, and sets the rotation angle of the thicker mask image to may be set to the rotation angle of the narrower mask image.

FIG. 8 is a diagram for explaining the process of calculating the diameter D of the pressure contact portion. The parameter calculator 214 rotates the press contact portion mask image so that the upper reinforcing bar and the lower reinforcing bar are vertical. The parameter calculator 214, for example, rotates the press-contact portion mask image by the rotation angle determined when calculating the thickness d of the upper reinforcing bar and the lower reinforcing bar. Next, the parameter calculator 214 acquires the maximum value (Xmax) and the minimum value (Xmin) in the X direction of the pressed portion mask image, and determines the difference between the maximum value and the minimum value as the diameter D of the pressed portion.

FIG. 9 is a diagram for explaining the process of calculating the length L (distance in the Y direction) of the pressure contact portion. The parameter calculator 214 obtains the center of gravity of the pressed portion using the pressed portion mask image. The parameter calculation unit 214 superimposes a rectangular image whose center is the obtained center of gravity and whose X-direction distance is the thickness d on the pressed portion mask image. Note that the Y-direction distance of the rectangular image may be approximately the same as that of the pressed portion mask image.

The parameter calculator 214 obtains the Y-direction maximum value Ymax and the Y-direction minimum value Ymin of points where the pressed portion mask image protrudes from the rectangular image in the X direction. The parameter calculator 214 calculates the difference between the Y-direction maximum value Ymax and the Y-direction minimum value Ymin as the length L of the pressure contact portion. The parameter calculator 214 only shows the Y-direction maximum value Ymax and the Y-direction minimum value Ymin on the right side of the figure, but the Y-direction maximum value Ymax and the Y-direction minimum value Ymin may also be obtained on the left side of the figure. The length L may be calculated based on either one of the Y-direction maximum value Ymax and the Y-direction minimum value Ymin, or the length L may be calculated based on both the Y-direction maximum value Ymax and the Y-direction minimum value Ymin. good.

FIG. 10 is a diagram for explaining the process of calculating the eccentricity e of the upper and lower reinforcing bars. The parameter calculation unit 214 calculates the center line of the upper reinforcing bar and the lower reinforcing bar calculated when calculating the thickness of the upper reinforcing bar and the lower reinforcing bar, and the maximum value Xmax and the minimum value Xmin when calculating the diameter D of the pressure contact part. Use a connecting line. The parameter calculator 214 calculates the X-direction position x1 of the intersection of the center line of the upper reinforcing bar and the line connecting the maximum value Xmax and the minimum value Xmin. The parameter calculator 214 calculates the X-direction position x2 of the intersection of the center line of the lower reinforcing bar and the line connecting the maximum value Xmax and the minimum value Xmin. The parameter calculator 214 determines the absolute value of the difference between the X-direction position x1 and the X-direction position x2 as the amount of eccentricity e.

FIG. 11 is a diagram for explaining the process of calculating the bending angle θ between the upper reinforcing bar and the lower reinforcing bar. The parameter calculation unit 214 calculates the center line of the upper reinforcing bar and the center line of the lower reinforcing bar from the inclination of the center line of the upper reinforcing bar calculated when calculating the thickness of the upper reinforcing bar and the line passing through the center of the lower reinforcing bar. Calculate the angle with a line passing through . The parameter calculator 214 determines the calculated angle as the bending angle θ between the upper reinforcing bar and the lower reinforcing bar.

FIG. 12 is a diagram for explaining the process of calculating the width Δh of the one-sided bulge at the pressure contact portion. The parameter calculator 214 acquires the maximum value Xmax and the minimum value Xmin in the X direction in the pressed portion mask image used when calculating the diameter D of the pressed portion. In addition, the parameter calculation unit 214 calculates the Y-direction positions of 0.25, 0.4, 0.5, 0.6, 0.25, 0.4, 0.5, 0.6, 0.25, 0.4, 0.5, 0.6, 0.25, 0.25, 0.4, 0.5, 0.6, 0.25, 0.25, 0.4, 0.5, 0.6, 0.25, 0.25, 0.4, 0.5, 0.6, 0.5 times higher for both the upper reinforcing bar and the lower reinforcing bar used when calculating the thickness d. Obtain the maximum and minimum values in the X direction for each of 75, and calculate the average maximum value AXmax and the average minimum value AXmin. The parameter calculator 214 determines the absolute value of the difference between Xmin and AXmin as the width h1 of the single bulge, and determines the absolute value of the difference between Xmax and AXmax as the width h2 of the single bulge.

<Inspection processing>
Inspection processing by the inspection unit 216 will be described below. The inspection unit 216 performs inspection using the parameters calculated by the parameter calculation unit 214 .

The inspection unit 216 inspects whether or not the diameter D of the pressure contact portion is within the appropriate size range (see FIG. 14). For example, when the diameter D of the pressure contact portion is larger than the value obtained by multiplying the thickness d of the reinforcing bar by the first coefficient, the inspection unit 216 determines that the diameter D of the pressure contact portion is within the appropriate size range. , otherwise, it is determined that the diameter D of the pressure contact portion is not within the appropriate size range. The first coefficient is, for example, 1.4, but is not limited to this, and may be changeable according to predetermined rules, types of reinforcing bars, and the like.

The inspection unit 216 inspects whether or not the length L of the pressure contact portion is within the appropriate dimension range (see FIGS. 15 and 16). For example, when the length L of the pressure contact portion is larger than the value obtained by multiplying the thickness d of the reinforcing bar by the second coefficient, the inspection unit 216 determines that the length L of the pressure contact portion is within the appropriate dimension range. If not, it is determined that the length L of the pressure contact portion is not within the proper dimension range. The second coefficient is, for example, 1.1, but is not limited to this, and may be changeable according to predetermined rules, types of reinforcing bars, and the like.

The inspection unit 216 inspects whether or not the one-sided bulge at the pressure contact portion is within the appropriate size range (see FIG. 17). For example, when the absolute value of the difference between the width Δh1 of the bulge at the press contact portion and the width Δh2 of the bulge at the press contact portion is smaller than the value obtained by multiplying the thickness d of the reinforcing bar by a third coefficient, First, it is determined that the one-sided bulge at the pressure contact portion is within the appropriate size range, and if not, it is determined that the one-sided bulge at the pressure contact portion is not within the appropriate size range. The third coefficient is, for example, 0.2, but is not limited to this, and may be changeable according to predetermined rules, types of reinforcing bars, and the like.

The inspection unit 216 inspects whether or not the amount of eccentricity e between the upper reinforcing bar and the lower reinforcing bar is within an appropriate size range (see FIG. 18). For example, if the ratio of the eccentricity e between the upper reinforcing bar and the lower reinforcing bar to the thickness d of the reinforcing bar (e/d) is smaller than a predetermined first threshold value, the inspection unit 216 detects the eccentricity between the upper reinforcing bar and the lower reinforcing bar. It is determined that the amount e is within an appropriate dimensional range, otherwise, it is determined that the eccentricity e between the upper reinforcing bar and the lower reinforcing bar is not within an appropriate dimensional range. The predetermined first threshold is, for example, 0.2, but is not limited to this, and may be changeable according to predetermined rules, types of reinforcing bars, and the like.

The inspection unit 216 inspects whether or not the bending angle θ between the upper reinforcing bar and the lower reinforcing bar is within an appropriate size range (see FIG. 19). For example, when the bending angle θ between the upper reinforcing bar and the lower reinforcing bar is smaller than a predetermined second threshold value, the inspection unit 216 determines that the bending angle θ between the upper reinforcing bar and the lower reinforcing bar is within an appropriate size range. , otherwise, it is determined that the bending angle θ between the upper reinforcing bar and the lower reinforcing bar is not within the appropriate dimension range. The predetermined second threshold is, for example, 2.0, but is not limited to this, and may be changeable according to predetermined rules, types of reinforcing bars, and the like.

The inspection unit 216 determined that the diameter D of the pressure contact portion, the length L of the pressure contact portion, the width of the bulge at the pressure contact portion, and the amount of eccentricity e between the upper reinforcing bar and the lower reinforcing bar are all within the appropriate range of dimensions. If so, it is determined that the entire reinforcing bar is suitable, and if not, it is determined that the entire reinforcing bar is not suitable.

<Display processing>
Display processing of the user terminal device 100 will be described below.
FIG. 13 is a diagram showing an example of a screen presenting inspection results for the entire reinforcing bar. The user terminal device 100 displays a screen for presenting the test result (hereinafter referred to as the test result screen) based on the information about the test result received from the test server device 200 . The user terminal device 100 displays, for example, the inspection result screen shown in FIG. 13 as an initial screen of the inspection result.

The inspection result screen shown in FIG. 13 includes an icon image group, a captured image, a mask image, and an inspection result image. The icon image group includes an icon image corresponding to the thickness d of the reinforcing bar, an icon image corresponding to the diameter D of the pressure contact portion, an icon image corresponding to the length L of the pressure contact portion, and an eccentricity e between the upper reinforcing bar and the lower reinforcing bar. It includes a corresponding icon image, an icon image corresponding to the bending angle θ between the upper reinforcing bar and the lower reinforcing bar, and an icon image corresponding to the width Δh of the one-sided bulge at the pressure contact portion.

The mask image is superimposed on the captured image and displayed. As a result, the user terminal device 100 can present the user with the recognition results of the upper reinforcing bar, pressure contact portion, and lower reinforcing bar in the inspection server device 200 . The inspection result image is an image for presenting the inspection result that the rebar as a whole has been determined to be appropriate, and for example, displays a message of "determination OK".

FIG. 14 is a diagram showing an example of an inspection result screen for the diameter D of the pressure contact portion. When the user terminal device 100 receives an operation (tap operation) for selecting an icon image corresponding to the diameter D of the pressure contact portion, the user terminal device 100 displays the inspection result screen shown in FIG. 14 . The inspection result screen shown in FIG. 14 includes an image (hereinafter referred to as a parameter image) showing each parameter of the thickness d of the reinforcing bar and the diameter D of the pressure contact portion, an image of the inspection details, and an image of the inspection result. . The inspection content image is an image showing the content of the inspection that determined the inspection result, and numerical values of the diameter D of the press-contact portion and the thickness d of the reinforcing bar.

FIG. 15 is a diagram showing an example of an inspection result screen of the length L of the pressure contact portion. When the user terminal device 100 receives an operation (tap operation) for selecting an icon image corresponding to the length L of the pressure contact portion, the user terminal device 100 displays the inspection result screen shown in FIG. 15 . The inspection result screen shown in FIG. 15 includes parameter images of the thickness d of the reinforcing bar and the length L of the pressure contact portion, an inspection content image, and an inspection result image. The inspection content image is an image showing the content of the inspection that determined the inspection result, and numerical values of the length L of the pressure contact portion and the thickness d of the reinforcing bar. FIG. 16 is a diagram showing another example of an inspection result image of the length L of the pressure contact portion. The user terminal device 100 may indicate the examination content in other expressions as the examination content image.

FIG. 17 is a diagram showing an example of an inspection result screen of the width Δh of the one-sided bulge at the pressure contact portion. The user terminal device 100 displays an inspection result screen shown in FIG. 17 when receiving an operation (tap operation) for selecting an icon image corresponding to the width Δh of the one-sided bulge at the press contact portion. The inspection result screen shown in FIG. 17 includes parameter images of the thickness d of the reinforcing bar, the widths Δh1 and Δh2 of the single bulges at the pressure contact portion, an inspection content image, and an inspection result image. The inspection content image is an image showing the content of the inspection that determined the inspection result, the absolute value of the difference between Δh1 and Δh2, and the numerical value of the thickness d of the reinforcing bar.

FIG. 18 is a diagram showing an example of an inspection result screen for the amount of eccentricity e between the upper reinforcing bar and the lower reinforcing bar. When the user terminal device 100 receives an operation (tap operation) for selecting an icon image corresponding to the eccentricity e between the upper reinforcing bar and the lower reinforcing bar, the user terminal device 100 displays the inspection result screen shown in FIG. 18 . The inspection result screen shown in FIG. 18 includes parameter images of the reinforcing bar thickness d and the eccentricity e between the upper and lower reinforcing bars, an inspection content image, and an inspection result image. The inspection content image is an image showing the content of the inspection that determined the inspection result and the numerical value of the amount of eccentricity e between the upper reinforcing bar and the lower reinforcing bar.

FIG. 19 is a diagram showing an example of an inspection result screen of the bending angle θ between the upper reinforcing bar and the lower reinforcing bar. When the user terminal device 100 receives an operation (tap operation) for selecting an icon image corresponding to the eccentricity e between the upper reinforcing bar and the lower reinforcing bar, the user terminal device 100 displays the inspection result screen shown in FIG. 19 . The inspection result screen shown in FIG. 19 includes a parameter image indicating the bending angle θ between the upper reinforcing bar and the lower reinforcing bar, an inspection content image, and an inspection result image. The inspection content image is an image showing the content of the inspection that determined the inspection result and the numerical value of the bending angle θ between the upper reinforcing bar and the lower reinforcing bar.

FIG. 20 is a diagram showing an example of a calculation result screen of the reinforcing bar thickness d. When the user terminal device 100 receives an operation (tap operation) to select an icon image corresponding to the thickness d of the reinforcing bar, the user terminal device 100 displays the calculation result screen shown in FIG. 20 . The calculation result screen shown in FIG. 20 includes a parameter image showing the thickness d of the reinforcing bar and a calculation result image.

FIG. 21 is a diagram showing an example of an inspection result screen when there is an abnormality in the inspection result. For example, when the user terminal device 100 receives the information about the inspection result that the length L of the pressure contact portion is abnormal, the inspection result screen shown in FIG. 21 is displayed as the initial screen of the inspection result. The user terminal device 100 displays the icon image corresponding to the length L of the pressure contact portion and the icon image corresponding to the other inspection item in different display formats. The user terminal device 100, for example, emphasizes and displays an icon image of an inspection item with an abnormality. In addition, the user terminal device 100 displays, for example, an image showing a message “determination NG” in order to present an inspection result indicating that the entire reinforcing bar is not appropriate as an inspection result image showing the inspection result of the entire reinforcing bar.

FIG. 22 is a diagram showing another example of the inspection result screen of the diameter D of the pressure contact portion. When the user terminal device 100 displays the inspection result screen shown in FIG. 21 and selects an icon image corresponding to the diameter D of the pressure contact portion as an inspection item with no abnormality, the inspection shown in FIG. Display the result screen.

FIG. 23 is a diagram showing another example of the inspection result screen of the length L of the pressure contact portion. The user terminal device 100 displays an inspection result screen shown in FIG. 23 when an operation (tap operation) for selecting an icon image corresponding to the length L of the pressure contact portion having an abnormality is received. The inspection result screen shown in FIG. 23 includes a parameter image, an inspection content image for the user to confirm the inspection details of the abnormality, and an inspection result image showing a message "judgment NG". Also, the user terminal device 100 may highlight the inspection content image and the inspection result image.

<Effects of Embodiment>
According to the building inspection system of the embodiment described above, the imaging step of imaging the building and the first certainty factor indicating the likelihood of the entire building included in the photographed image that is the image obtained by imaging the building are calculated. and determining whether or not the building as a whole is included based on the first degree of certainty; calculating each and determining whether or not each component part of the building is included based on each of the second degrees of confidence; When it is determined that the building is inspected, it is possible to realize a building inspection method having an inspection step of performing an inspection based on an image showing a component.
As a result, according to the building inspection system of the embodiment, it is possible to recognize the entire building and its component parts with high accuracy, and to divide the building into its component parts for inspection. As a result, according to the building inspection system of the embodiment, the building can be inspected with high accuracy.

According to the building inspection system of the embodiment, since the guide image based on the shape of the building is superimposed on the captured image and displayed, it is possible to recognize and extract the entire building to be inspected and its constituent parts. You can take pictures of buildings. As a result, according to the building inspection system of the embodiment, even when it is necessary to capture a high-definition photographed image for highly accurate inspection of a building, A high-definition photographed image can be easily captured simply by photographing.

According to the building inspection system of the embodiment, a plurality of icon images corresponding to inspection items corresponding to one or more parameters, respectively, and the overall inspection results of the one or more inspection items are displayed, Inspection using an image showing parameters of an inspection item corresponding to the selected icon image and the parameters of the inspection item corresponding to the selected icon image when an operation to select one of a plurality of icon images is accepted An image showing the contents and the inspection result of the inspection item corresponding to the selected icon image are displayed. Thus, according to the building inspection system of the embodiment, when there is an abnormality in the overall inspection result, it is possible to present to the user what kind of abnormality is found in which inspection item.

According to the inspection server device 200 of the embodiment, the first degree of certainty indicating the likelihood of the entire building included in the inspection image, which is an image of the building, is calculated. A first determination unit (image recognition engine 222) that determines whether or not the entire object is included, and a second certainty factor that indicates the likelihood of each of the plurality of constituent parts of the building included in the inspection image are calculated. Then, a second determination unit (image recognition engine 222) that determines whether or not each component part of the building is included based on each of the second degrees of certainty, and the first determination unit and the second determination unit Realization of an information processing apparatus comprising an inspection unit (parameter calculation unit 214 and inspection unit 216) that performs inspection based on the relationship between a plurality of component parts when it is determined that the entire object and a plurality of component parts are included. can do.
According to the inspection server device 200 of the embodiment, it is possible to recognize the entire building and its component parts with high accuracy, and perform inspection by dividing the building into its component parts. As a result, according to the inspection server device 200 of the embodiment, a building can be inspected with high accuracy.

According to the user terminal device 100 of the embodiment, the display unit 104 displays a guide image based on the shape of the building, and the imaging unit 102 captures an image of the building while the guide image is displayed on the display unit 104. The display unit 104 displays a plurality of icon images corresponding to each of the inspection items corresponding to one or more parameters calculated based on the inspection image captured by the imaging unit 102, and one or more inspections. An image showing the parameters of the inspection item corresponding to the selected icon image and the selected icon image when an operation to select any one of a plurality of icon images is received. It is possible to realize a terminal device that displays an image showing the inspection content using the parameter of the inspection item corresponding to the icon image and the inspection result of the inspection item corresponding to the selected icon image.
According to the user terminal device 100 of the embodiment, it is possible to allow the inspection server device 200 to recognize and extract the building as a whole and its constituent parts to be inspected. As a result, according to the user terminal device 100 of the embodiment, even when it is necessary to capture a high-definition photographed image for highly accurate inspection of a building, A high-definition photographed image can be easily captured simply by photographing. Furthermore, according to the user terminal device 100 of the embodiment, when there is an abnormality in the overall examination result, it is possible to present to the user what kind of abnormality is present in which examination item.

<Modification>
Although the building inspection system of the above-described embodiment uses the guide image to image the reinforcing bars to be inspected, the present invention is not limited to this, and the reinforcing bars to be inspected may be selected from among the reinforcing bars included in the captured image. . Furthermore, the building inspection system of the embodiment may inspect each of the plurality of reinforcing bars when the photographed image includes the plurality of reinforcing bars. In this case, the building inspection system determines whether or not each of the plurality of reinforcing bars is included in the captured image with a size that enables highly accurate inspection. Therefore, only the reinforcing bars included in the photographed image may be inspected, and it may be determined that the reinforcing bars included in the photographed image are not subject to inspection. As a result, the building inspection system can perform a highly accurate inspection of a plurality of reinforcing bars with one shot.

Although each embodiment and modifications have been described, these are only examples and are not limited to these. A section may be combined with one or more other embodiments or one or more other modifications to realize one aspect of the present invention.

A program for executing each process of the user terminal device 100 and the inspection server device 200 in this embodiment is recorded in a computer-readable recording medium, and the program recorded in the recording medium is transferred to the computer system. The above-described various processes related to the user terminal device 100 and the inspection server device 200 may be performed by loading and executing them.

Note that the “computer system” referred to here may include hardware such as an OS and peripheral devices. The "computer system" also includes the home page providing environment (or display environment) if the WWW system is used. In addition, "computer-readable recording medium" means writable non-volatile memory such as flexible disk, magneto-optical disk, ROM, flash memory, portable medium such as CD-ROM, hard disk built in computer system, etc. storage device.

Furthermore, "computer-readable recording medium" means a volatile memory (e.g., DRAM (Dynamic
Random Access Memory)), which holds a program for a certain period of time. Also, the program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium.

Here, the "transmission medium" for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the program may be for realizing part of the functions described above. Further, it may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

As described above, the embodiments of the present invention have been described in detail with reference to the drawings, but the specific configuration is not limited to these embodiments, and includes designs within the scope of the gist of the present invention. For example, as an embodiment of the present invention, the user terminal device 100 may be used alone.

100 User terminal device 102 Imaging unit 104 Display unit 106 Control unit 200 Inspection server device 210 Cooperation API unit 212 API unit 214 Parameter calculation unit 216 Inspection unit 218 Control unit 220 Inspection object recognition unit 222 Image recognition engine 230 Learning image database 232 Learning Processing unit 234 Recognition processing unit 234A Recognition model 300 Management terminal 2341 Learning result data storage unit 2342 Judging unit

Claims (13)

an imaging step of imaging a building;
calculating a first degree of certainty indicating the likelihood of the entire building being included in a photographed image that is an image of the building, and determining whether or not the entire building is included based on the first degree of certainty; calculating a second degree of certainty indicating the likelihood of each of a plurality of constituent parts of the building included in the photographed image; a determining step comprising determining whether each of the constituent parts of
an inspection step of performing an inspection based on an image showing the component parts when it is determined that the entire building and the plurality of component parts are included;
Inspection method for a building having In the imaging step, a guide image based on the shape of the building is superimposed on the captured image and displayed.
The building inspection method according to claim 1 . The determining step uses a recognition model for the entire building and the plurality of constituent parts to determine,
The recognition model is a recognition model learned using learning images including the building, and is designed to recognize that the entire building and the plurality of constituent parts exist when the photographed image is input. processing parameters are learned in
The building inspection method according to claim 1 or 2. The image representing the component part is an image representing an area of the captured image recognized as the component part by the recognition model.
The inspection method for a building according to claim 3. The plurality of component parts include a first reinforcing bar, a second reinforcing bar, and a connecting portion where the first reinforcing bar and the second reinforcing bar are connected,
The building inspection method according to any one of claims 1 to 4. In the inspection step, as parameters for inspection using an image showing the component, the thickness of the first reinforcing bar and the second reinforcing bar, the thickness of the connecting portion, the length of the connecting portion, the Calculate at least one of the eccentricity of the first reinforcing bar and the second reinforcing bar, the bending angle of the first reinforcing bar and the second reinforcing bar, and the width of the bulge of the connecting portion, and perform inspection using the calculated parameters.
The inspection method for a building according to claim 5. Having a display step of displaying inspection results from the inspection step,
The display step includes:
displaying a plurality of icon images corresponding to each of the inspection items corresponding to each of the one or more parameters and an overall inspection result of the one or more inspection items;
When an operation to select one of the plurality of icon images is accepted, an image showing parameters of an inspection item corresponding to the selected icon image and parameters of an inspection item corresponding to the selected icon image are used. displaying an image showing inspection details and the inspection result of the inspection item corresponding to the selected icon image;
The inspection method for a building according to claim 6. The display step includes displaying an icon image corresponding to an inspection item determined to be abnormal by the inspection step among the plurality of inspection items, and an inspection item determined to be normal by the inspection step among the plurality of inspection items. displaying in a display form different from the display form of the icon image corresponding to the item;
The inspection method for a building according to claim 7. calculating a first degree of certainty indicating the likelihood of the entire building being included in a captured image that is an image of the building, and determining whether or not the entire building is included based on the first degree of certainty; a first determination unit that determines
A second degree of certainty indicating the likelihood of each of the plurality of constituent parts of the building included in the photographed image is calculated, and each of the constituent parts of the building is included based on each of the second certainties. a second determination unit that determines whether or not
an inspection unit that, when the first determination unit and the second determination unit determine that the entire building and the plurality of component parts are included, performs an inspection based on an image showing the component parts;
Information processing device. In the computer of the information processing equipment,
A first degree of certainty indicating the likelihood of the entire building being included in an inspection image, which is an image of the building, is calculated, and whether or not the entire building is included based on the first degree of certainty. a step of determining
A second degree of certainty indicating the likelihood of each of the plurality of constituent parts of the building included in the inspection image is calculated, and each of the constituent parts of the building is included based on each of the second certainties. a step of determining whether or not
If it is determined that the entire building and the plurality of component parts are included, performing an inspection based on an image showing the component parts;
program to run. a display unit that displays a guide image based on the shape of a building;
an imaging unit configured to capture an image of the building while the guide image is displayed on the display unit;
The display unit displays a plurality of icon images corresponding to inspection items respectively corresponding to one or more parameters calculated based on inspection images captured by the imaging unit, and the one or more inspection items. to display the overall test results of the
When an operation to select one of the plurality of icon images is accepted, an image showing parameters of an inspection item corresponding to the selected icon image and parameters of an inspection item corresponding to the selected icon image are used. displaying an image showing the inspection content and the inspection result of the inspection item corresponding to the selected icon image;
Terminal equipment. on the terminal computer,
a display step of displaying a guide image based on the shape of the building;
and an image capturing step of capturing an image of the building while the guide image is displayed on a display unit, comprising:
The display step includes: a plurality of icon images corresponding to inspection items respectively corresponding to one or more parameters calculated based on an inspection image captured by an imaging unit; Display the overall test results and
When an operation to select one of the plurality of icon images is accepted, an image showing parameters of an inspection item corresponding to the selected icon image and parameters of an inspection item corresponding to the selected icon image are used. displaying an image showing the inspection content and the inspection result of the inspection item corresponding to the selected icon image;
program. a terminal device including an imaging unit that captures an image of a building;
An image of the building is received from the terminal device, a first certainty factor indicating the likelihood of the entire building included in an inspection image that is an image of the building is calculated, and the first certainty is calculated. a first determination unit that determines whether or not the entire building is included based on the degree of certainty; and a second degree of certainty that indicates the likelihood of each of the plurality of constituent parts of the building included in the inspection image respectively, and based on each of the second degrees of certainty, a second determination unit that determines whether each component part of the building is included, and the first determination unit and the second determination unit When it is determined that the entire building and the plurality of component parts are included, an inspection unit that performs an inspection based on an image showing the component parts, and information for transmitting the inspection result of the inspection unit to the terminal device a processor;
A building inspection system comprising:

Images