WO2024084673A1

WO2024084673A1 - Reinforcement inspection device, learning device, reinforcement inspection system, and reinforcement inspection method

Info

Publication number: WO2024084673A1
Application number: PCT/JP2022/039215
Authority: WO
Inventors: 健宮本; 友哉澤田; 理高橋; 高明宮本
Original assignee: 三菱電機株式会社
Priority date: 2022-10-21
Filing date: 2022-10-21
Publication date: 2024-04-25

Abstract

This reinforcement inspection device (2) comprises: an acquisition unit (221) that acquires image information; a selection unit (223) that selects a designated learning model from among a plurality of learning models for inferring a sub-component member appearing in the image information, the learning models being provided with respect to each external appearance feature including the type, color, and shape of the sub-component member; and an inference unit (224) that infers a sub-component member from the image information using the selected learning model.

Description

Reinforcement inspection device, learning device, reinforcement inspection system, and reinforcement inspection method

This disclosure relates to a reinforcing bar inspection device, a learning device, a reinforcing bar inspection system, and a reinforcing bar inspection method.

In the construction of reinforced concrete structures, inspections (reinforcement inspections) are conducted to check whether reinforcing bars have been arranged as designed in reinforcement structures in which multiple reinforcing bars are arranged. For example, Patent Document 1 describes a reinforcement as-built management system that uses at least one of pattern matching and machine learning techniques to generate data such as the type of reinforcing bars, number of reinforcing bars, reinforcing bar pitch, length of reinforcing bars, thickness of reinforcing bars, shape and position of joints, etc. within a set shooting range.

JP 2020-27058 A

　An actual reinforced concrete structure is composed of not only the main reinforcing bars, which are the main reinforcing bars, but also various other components that are installed in addition to the main reinforcing bars. Components installed in reinforced concrete structures other than the main reinforcing bars include components installed as part of the main reinforcing bars, such as joints, and also components installed as separate components from the main reinforcing bars, such as shear reinforcement bars. However, in conventional reinforcement inspections, although technology has been proposed to automatically detect the main reinforcing bars, there is nothing that can automatically detect various other components apart from the main reinforcing bars, and the current situation is that inspectors must inspect manually.

The reinforcement as-built management system described in Patent Document 1 automatically detects main reinforcement and the joints installed in some of the main reinforcement, but does not anticipate the detection of components separate from the main reinforcement, such as shear reinforcement. For this reason, even in the reinforcement as-built management system described in Patent Document 1, it is expected that inspectors will still manually inspect components installed separately from the main reinforcement in reinforcement structures.

The present disclosure aims to solve the above problem by providing a reinforcement inspection device, learning device, reinforcement inspection system, and reinforcement inspection method that can automatically detect secondary components other than the main reinforcement in a reinforcement structure.

The reinforcing steel inspection device disclosed herein is a reinforcing steel inspection device that inspects a reinforcing steel structure in which multiple reinforcing steel bars are arranged as main reinforcement and which includes secondary components other than the main reinforcement, and includes an acquisition unit that acquires image information, a selection unit that selects a specified learning model from multiple learning models provided for each external appearance feature of the secondary component, including the type, color, and shape, for inferring the secondary component shown in the image information, and an inference unit that infers the secondary component from the image information using the selected learning model.

According to the present disclosure, a specified learning model is selected from among multiple learning models for inferring secondary components shown in image information, each of which is provided for a different external characteristic including the type, color, and shape of the secondary component, and the secondary component is inferred from the image information using the selected learning model. This allows the reinforcement inspection device according to the present disclosure to automatically detect secondary components provided other than the main reinforcement in a reinforcement structure.

1 is a block diagram showing a configuration of a bar arrangement inspection system according to a first embodiment; 1 is a block diagram showing a hardware configuration for realizing the functions of a bar arrangement inspection device according to a first embodiment. FIG. FIG. 4 is a screen diagram showing an example of an operation screen. FIG. 11 is a diagram showing an example of registered contents of a selection candidate database (hereinafter, referred to as DB). FIG. 11 is a screen diagram showing an example of image information obtained by photographing a reinforcement structure. 4 is a flowchart showing a reinforcement bar inspection method according to the first embodiment. FIG. 11 is a diagram illustrating an example of learning data. FIG. 2 is a block diagram showing a configuration of a learning unit. FIG. 13 is a diagram showing evaluation results of a learning model. FIG. 13 is an explanatory diagram illustrating a schematic diagram of a matching determination of an embedding vector in a learning model. 13 is a flowchart showing a process for creating a preset model. 13 is a flowchart showing a process for creating a custom model.

Embodiment 1.
Fig. 1 is a block diagram showing the configuration of a reinforcement inspection system 1 according to a first embodiment. In Fig. 1, the reinforcement inspection system 1 is a system in which a reinforcement inspection device 2 and a learning device 3 are connected by communication, and inspects a reinforcement structure before concrete is poured. The reinforcement inspection device 2 acquires image information showing the reinforcement structure, and detects secondary components in the reinforcement structure using a learning model for inferring secondary components in the reinforcement structure shown in the acquired image information. The learning device 3 creates a learning model used for the detection of secondary components by the reinforcement inspection device 2.

The reinforcing bar inspection device 2 uses image information of the reinforcing bar structure captured by the camera device to inspect at least the secondary components in the reinforcing bar structure, and outputs the inspection results to the display unit 23. For example, the reinforcing bar inspection device 2 inspects the number of secondary components of multiple types contained in the reinforcing bar structure that are arranged, or inspects whether the secondary components have been created as designed. The reinforcing bar inspection device 2 is, for example, a tablet terminal, a smartphone, or a personal computer (PC). The learning device 3 is, for example, a server that provides the reinforcing bar inspection device 2 with a learning model used to detect secondary components.

A reinforced concrete structure is composed of multiple main bars, which are arranged reinforcing bars, and secondary components other than the main bars. The main bars are the main reinforcing bars that bear the load of a building or other structure that is based on a reinforced concrete structure, and are also called distribution bars. For example, some reinforced concrete structures have a structure in which multiple layers of flat surfaces are provided with multiple main bars arranged in a lattice pattern.

A secondary component is a component provided in a reinforcement structure to supplement the main reinforcement or to give the reinforcement structure a function different from that of the main reinforcement, and is a part of the main reinforcement or a component completely separate from the main reinforcement. For example, secondary components include shear reinforcement, lap joints, spacer blocks, sheath tubes, and compression joints. Hereinafter, it is assumed that the reinforcement structure inspected by the reinforcement inspection system 1 is provided with at least one of shear reinforcement, lap joints, spacer blocks, sheath tubes, and compression joints.

Shear reinforcement is a reinforcing bar that restrains and reinforces the main bars. By attaching shear reinforcement to the main bars, the shear force acting on the main bars is suppressed. Shear reinforcement can be U-shaped, V-shaped, C-shaped, or other shapes. U-shaped or V-shaped shear reinforcement has longer reinforcing bars on both sides, and may be installed to surround the main bars that make up columns, etc. C-shaped shear reinforcement has shorter reinforcing bars on both sides than U-shaped or V-shaped reinforcing bars, and for example, one end is connected to one of the main bars spaced apart, and the other end is connected to the other main bar.

The ends of the shear reinforcement are hooked to connect to the main bars. There are variations in hook shape, such as right angle hooks, acute angle hooks, or semicircular hooks.
In addition, some reinforcing bars that make up the main reinforcement and shear reinforcement are coated with resin to improve corrosion resistance, and their appearance is a different color from the base metal. For example, reinforcing bars coated with epoxy resin are blue or green, and some reinforcing bars that have been specially surface-treated are gray. Thus, shear reinforcement bars have various appearance characteristics, including color and shape. Furthermore, the appearance characteristics of shear reinforcement bars vary depending on the manufacturer and the model.

A lap joint is a joint formed by overlapping the ends of two reinforcing bars, which are the main reinforcing bars. Secondary components also include components that are provided as part of such main reinforcing bars. In other words, the reinforcing bars that make up a lap joint may also be coated with a resin to improve corrosion resistance, and their appearance is a different color from the base metal.

Furthermore, lap joints come in a variety of shapes depending on the type of rebar and the application of the reinforced structure. For example, the length over which the ends of two rebars in a lap joint overlap varies depending on the application of the reinforced structure. The shape of a lap joint differs depending on whether or not the end of the rebar has a hook, and also differs depending on the shape of the hook. Examples of hook shapes include right-angle hooks, acute-angle hooks, and semicircular hooks. Thus, lap joints come in a variety of appearance features, including color and shape.

Spacer blocks are components used to maintain the cover of rebar and prevent disturbance of the rebar arrangement during work. "Cover" is the minimum distance from the concrete surface to the rebar. Spacer blocks consist of spacers and bar supports. Spacers ensure the cover of rebar on the sides, and bar supports ensure the cover of rebar in the horizontal direction.

Spacer blocks are made of concrete, steel, or plastic, and come in colors that correspond to the material they are made of. Spacer blocks are also available in dice shapes, grooved shapes with grooves for passing rebar through, and inverted V shapes. Thus, spacer blocks come in a variety of external features, including color and shape. Furthermore, the external features of spacer blocks vary depending on the manufacturer and the model.

　Sheath tubes are metal tubes through which steel wires are passed. For example, sheath tubes are made of galvanized steel sheets and have a silver appearance. Sheath tubes are installed at various positions in the reinforcement structure depending on their use, and their lengths also vary. Sheath tubes are also colored according to their material. Thus, sheath tubes have various appearance characteristics, including color and shape. Furthermore, the appearance characteristics of sheath tubes differ depending on the manufacturer and the model.

A compression joint is a joint formed by butting together the ends of two reinforcing bars, which are the main reinforcement. In other words, a compression joint is a secondary component that is installed as part of the main reinforcement. In addition, the reinforcing bars that make up a compression joint may also be coated with a resin to improve corrosion resistance, giving them a different color than the base metal.

The shape of the compression joint is subject to the reinforcement inspection. For example, the law requires that the diameter of the bulge at the joint where the ends of the reinforcing bars are butted together is at least 1.4 times the diameter of the reinforcing bars, and that the length of the joint where the ends of the reinforcing bars are butted together is at least 1.1 times the diameter of the reinforcing bars. In addition, the eccentricity of the compression joint from the central axis of the reinforcing bars is at most 1/5 of the diameter of the reinforcing bars, and the deviation of the compression surface from the top of the bulge is at most 1/4 of the diameter of the reinforcing bars.
As such, compression fittings come in a variety of appearance features including color and shape.

As shown in FIG. 1, the reinforcing bar inspection device 2 includes a communication unit 21, a calculation unit 22, a display unit 23, an operation input unit 24, and a memory unit 25. The learning device 3 includes a communication unit 31, a calculation unit 32, and a memory unit 33. Note that while FIG. 1 shows a case in which the reinforcing bar inspection device 2 and the learning device 3 are separate devices, the reinforcing bar inspection device 2 may also include the learning device 3. In this case, the

communication units

21 and 31 may be communication devices that exchange data within a common device. Furthermore, the

memory units

25 and 33 may be memory areas constructed in a common storage device.

Furthermore, the reinforcement inspection system 1, or the reinforcement inspection device 2 equipped with the learning device 3, may provide a reinforcement inspection service in the form of SaaS (Software as a Service) to a user terminal (not shown in FIG. 1) that can communicate with the reinforcement inspection device 2. For example, a reinforcement inspection application for providing the reinforcement inspection service is executed by the reinforcement inspection device 2, and the user terminal can receive the reinforcement inspection service on a web browser without having to install a dedicated application for the service.

The user terminal transmits image information of the reinforcement structure captured by a camera provided on the terminal to the reinforcement inspection device 2. The reinforcement inspection device 2 uses the image information received from the user terminal to inspect secondary components and the like shown in the image information and returns the inspection results to the user terminal. The user terminal receives the inspection results from the reinforcement inspection device 2 and can display the inspection results in an appropriate manner on a display unit (not shown) of the user terminal.

The communication unit 21 communicates with the learning device 3 via a communication line. For example, the communication unit 21 can communicate via a communication line with the learning device 3 capable of communication using a communication method such as LTE, 3G, 4G, or 5G. The communication unit 31 communicates with the reinforcing bar inspection device 2 via a communication line. For example, like the communication unit 21, the communication unit 31 can communicate via a communication line with the reinforcing bar inspection device 2 capable of communication using a communication method such as LTE, 3G, 4G, or 5G.

The calculation unit 22 controls the overall operation of the reinforcement inspection device 2. The calculation unit 22 includes an acquisition unit 221, a preprocessing unit 222, a selection unit 223, and an inference unit 224. The calculation unit 22 executes a reinforcement inspection application to realize the functions of the acquisition unit 221, the preprocessing unit 222, the selection unit 223, and the inference unit 224.

The calculation unit 32 controls the overall operation of the learning device 3. The calculation unit 32 includes a data acquisition unit 321, a learning unit 322, and a search unit 323. The calculation unit 32 executes a learning application to realize the functions of the data acquisition unit 321, the learning unit 322, and the search unit 323.

The display unit 23 is a display device provided in the reinforcing bar inspection device 2. The display unit 23 is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence) display device.

The operation input unit 24 is an input device that accepts operations on an operation screen (described below) displayed on the display unit 23. When the reinforcing bar inspection device 2 is a smartphone or tablet terminal, the operation input unit 24 is, for example, a touch panel that is integrated with the screen of the display unit 23. When the reinforcing bar inspection device 2 is a PC, the operation input unit 24 is, for example, a mouse or keyboard.

The storage unit 25 stores, for example, a reinforcement inspection application and information used in the calculation process by the calculation unit 22. The storage unit 25 stores, as information used in the calculation process, for example, image information, position information of objects reflected in the image information, and a learning model acquired from the learning device 3. The storage unit 25 is a storage device provided in a computer functioning as the reinforcement inspection device 2, and includes storage such as a HDD (Hard Disk Drive) or SSD (Solid State Drive), or memory 103 in FIG. 2 described later.

The storage unit 33 stores, for example, a selection candidate DB 331, a pre-learning model DB 332, and a learning DB 333 in addition to a learning application. The storage unit 33 is a storage device provided in a computer that functions as the learning device 3, and includes, for example, a storage device such as an HDD or SSD, or a memory 103 in FIG. 2 described later.

The selection candidate DB 331 stores one or more preset models. A preset model is a first learning model that is provided for each external appearance feature, including the type, color, and shape of a secondary component, and is used to infer a secondary component shown in image information acquired by the acquisition unit 221. The preset models are created in advance by the learning device 3 before being specified by the selection unit 223. These preset models are linked to each external appearance feature, including the type, color, and shape of a secondary component, and stored in the selection candidate DB 331.

The pre-training model DB332 stores pre-training models. The pre-training model is a learning model that has been trained to infer objects depicted in image information. For example, the pre-training model is a learning model that has been trained using a large amount of training data set such as COCO (Common Objects in Context). As a learning method, for example, it is possible to set the parameters of the neural network using the stochastic gradient descent method.

The learning DB 333 stores learning data including image information and positional information of objects shown in the image information. The positional information of objects shown in the image information is set, for example, using the operation input unit 24 in the reinforcement inspection device 2. When the learning device 3 receives designation information for a learning model from the reinforcement inspection device 2, it reads out a pre-learning model from the pre-learning model DB 332 and reads out learning data from the learning DB 333. The learning device 3 inputs the learning data into the pre-learning model to create a custom model for inferring secondary components shown in the image information. The custom model is created for each external appearance feature including the type, color, and shape of the secondary component, and is a second learning model for inferring secondary components shown in the image information acquired by the acquisition unit 221.

FIG. 2 is a block diagram showing the hardware configuration that realizes the functions of the reinforcement inspection device 2. For example, the reinforcement inspection device 2 has a communication interface 100, an input/output interface 101, a processor 102, and a memory 103 as its hardware configuration. The functions of the acquisition unit 221, preprocessing unit 222, selection unit 223, and inference unit 224 provided in the reinforcement inspection device 2 are realized by executing a reinforcement inspection application in this hardware configuration.

The communication interface 100 outputs the learning model received from the learning device 3 via the communication line to the processor 102, and transmits designation information for the learning model generated by the processor 102 to the learning device 3 via the communication line. The processor 102 reads and writes data from the memory unit 25 in FIG. 1 via the input/output interface 101. Furthermore, the processor 102 acquires image information from an external device via the input/output interface 101. The external device is, for example, a camera device that photographs the reinforcement structure, or an external storage device that stores image information photographed by the camera device.

The programs constituting the reinforcement inspection application for realizing the functions of the acquisition unit 221, preprocessing unit 222, selection unit 223, and inference unit 224 are stored in the storage unit 25. The processor 102 reads the programs stored in the storage unit 25 via the input/output interface 101, loads them into the memory 103, and executes the programs loaded into the memory 103. In this way, the processor 102 realizes the functions of the acquisition unit 221, preprocessing unit 222, selection unit 223, and inference unit 224. The memory 103 is, for example, a RAM (Random Access Memory).

When the functions of the learning device 3 are realized by the hardware configuration shown in FIG. 2, the functions of the data acquisition unit 321, learning unit 322, and search unit 323 of the learning device 3 are realized by executing a learning application in the above hardware configuration.

The communication interface 100 outputs the learning model specification information received from the bar arrangement inspection device 2 via the communication line to the processor 102, and transmits the learning model searched by the processor 102 to the bar arrangement inspection device 2 via the communication line. The processor 102 reads and writes data from the memory unit 33 in FIG. 1 via the input/output interface 101.

The programs constituting the learning application for realizing the functions of the data acquisition unit 321, the learning unit 322, and the search unit 323 are stored in the storage unit 33. The processor 102 reads out the programs stored in the storage unit 33 via the input/output interface 101, loads them into the memory 103, and executes the programs loaded into the memory 103. In this way, the processor 102 realizes the functions of the data acquisition unit 321, the learning unit 322, and the search unit 323.

The functional components of the bar arrangement inspection device 2 will be described.
The acquisition unit 221 acquires image information. For example, the acquisition unit 221 is connected to a camera device via wireless communication or wired communication, and receives image information (still images or video) of the reinforcement structure from the camera device. The information acquired by the acquisition unit 221 is output to the pre-processing unit 222. The acquisition unit 221 may also output the acquired information to the memory 103 shown in FIG. 2 for storage.

The camera device is assumed to be, for example, a monocular camera, but may also be a stereo camera or an infrared camera. With a stereo camera or infrared camera, distance information between the reinforcement structure and the camera device can also be obtained. The acquisition unit 221 may acquire this distance information in addition to the image information. If the reinforcement inspection device 2 is a smartphone or tablet terminal, the camera device may be a camera equipped on the smartphone or tablet terminal.

The preprocessing unit 222 preprocesses the image information acquired by the acquisition unit 221 so that the image information is in a form suitable for the inference process performed by the inference unit 224. For example, the preprocessing unit 222 normalizes the image information. Normalization is a process of adjusting pixel values on the screen of the display unit 23 that displays the image information to values within a certain range. When the image information is a color image, the color value of the i-th pixel in the image can be expressed as (r _i , g i , b i ), where r _i (red ₎ , g _i (green ₎ , and b _i (blue) take values from 0 to 255.

When a learning model is created by deep learning (hereinafter, referred to as DL), DL generally handles color values of 0 to 255 in the range of 0 to 1. Therefore, the preprocessing unit 222 calculates color values ( _ri , _gi , _bi ) obtained by normalizing the color value ( _ri , _gi , _bi ) of the i-th pixel according to the following formula (1). Note that normalization is performed for each frame image when the image information is a moving image consisting of frame images shot at a constant frame rate. This allows the inference unit 224 to smoothly infer the secondary components.

The pre-processing unit 222 may also perform normalization after orienting the image indicated by the image information. An orientated image is an image in which the distance between the camera device and the reinforcement structure is constant and the reinforcement structure is orientated directly relative to the camera device. For example, the pre-processing unit 222 specifies the four corner vertices of any rectangle on the reinforcing bars arranged in a lattice pattern in the image information, and estimates a transformation matrix using the position coordinates of the specified four vertices. Then, based on the estimated transformation matrix, the pre-processing unit 222 converts the image indicated by the image information into an orientated image in which the plane of the inspection target is orientated directly to the camera device.

The selection unit 223 is provided for each external appearance feature including the type, color, and shape of the secondary component, and selects a specified learning model from among multiple learning models for inferring the secondary component shown in the image information. For example, the selection unit 223 outputs display control information for displaying an operation screen to the display unit 23. The display unit 23 displays the operation screen according to the display control information from the selection unit 223. This operation screen is a screen that allows an operation to specify a learning model to be performed.

When a learning model is specified by an operation received using the operation input unit 24 based on the operation screen, the selection unit 223 creates specification information for the learning model and transmits the specification information to the learning device 3 via the communication line using the communication unit 21. When the learning device 3 receives the specification information from the reinforcement inspection device 2, it searches for the learning model indicated by the specification information from among the multiple learning models stored in the memory unit 33, and returns information indicating the learning model of the search result to the reinforcement inspection device 2.

Furthermore, the learning models to be selected include preset models and custom models. The preset model is a learning model for inferring secondary components in a reinforcement structure from image information, and is created in advance by the learning device 3. For example, a large amount of learning data set such as COCO is used to create the preset model, and the selection candidate DB 331 stores preset models whose evaluation results are equal to or above the lower limit of tolerance. For example, the accuracy rate with respect to the correct data (Precision) and the recall rate of the inference results (Recall) are used for the evaluation.

A custom model, like a preset model, is a learning model for inferring secondary components in a reinforcement structure from image information. However, creation of a custom model is initiated by the learning device 3 when the selection unit 223 accepts the designation of the custom model. To create a custom model, it is not possible to prepare a large data set as learning data, but image information captured on-site and in which the positions of secondary components are identified is used as learning data. This allows the reinforcement inspection device 2 to infer secondary components using a highly rated preset model, and to infer secondary components using a custom model suited to the situation on-site.

FIG. 3 is a screen diagram showing an example of operation screen 23A. Selection unit 223 outputs display control information to display unit 23 for displaying an operation screen on which an operation for specifying a preset model and a custom model is performed. Display unit 23 displays operation screen 23A as shown in FIG. 3 in accordance with the display control information from selection unit 223. Operation screen 23A displays selection buttons 23A-1, 23A-2, 23A-3, ... for selecting each of a plurality of preset models.

In FIG. 3, selection button 23A-1 is a selection button for specifying a preset model for inferring shear reinforcement having the external characteristics of brown and U-shape. Selection button 23A-2 is a selection button for specifying a preset model for inferring shear reinforcement having the external characteristics of blue and C-shape. Selection button 23A-3 is a selection button for specifying a preset model for inferring a rebar lap joint having the external characteristics of blue and cylindrical shape. These selection buttons slide to visible and selectable positions on operation screen 23A by operating slide bar 23Ab using operation input unit 24.

For example, an inspector of a reinforcement structure refers to design data for a structure that includes the reinforcement structure to identify a secondary component that is to be used in the reinforcement structure, and uses the operation input unit 24 to press a selection button for a learning model that corresponds to the identified secondary component. Operation information indicating which selection button was pressed is output from the operation input unit 24 to the selection unit 223. The selection unit 223 creates specification information for a preset model that corresponds to the operation information, and the communication unit 21 transmits the specification information to the learning device 3 via a communication line.

FIG. 4 is a diagram showing an example of the registered contents of the selection candidate DB 331. As shown in FIG. 4, the memory unit 33 of the learning device 3 stores the selection candidate DB 331 in which multiple preset models are stored and linked to appearance features including the type, color, and shape of the secondary component. When the learning device 3 receives the above-mentioned specification information from the reinforcement inspection device 2, it searches for the preset model indicated by the specification information from the multiple preset models stored in the selection candidate DB 331, and returns information indicating the preset model as a search result to the reinforcement inspection device 2.

For example, when the selection buttons 23A-1, 23A-2, and 23A-3 shown in FIG. 3 are pressed, the learning device 3 searches the selection candidate DB 331 for models A1, A2, and B1 shown in FIG. 4, and returns information indicating the search results for models A1, A2, and B1 to the reinforcement inspection device 2. Information indicating a learning model such as model A1 is, for example, a parameter required to construct a neural network that functions as a learning model, such as a weighting coefficient for a node. When the selection unit 223 receives information indicating models A1, A2, and B1 via the communication line from the communication unit 21, the selection unit 223 outputs the received information to the inference unit 224 and further stores it in the memory unit 25. This allows the selection unit 223 to accurately select a preset model for inferring a secondary component.

Note that Figure 4 shows model B1, which infers a lap joint of rebars with the external characteristics of being blue and cylindrical, but the shape of the lap joint is not limited to the shape of the rebar in which the lap joint is formed. For example, the shape of the lap joint includes the length over which the ends of the two rebars overlap. Furthermore, the shape of the lap joint also includes the presence or absence of a hook at the end of the rebar. Furthermore, if the rebar has a hook, the shape of the hook also includes, for example, a right-angle hook, an acute-angle hook, or a semicircular hook.

The shape of a compression joint includes the shape of the rebar in which the compression joint is formed, as well as the shape of the bulge at the part where the ends of the rebar are butted together and joined. The shape of the bulge is determined, for example, by the diameter of the bulge, the length of the bulge where the ends of the rebar are butted together and joined, the amount of eccentricity of the bulge from the central axis of the rebar, and the deviation of the compression surface from the top of the bulge.

The selection unit 223 may also automatically specify and select a preset model from a plurality of preset models. For example, design data of a structure including a reinforcement structure is stored in the storage unit 25. When an inspector of the reinforcement structure issues an instruction to start inspecting the reinforcement structure using the operation input unit 24, the selection unit 223 automatically identifies secondary components included in the reinforcement structure from the design data stored in the storage unit 25.

The design data may be, for example, a three-dimensional model that realizes Building Information Modeling (BIM) for a structure that includes a reinforcement structure, or design drawing data for a structure that includes a reinforcement structure. The selection unit 223 creates designation information that specifies a preset model for inferring the identified secondary component, and acquires the preset model searched for by the learning device 3 based on the designation information. This enables the selection unit 223 to accurately select a preset model for inferring the secondary component.

Furthermore, the selection unit 223 may select all preset models stored in the selection candidate DB 331. For example, when an instruction to start an inspection of a reinforcement structure is given using the operation input unit 24, the selection unit 223 selects all preset models stored in the selection candidate DB 331. The inference unit 224 infers secondary components using each of all preset models selected by the selection unit 223, and outputs the result obtained from the model with the highest evaluation as the final inference result.

In FIG. 3, selection button 23Ac is a selection button for creating a custom model. When selection button 23Ac is pressed using operation input unit 24, selection unit 223 creates designation information indicating the designation of a custom model, and further creates learning data including image information used to create the custom model. Selection unit 223 transmits the custom model designation information and learning data including image information to learning device 3 via communication line through communication unit 21.

The learning device 3 uses the learning data received from the reinforcement inspection device 2 to create a custom model corresponding to the specified information, and transmits the created custom model to the reinforcement inspection device 2 via the communication line by the communication unit 31. When the selection unit 223 receives information indicating a custom model via the communication line by the communication unit 21, it outputs the received information to the inference unit 224 and further stores it in the memory unit 25.

The training data used to create the custom model is image information taken on-site and in which the locations of sub-components are identified.
Fig. 5 is a screen diagram showing an example of image information of a reinforcement structure. The reinforcement structure shown in the image information shown in Fig. 5 has a plurality of reinforcing bars 11 arranged in a lattice pattern as main reinforcements, and includes shear reinforcement bars 12, spacer blocks 13, lap joints 14, sheath tubes 15, and compression joints 16 as secondary components.

When the selection button 23Ac is pressed using the operation input unit 24, the selection unit 223 causes the display unit 23 to display the screen shown in FIG. 5. The inspector uses the operation input unit 24 to identify each of the secondary components on the screen. For example, the inspector uses the operation input unit 24 to surround the area on the screen shown in FIG. 5 where the shear reinforcement 12 is displayed with a bounding box 23B-1, and inputs the color and name (in this case, "shear reinforcement") of the shear reinforcement 12 surrounded by the bounding box 23B-1.

The inspector also uses the operation input unit 24 to enclose the area on the screen shown in Figure 5 where the spacer block 13 appears with a bounding box 23B-2, and inputs the color and name of the spacer block 13 enclosed by the bounding box 23B-2 (in this case, "spacer block"). The inspector also uses the operation input unit 24 to enclose the area on the screen shown in Figure 5 where the lap joint 14 appears with a bounding box 23B-3, and inputs the color and name of the lap joint 14 enclosed by the bounding box 23B-3 (in this case, "lap joint").

Furthermore, the examiner uses the operation input unit 24 to surround the area on the screen shown in Figure 5 where the sheath tube 15 is displayed with a bounding box 23B-4, and inputs the color and name (in this case, "sheath tube") of the sheath tube 15 surrounded by the bounding box 23B-4.
Furthermore, the inspector uses the operation input unit 24 to surround the area on the screen shown in Figure 5 where the compression joint 16 is displayed with a bounding box 23B-5, and inputs the color and name (in this case, "compression joint") of the compression joint 16 surrounded by the bounding box 23B-5.

When the input of the bounding box is completed, the selection unit 223 extracts, for example, the position coordinates of the upper left vertex and the lower right vertex of the rectangular bounding box, and creates position information including the extracted position coordinates. This position information is linked to the color and name of the secondary component within the bounding box. In other words, the position information is information for identifying the bounding box that represents the correct label of the secondary component. Data including the image information identifying the secondary component by the bounding box and the above position information created by the selection unit 223 is used as learning data for creating a custom model.

In the example of FIG. 5, the shear reinforcement 12 is linked to the bounding box 23B-1, the spacer block 13 is linked to the bounding box 23B-2, the lap joint 14 is linked to the bounding box 23B-3, the sheath tube 15 is linked to the bounding box 23B-4, and the compression joint 16 is linked to the bounding box 23B-5. The learning device 3 creates custom models for each of the shear reinforcement 12, the spacer block 13, the lap joint 14, the sheath tube 15, and the compression joint 16.

Although the correct labels of the secondary components are shown in the form of bounding boxes in the above embodiment, the present invention is not limited to this and the labels may be shown as lines or mask images.
For example, the examiner uses the operation input unit 24 to draw a line along the longitudinal direction of the sheath tube 15 on the screen shown in Fig. 5. The selection unit 223 extracts the position coordinates of the start point and the end point of the line, and creates position information that links the extracted position coordinates with the sheath tube 15. The selection unit 223 creates learning data including this position information.

The examiner also uses the operation input unit 24 to mask everything except the sheath tube 15 on the screen shown in FIG. 5. The selection unit 223 extracts the position coordinates of the area on the screen shown in FIG. 5 where the unmasked sheath tube 15 is displayed, and creates position information that links the extracted position coordinates with the sheath tube 15. The selection unit 223 creates learning data that includes this position information.

The selection unit 223 may automatically set a bounding box surrounding an object present on the screen of FIG. 5 by performing image analysis such as pattern matching on the image information or by using a learning model that roughly detects objects shown in the image information. In this case, the inspector uses the operation input unit 24 to check whether a secondary component is present within the automatically set bounding box. The selection unit 223 will create the above-mentioned position information for a bounding box that is confirmed to contain a secondary component.

The reinforcement inspection device 2 may also infer secondary components from image information using at least one of a preset model or a custom model. For example, if the reinforcement inspection device 2 infers secondary components using only a custom model, the selection unit 223 automatically specifies a custom model when an instruction to start reinforcement inspection is given using the operation input unit 24, and proceeds to the above-mentioned custom model creation process.

The selection unit 223 may also select multiple preset models for a common secondary component. In this case, the selection unit 223 selects multiple preset models for the common secondary component by specifying a preset model using the operation input unit 24 or by automatically specifying a preset model. For example, if the type of secondary component is "shear reinforcement", the selection unit 223 selects all preset models whose type is "shear reinforcement". The inference unit 224 infers the secondary component using each of all the preset models selected by the selection unit 223, and outputs the result obtained from the model with the highest evaluation as the final inference result.

The inference unit 224 infers the secondary components from the preprocessed image information using the learning model selected as described above. For example, the inference unit 224 inputs the image information preprocessed by the preprocessing unit 222 for the preset model or custom model selected by the selection unit 223. The preset model or custom model infers the secondary components appearing in the input image information. For example, the positions and external features of the secondary components in the image are inferred by these learning models.
The pre-processing of the image information may be performed by an external device provided separately from the reinforcement inspection device 2. In this case, the reinforcement inspection device 2 does not need to include the pre-processing unit 222. The acquisition unit 221 acquires the pre-processed image information from the external device, and the inference unit 224 uses the image information acquired by the acquisition unit 221 as it is to infer the secondary components shown in the image information.
Furthermore, the inference unit 224 may infer the secondary components shown in the image information by directly using the image information of the reinforcement structure that has not been preprocessed. In this case as well, the reinforcement inspection device 2 does not need to be equipped with the preprocessing unit 222. For example, the learning device 3 creates a learning model using the image information that has not been preprocessed as learning data. By using this learning model, the inference unit 224 can infer the secondary components using the image information that has not been preprocessed.

The inference unit 224 inspects the secondary components based on the inference result, creates display control information for displaying the inspection result, and outputs the created display control information to the display unit 23 .
For example, the inference unit 224 inspects the number of shear reinforcement bars in a reinforcement structure based on the positions and appearance features of the shear reinforcement bars on the image inferred using the learning model.The inference unit 224 then creates display control information for superimposing an electronic whiteboard on which the inspection results are described on image information showing the reinforcement structure.The display unit 23 superimposes an electronic whiteboard on which the number, color, and shape of the shear reinforcement bars are described on the image showing the reinforcement structure based on the display control information.The electronic whiteboard is electronic image data on which the inspection results are described.

The inference unit 224 also inspects the number of lap joints in the reinforcement structure and the degree of deviation of the shape from the design value based on the position and appearance features of the lap joints on the image inferred using the learning model. For example, the inference unit 224 may inspect the amount of deviation of the overlapping length of the ends of two reinforcing bars from the design value for each lap joint, and display the inspection results for each lap joint on an electronic whiteboard.

Furthermore, the inference unit 224 checks the number of compression joints in the reinforcement structure and the degree of deviation of the shape from the design value based on the position and appearance features of the compression joints on the image inferred using the learning model. For example, the inference unit 224 may check the deviation from the design value for at least one of the diameter of the bulge in the inferred compression joint, the length of the bulge, the eccentricity of the bulge from the central axis of the rebar, and the deviation of the compression surface from the top of the bulge, and display this as the inspection result on an electronic whiteboard.
For example, for each compression joint installed in the reinforced concrete structure being inspected, the inspection results are displayed on the electronic whiteboard, indicating whether the diameter of the bulge at the part where the ends of the reinforcing bars are butt-jointed is 1.4 times or more the diameter of the reinforcing bars, whether the length of the part where the ends of the reinforcing bars are butt-jointed is 1.1 times or more the diameter of the reinforcing bars, whether the eccentricity from the central axis of the reinforcing bars is less than one-fifth of the diameter of the reinforcing bars, and whether the deviation of the compression surface from the top of the bulge is less than one-fourth of the diameter of the reinforcing bars.

The inference unit 224 may use the measurement results of the secondary components to determine whether the inference result is incorrect. For example, the acquisition unit 221 acquires point cloud data that represents the reinforcement structure as a three-dimensional point cloud. The point cloud data is data indicating the distance to the reinforcement structure detected by a sensor such as a stereo camera, an infrared camera, or a LIDAR. The inference unit 224 reads out the image information acquired by the acquisition unit 221 and stored in the storage unit 25, and assigns the distance d _i between the three-dimensional point on the object and the sensor to the pixel value (r _i , g _i , b _i ) in the image area in which the object is reflected in the image information. As a result, a pixel value (r _i , g _i , b _i , b _i ) having four elements is obtained as the pixel value in the image area in which the object is reflected.

The inference unit 224 estimates the size of the image area in which the secondary component is shown, calculates the size (1) of the secondary component based on this estimated value, and further calculates the size (2) of the secondary component using a distance d _i between a three-dimensional point in the image area in which the secondary component is shown and the pixel value of a pixel corresponding to the three-dimensional point. Then, the inference unit 224 determines whether the secondary component has been erroneously inferred based on the result of comparing the size (1) and the size (2) of the secondary component. Here, the size (2) corresponds to the actual size of the secondary component. If the value of the size (1) exceeds the upper limit of the allowable threshold range with respect to the value of the size (2), or falls below the lower limit of the allowable threshold range, the inference unit 224 determines that the inference of the secondary component by the learning model is erroneous.

The inference unit 224 may also calculate the inference accuracy of the learning model using the determination result of whether the inference result is incorrect. For example, the inference unit 224 causes each of the multiple learning models to perform inference using common image information, and the inference unit 224 determines whether the inference result is incorrect for each of the multiple learning models, and calculates the ratio of the number of correct inference results to the number of inferences as the inference accuracy.
By determining whether the inference result is an error in this manner, the reinforcement inspection device 2 can accurately infer the secondary components.

So far, we have shown the case where the reinforcement inspection device 2 detects secondary components in a reinforcement structure, but it may also automatically detect main reinforcement in addition to secondary components. For example, the preset models and custom models include models related to secondary components as well as models related to main reinforcement. A model related to main reinforcement is a third learning model that is provided for each external appearance feature including the type, color, and shape of the main reinforcement, and is used to infer the main reinforcement reflected in the image information. The third learning model may be a preset model or a custom model.

The selection unit 223 also displays a selection button for specifying a preset model for inferring the main reinforcement on the operation screen 23A shown in FIG. 3. When the selection button for a preset model related to the main reinforcement is operated using the operation input unit 24, the selection unit 223 creates specification information for the preset model related to the main reinforcement. When the inference unit 224 acquires a learning model indicated by the specification information from the learning device 3, it uses the acquired learning model to infer the main reinforcement from the image information preprocessed by the preprocessing unit 222. This allows the reinforcement inspection device 2 to automatically detect the main reinforcement in addition to secondary components in a reinforcement structure.

The functional components of the learning device 3 will now be described.
The data acquisition unit 321 acquires learning data including image information and positional information of objects in the image information. For example, when the data acquisition unit 321 receives designation information of a custom model from the reinforcement inspection device 2 via the communication line by the communication unit 31, the data acquisition unit 321 acquires learning data including image information of a captured reinforcement structure and positional information of secondary components shown in the image information from the reinforcement inspection device 2. The data acquisition unit 321 stores the learning data acquired from the reinforcement inspection device 2 in the learning DB 333.

The learning unit 322 uses the learning data to create and store a learning model for inferring secondary components shown in the image information. For example, the learning unit 322 creates a preset model or a custom model using a pre-learning model stored in the pre-learning model DB 332 and learning data stored in the learning DB 333. Furthermore, the learning unit 322 evaluates the created learning model, and determines the learning model whose evaluation value is equal to or greater than the allowable value as the learning model to be used by the reinforcement inspection device 2. For example, the matching rate with respect to the correct data and the recall rate of the inference result are used as indices to evaluate the learning model.

The search unit 323 searches for a learning model specified in the reinforcement inspection device 2 from the learning models created for each external appearance feature including the type, color, and shape of the secondary component, and outputs the learning model obtained by the search to the reinforcement inspection device 2. For example, when the search unit 323 acquires designation information of a preset model from the reinforcement inspection device 2, it searches the selection candidate DB 331 based on information about the secondary component included in the acquired designation information. The search unit 323 then transmits information indicating the preset model of the search result to the reinforcement inspection device 2 via the communication line by the communication unit 31. This allows the reinforcement inspection device 2 to acquire the designated learning model.

FIG. 6 is a flowchart showing the reinforcement bar inspection method according to the first embodiment.
The acquisition unit 221 acquires image information (step ST1). For example, the acquisition unit 221 acquires image information of a reinforcement structure captured by a monocular camera or a stereo camera provided in the reinforcement inspection device 2.
The pre-processing unit 222 pre-processes the image information (step ST2). For example, the pre-processing unit 222 calculates image information by normalizing the color values of pixels by the maximum color value.
In addition, when the acquisition unit 221 acquires preprocessed image information, or when the inference unit 224 infers secondary components using image information that has not been preprocessed, the process may proceed to step ST3 without performing step ST2.

The selection unit 223 selects a specified learning model from among multiple learning models held by the learning device 3 (step ST3). For example, the learning device 3 manages multiple preset models provided for each external appearance feature including the type, color, and shape of the secondary component, and further creates a custom model specified by the reinforcement inspection device 2. The selection unit 223 selects the learning model to be used by the inference unit 224 by specifying a preset model or a custom model to the learning device 3.

The inference unit 224 infers the secondary component from the preprocessed image information using the learning model selected by the selection unit 223 (step ST4). For example, the inference unit 224 causes the display unit 23 to display the inference result of the secondary component and the inspection result of the secondary component using the inference result.
By executing this method with the reinforcement inspection device 2, secondary components provided other than the main reinforcement in a reinforcement structure can be automatically detected.

Next, the advance creation of a preset model by the learning device 3 will be described.
First, the data acquisition unit 321 acquires learning data used to create a preset model. The learning data includes, for example, image information in which an object on an image is specified using the operation input unit 24. The learning data is stored in the learning DB 333 by the data acquisition unit 321. Fig. 7 is a diagram showing an example of the learning data. As shown in Fig. 7, the learning data is data provided for each external appearance feature including the type, color, and shape of the secondary component.

For example, in the learning data No. 1 in FIG. 7, bounding box positions A, B, and C are set in the image areas in image A where blue, U-shaped shear reinforcement bars are shown. In addition, in the learning data No. 2, bounding box positions D and E are set in the image areas in image B where blue, cylindrical lap joints on reinforcing bars are shown. Furthermore, in the learning data No. 3, bounding box position F is set in the image area in image C where a silver, cylindrical sheath tube is shown.

Fig. 8 is a block diagram showing the configuration of the learning unit 322. As shown in Fig. 8, the learning unit 322 includes a data classification unit 3221, a model creation unit 3222, and an evaluation unit 3223.
The data classification unit 3221 extracts a plurality of pieces of image information showing secondary components from the image information stored in the learning DB 333 by the data acquisition unit 321, and divides the extracted image information into pieces for learning and pieces for evaluation.

For example, when creating a preset model for inferring blue, U-shaped shear reinforcement, the data classification unit 3221 reads out the learning data No. 1 shown in FIG. 7 from the learning DB 333 and separates this data into data for learning and data for evaluation. This allows image information relating to the same shear reinforcement to be classified into data for learning and data for evaluation.

The model creation unit 3222 uses the image information for learning to create a learning model for inferring a secondary component shown in the image information. For example, the model creation unit 3222 uses a pre-learning model extracted from the pre-learning model DB 332 to learn the secondary component shown in the image information for learning to create a learning model for inferring the secondary component.
In addition, for example, a Siamese network is used as the learning model including the pre-learning model.

The evaluation unit 3223 evaluates the learning model using the image information for evaluation, and saves the learning model that satisfies the evaluation conditions as a preset model. For example, the evaluation unit 3223 evaluates the learning model created by the model creation unit 3222 using the image information for evaluation obtained from the data classification unit 3221.

FIG. 9 shows the evaluation results of the learning model, and is the evaluation result for the learning data No. 1 shown in FIG. 7. The learning data and evaluation data in FIG. 9 are image information classified by the data classification unit 3221 into learning data and evaluation data. The evaluation unit 3223 creates a data set that associates the type of pre-learning model used when the model creation unit 3222 created the learning model with the learning method (e.g., stochastic gradient descent method, etc.) for the learning data and evaluation data.

In the data set No. 1 shown in FIG. 9, the evaluation unit 3223 uses model A, which is a pre-learning model, to infer shear reinforcement using images A, I, and J and the position information of the shear reinforcement in these images as training data. Next, the evaluation unit 3223 uses the same model A to infer shear reinforcement using images K, L, and M and the position information of the shear reinforcement in these images as evaluation data. Using these results, the evaluation unit 3223 calculates the precision rate A and the recall rate A.

The evaluation unit 3223 calculates the precision and recall of the data sets after data set No. 2 in the same manner. Based on the evaluation condition of selecting the learning model with the highest precision and recall, the evaluation unit 3223 compares the precision and recall of all data sets related to the learning data No. 1 shown in FIG. 7. The evaluation unit 3223 determines the learning model with the highest precision and recall from the comparison results as the preset model and stores this model in the selection candidate DB 331. Note that, although the evaluation condition of selecting the learning model with the highest precision and recall has been shown, the present invention is not limited to this. For example, the evaluation condition may be to select a learning model with a precision and recall higher than a certain threshold. The evaluation condition may also be to select a learning model with either the precision or recall highest or higher than a threshold.

Furthermore, although precision and recall are shown as evaluation indices for the learning model, other evaluation indices may be used. For example, the evaluation indices may be MIOU (Mean Intersection Over Union), which is the ratio of the overlapping area between the bounding box (correct rectangle) surrounding the sub-component, which is the correct data, and the bounding box (inferred rectangle) surrounding the estimated sub-component.

FIG. 10 is an explanatory diagram that shows an outline of the matching judgment of embedded vectors in a learning model. In FIG. 10, the network to which image information (1) is input and the network to which image information (2) is input are a common network. Image information (1) is image information to which a correct answer label has been assigned using a bounding box or the like. Image information (2) is data that has been classified for learning purposes by the data classification unit 3221 from among the image information stored in the learning DB 333. For example, image information (2) is a partial image in which an image area showing any of the secondary components is extracted. Note that, for example, a Siamese Network may be used as the machine learning model that is the common network.

The model creation unit 3222 inputs the correct answer data, which is image information (1), into the network, and calculates an embedding vector corresponding to the image information (1). For example, in PaDiM, the embedding vector is calculated by combining the outputs from the first to third layers. This embedding vector is stored in the memory unit 33 by the model creation unit 3222.

Next, the model creation unit 3222 calculates an embedding vector corresponding to the image information using the learning image information (2), and creates a learning model as a preset model if it determines that the calculated embedding vector matches the embedding vector stored in the memory unit 33.
In this way, by using an embedding vector that can be created before calculating the final inference result, the time required to create a learning model can be reduced.

FIG. 11 is a flowchart showing the process of creating a preset model.
The data classification unit 3221 extracts multiple pieces of image information showing secondary components from the image information acquired by the data acquisition unit 321 and stored sequentially in the learning DB 333, and separates the extracted image information into pieces for learning and pieces for evaluation (step ST1A).
The model creation unit 3222 uses the image information for learning to create a learning model for inferring the secondary component shown in the image information (step ST2A).
The evaluation unit 3223 evaluates the learning model using the image information for evaluation, and saves the learning model that satisfies the evaluation conditions as a preset model (step ST3A).

FIG. 12 is a flowchart showing the process of creating a custom model.
The data classification unit 3221 acquires multiple pieces of image information showing secondary components from the image information stored in the learning DB 333, classifies the acquired multiple pieces of image information into those for learning and those for evaluation, and outputs the image information for learning to the model creation unit 3222 as learning data (step ST1B).
Next, the model creation unit 3222 acquires a pre-training model from the pre-training model DB 332 (step ST2B).
The model creation unit 3222 uses the pre-learning model and the learning data to create a learning model for inferring the secondary component shown in the image information (step ST3B).

The evaluation unit 3223 evaluates the learning model using the evaluation image information from the data classification unit 3221, and transmits the learning model that satisfies the evaluation conditions as a custom model to the reinforcement inspection device 2 via the communication line by the communication unit 31 (step ST4B).
If the inspector is not satisfied with the inference accuracy of the custom model, he/she may instruct re-creation using the operation input unit 24, thereby repeatedly creating learning data, creating the custom model shown in Fig. 12, and evaluating the performance of the custom model in the bar arrangement inspection system 1. Furthermore, the performance evaluation of the custom model may use the precision rate and the recall rate, or may use the MIOU.

As described above, the reinforcement inspection device 2 according to the first embodiment includes an acquisition unit 221 that acquires image information, a selection unit 223 that is provided for each external appearance feature including the type, color, and shape of the secondary component and that selects a specified learning model from among a plurality of learning models for inferring the secondary component shown in the image information, and an inference unit 224 that infers the secondary component from the image information using the selected learning model. This allows the reinforcement inspection device 2 to automatically detect secondary components that are provided other than the main reinforcement in a reinforcement structure.

In the reinforcement inspection device 2 according to the first embodiment, the multiple learning models include one or more preset models that are created before being specified, and a custom model that is created after being specified. This allows the reinforcement inspection device 2 to infer secondary components using preset models with high inference accuracy that are prepared in advance, and to infer secondary components using custom models that are suited to the on-site situation.

The reinforcement inspection device 2 according to the first embodiment includes a preprocessing unit 222 that preprocesses image information into a form suitable for the inference processing performed by the inference unit 224. The inference unit 224 uses a learning model to infer secondary components from the preprocessed image information. This allows the inference unit 224 to smoothly infer secondary components.

In the reinforcement bar inspection device 2 according to the first embodiment, the preprocessing unit 222 normalizes the image information. The normalized image information can be used as input data for the learning model created by DL.

In the reinforcement inspection device 2 according to the first embodiment, the selection unit 223 outputs display control information for displaying an operation screen on which an operation for specifying a learning model is performed, and selects the learning model specified by the accepted operation based on the operation screen. This enables the selection unit 223 to accurately select a learning model for inferring secondary components.

The reinforcing bar inspection device 2 according to the first embodiment includes a display unit 23 that displays an operation screen 23A based on the display control information. This allows the reinforcing bar inspection device 2 to display an operation screen that allows an operation to specify a learning model.

The bar arrangement inspection device 2 according to the first embodiment includes an operation input unit 24 that accepts an operation to specify a learning model from a plurality of learning models on an operation screen 23A displayed on the display unit 23.
This allows the learning model to be specified by operating the operation input unit 24.

In the reinforcement inspection device 2 according to the first embodiment, the selection unit 223 automatically specifies and selects a learning model from a plurality of learning models. This enables the selection unit 223 to accurately select a learning model for inferring secondary components.

In the reinforcement inspection device 2 according to the first embodiment, the acquisition unit 221 acquires point cloud data that represents the reinforcement structure as a three-dimensional point cloud. The inference unit 224 determines whether or not the secondary component has been erroneously inferred based on the result of comparing the secondary component calculated using the image information with the secondary component calculated using the point cloud data. This allows the reinforcement inspection device 2 to accurately infer the secondary component.

In the reinforcement inspection device 2 according to the first embodiment, the preset models and custom models include learning models that are provided for each external appearance feature, including the type, color, and shape of the main reinforcement, and are used to infer the main reinforcement reflected in the image information. The inference unit 224 uses these learning models to infer the main reinforcement from the preprocessed image information. This allows the reinforcement inspection device 2 to automatically detect the main reinforcement in addition to the secondary components in the reinforcement structure.

In the reinforcement inspection device 2 according to the first embodiment, the secondary component is at least one of a shear reinforcement bar, a lap joint, a spacer block, a sheath tube, or a compression joint. In this way, the reinforcement inspection device 2 is capable of detecting various components as secondary components.

The learning device 3 according to the first embodiment includes a data acquisition unit 321 that acquires learning data including image information and positional information of objects in the image information, a learning unit 322 that uses the learning data to create and store a learning model for inferring secondary components shown in the image information, and a search unit 323 that searches for a learning model specified in the reinforcement inspection device 2 from the learning models created for each external appearance feature including the type, color, and shape of the secondary component, and outputs the learning model obtained by the search to the reinforcement inspection device 2. This allows the learning device 3 to create a learning model for inferring secondary components shown in the image information for each external appearance feature including the type, color, and shape of the secondary component.

In the learning device 3 according to the first embodiment, the multiple learning models include one or more preset models created before being specified, and a custom model created after being specified. The data acquisition unit 321 acquires image information and stores it sequentially. The learning unit 322 includes a data classification unit 3221 that extracts multiple pieces of image information showing secondary components from the stored image information and separates the extracted image information into image information for learning and image information for evaluation, a model creation unit 3222 that uses the image information for learning to create a learning model for inferring the secondary components shown in the image information, and an evaluation unit 3223 that evaluates the learning model using the image information for evaluation and stores the learning model that satisfies the evaluation conditions as a preset model. This allows the learning device 3 to create a learning model with high inference accuracy as a preset model.

In the learning device 3 according to the first embodiment, the model creation unit 3222 calculates and stores an embedding vector corresponding to correct data, which is image information, calculates an embedding vector corresponding to image information using image information for learning, and creates a learning model in which it is determined that the calculated embedding vector matches the stored embedding vector. This allows the model creation unit 3222 to reduce the time required to create a learning model.

In the learning device 3 according to the first embodiment, the learning unit 322 creates a custom model specified in the reinforcement inspection device 2 using a pre-learned model that has been trained to infer objects shown in image information. This allows the learning device 3 to create a custom model with high inference accuracy even when there is a small amount of learning data.

In the learning device 3 according to the first embodiment, the data acquisition unit 321 repeatedly acquires learning data and the learning unit 322 repeatedly creates a custom model until the custom model meets the target inference accuracy. This allows the learning device 3 to create a custom model with high inference accuracy.

The reinforcement inspection system 1 according to the first embodiment includes a reinforcement inspection device 2 and a learning device 3. As a result, the reinforcement inspection system 1 can provide a reinforcement inspection device 2 that can automatically detect secondary components provided in addition to the main reinforcement in a reinforcement structure.

The reinforcement inspection method according to the first embodiment includes a step in which the acquisition unit 221 acquires image information, a step in which the selection unit 223 is provided for each external appearance feature including the type, color, and shape of the secondary component and selects a specified learning model from among a plurality of learning models for inferring the secondary component shown in the image information, and a step in which the inference unit 224 infers the secondary component from the image information using the selected learning model. By executing this method by the reinforcement inspection device 2, secondary components provided other than the main reinforcement in a reinforcement structure can be automatically detected.

In addition, any of the components of the embodiments may be modified or omitted.

The reinforcing bar inspection device disclosed herein can be used, for example, to inspect reinforcing bar structures before concrete is poured.

1 Reinforcement inspection system, 2 Reinforcement inspection device, 3 Learning device, 11 Steel bar, 12 Shear reinforcement bar, 13 Spacer block, 14 Lap joint, 15 Sheath tube, 16 Compression joint, 21 Communication unit, 22 Calculation unit, 23 Display unit, 23A Operation screen, 23A-1 to 23A-3, 23Ac Selection button, 23Ab Slide bar, 23B-1 to 23B-5 Bounding box, 24 Operation Input unit, 25 memory unit, 31 communication unit, 32 calculation unit, 33 memory unit, 100 communication interface, 101 input/output interface, 102 processor, 103 memory, 221 acquisition unit, 222 preprocessing unit, 223 selection unit, 224 inference unit, 321 data acquisition unit, 322 learning unit, 323 search unit, 3221 data classification unit, 3222 model creation unit, 3223 evaluation unit.

Claims

A reinforcing bar inspection device for inspecting a reinforcing bar structure including a plurality of reinforcing bars arranged as main reinforcements and including secondary components other than the main reinforcements,
An acquisition unit that acquires image information;
a selection unit that selects a designated learning model from among a plurality of learning models for inferring the secondary component shown in the image information, the learning model being provided for each external appearance feature including the type, color, and shape of the secondary component;
an inference unit that infers the secondary component from the image information by using the selected learning model.
The bar arrangement inspection device according to claim 1, characterized in that the multiple learning models include one or more first learning models that are created in advance before being specified, and a second learning model that is created after being specified.
a pre-processing unit that pre-processes the image information so that the image information is in a form suitable for inference processing performed by the inference unit;
The bar arrangement inspection device according to claim 1 or 2, wherein the inference unit infers the secondary component from the preprocessed image information by using the learning model.
The bar arrangement inspection device according to claim 3 , wherein the preprocessing unit normalizes the image information.
The reinforcement inspection device according to any one of claims 1 to 4, characterized in that the selection unit outputs display control information for displaying an operation screen on which an operation for specifying the learning model is performed, and selects the learning model specified by an operation accepted based on the operation screen.
The bar arrangement inspection device according to claim 5 , further comprising a display unit that displays the operation screen based on the display control information.
The bar arrangement inspection device according to claim 6, further comprising an operation input unit that accepts an operation to specify the learning model from a plurality of the learning models on the operation screen displayed on the display unit.
The bar arrangement inspection device according to claim 1 , wherein the selection unit automatically selects the learning model from a plurality of the learning models.
The acquisition unit further acquires point cloud data representing the reinforcement structure as a three-dimensional point cloud,
The reinforcing bar inspection device according to claim 1, characterized in that the inference unit determines whether or not the secondary component has been erroneously inferred based on a result of comparing the secondary component calculated using the image information with the secondary component calculated using the point cloud data.
The first learning model and the second learning model include a third learning model provided for each appearance feature including the type, color, and shape of the main reinforcement, for inferring the main reinforcement shown in the image information,
The bar arrangement inspection device according to claim 2 , wherein the inference unit infers the main reinforcement bars from the preprocessed image information by using the third learning model.
The reinforcing bar inspection device according to any one of claims 1 to 10, wherein the secondary component is at least one of a shear reinforcement bar, a lap joint, a spacer block, a sheath tube, and a compression joint.
A learning device that uses image information of a reinforcement structure in which a plurality of reinforcing bars are arranged as main reinforcements and which includes secondary components other than the main reinforcements to create a learning model for inferring the secondary components shown in the image information,
A data acquisition unit that acquires learning data including the image information and position information of an object in the image information;
a learning unit that uses the learning data to create and store the learning model for inferring the secondary component shown in the image information;
a search unit that searches for the learning model specified in the reinforcement inspection device from the learning models created for each appearance feature including the type, color, and shape of the secondary component, and outputs the learning model obtained by the search to the reinforcement inspection device.
The plurality of learning models include one or more first learning models that are created before being designated, and a second learning model that is created after being designated;
The data acquisition unit acquires and sequentially stores the image information;
The learning unit is
a data classification unit that extracts a plurality of pieces of image information showing the secondary component from the stored image information and classifies the extracted image information into pieces of image information for learning and pieces of image information for evaluation;
a model creation unit that creates the learning model for inferring the secondary component shown in the image information by using the image information for learning;
The learning device according to claim 12, further comprising an evaluation unit that evaluates the learning model using the image information for evaluation and stores the learning model that satisfies an evaluation condition as the first learning model.
The learning device according to claim 13, characterized in that the model creation unit calculates and stores an embedding vector corresponding to the correct answer data which is the image information, calculates an embedding vector corresponding to the image information using the image information for learning, and creates the learning model in which it is determined that the calculated embedding vector matches the stored embedding vector.
The learning device according to claim 13 or 14, characterized in that the learning unit creates the second learning model specified in the reinforcement inspection device by using a pre-learning model that has been trained to infer an object shown in the image information.
The learning device according to claim 15, characterized in that the data acquisition unit repeatedly acquires the learning data and the learning unit repeatedly creates the second learning model until the second learning model meets a target inference accuracy.
The bar arrangement inspection device according to any one of claims 1 to 11,
A learning device according to any one of claims 12 to 16,
Reinforcement inspection system equipped with
A reinforcing bar inspection method for inspecting a reinforcing bar structure in which a plurality of reinforcing bars are arranged as main reinforcing bars and which includes secondary components other than the main reinforcing bars, comprising:
An acquisition unit acquires image information;
A selection unit is provided for each appearance feature including a type, color, and shape of the secondary component, and selects a designated learning model from among a plurality of learning models for inferring the secondary component shown in the image information;
and a step of inferring the secondary component from the image information by an inference unit using the selected learning model.