JP7469987B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program for use in reinforcement inspection.

Traditionally, in the construction of reinforced concrete structures, a rebar inspection is carried out to check whether rebars are correctly arranged. Inspection items in a rebar inspection include rebar diameter type, surface treatment type, number of bars arranged, and rebar pitch. In this rebar inspection, a scale is used to measure the rebar pitch at the rebar position. Specifically, in the civil engineering construction management standards, the standard value for rebar spacing is ±20 mm, and the measurement standard is to measure three locations per span, at both ends and in the center, and at one measurement location, all rebars in the bridge axis direction and rebars perpendicular to the bridge axis direction are measured within a 2m range for each processing shape. In conventional rebar inspection methods, multiple workers measure the spacing of rebars one by one in these designated sections and ranges.

Patent Document 1 also discloses a technology in which a shooting range is set by providing reference data that serves as a standard for the length and thickness of reinforcing bars, the reinforcement state is photographed within the set shooting range, and the reinforcement state is recognized from the photographed data of the reinforcement state using at least one of pattern matching and machine learning techniques to generate reinforcement state data.

JP 2020-27058 A

The conventional reinforcing bar inspection method described above has a problem in that it is not possible to measure the condition of reinforcing bars in areas outside the measurement target. In addition, because the spacing between reinforcing bars is measured manually by an operator using a tape measure or the like, this requires a great deal of effort and there is a possibility of manual measurement errors. Furthermore, because the operator manually inputs the data of the measurement results into an information processing device, there is a possibility of problems such as data input errors. Therefore, if an attempt is made to adopt the technology described in the above-mentioned Patent Document 1, the data to be processed will become large, resulting in a new problem of the time required for image processing, etc. For this reason, there has been a demand for technology that can reduce the data to be processed in reinforcing bar inspection, and improve reliability while realizing labor savings.

The present invention has been made in consideration of the above, and its purpose is to provide an information processing device, information processing method, and program that can reduce the amount of data to be processed in reinforcement inspection, thereby improving reliability while reducing manpower.

In order to solve the above-mentioned problems and achieve the object, an information processing device according to one aspect of the present invention is an information processing device that includes a control unit that performs image processing on reinforcement image data obtained by imaging reinforcing bars in a reinforced concrete structure in which the reinforcing bars are arranged at their intersections, and the control unit acquires the captured reinforcement image data and stores it in a memory unit, identifies multiple intersections where the reinforcing bars intersect for the reinforcement image data read from the memory unit, and generates intersection data including coordinate information for the identified multiple intersections.

In the information processing device according to one aspect of the present invention, in the above invention, the control unit acquires the reinforcement image data from the storage unit as an input parameter, inputs the reinforcement image data read from the storage unit into an intersection identification learning model, outputs intersection image data in which a detection frame that extracts the intersection is added to the reinforcement image data as an output parameter, and generates the intersection data based on the output intersection image data, and the intersection identification learning model is a learning model generated by machine learning using the reinforcement image data as a learning input parameter and the intersection image data in which a detection frame indicating the intersection is added to the reinforcement image data as a learning output parameter.

In one aspect of the present invention, the information processing device is the above invention, in which the reinforced concrete structure comprises a plurality of consecutive reinforcing bars, a unique reinforcing bar number is set for each of the consecutive reinforcing bars, and the control unit performs clustering based on coordinate information of the plurality of intersections included in the intersection data, thereby dividing and forming subsets of the plurality of identified intersections that are determined to have the same reinforcing bar number.

In the information processing device according to one aspect of the present invention, in this configuration, the reinforced concrete structure comprises multiple consecutive reinforcing bars, and a unique reinforcing bar number is set for each consecutive reinforcing bar, and the control unit extracts multiple intersections determined to have the same reinforcing bar number from the multiple identified intersections, derives a reinforcing bar formula for each reinforcing bar number that approximates the multiple intersections determined to have the same reinforcing bar number, and determines whether the reinforcing bar number is correct for each of the multiple intersections determined to have the same reinforcing bar number based on the multiple reinforcing bar formulas derived for each reinforcing bar number and the coordinate information of each of the multiple intersections determined to have the same reinforcing bar number.

In this configuration of an information processing device according to one aspect of the present invention, when the control unit determines that the rebar number of the intersection is different, it corrects the rebar number of the intersection based on the rebar formula of the rebar number before or after the rebar number at which the intersection was determined.

In the information processing device according to one aspect of the present invention, in the above invention, the control unit calculates the distance between each of the identified intersections, extracts two intersections whose distance between the intersections is equal to or less than a predetermined distance, and determines that the two intersections are intersections of the joint portion of the reinforcing bar.

In the information processing device according to one aspect of the present invention, in this configuration, the reinforced concrete structure comprises continuous reinforcing bars, the continuous reinforcing bars have joint parts where at least two reinforcing bars are fixed along the length direction, a joint number unique to the joint part is set for each joint part, and the control unit performs clustering based on coordinate information of the intersections of the multiple joint parts, extracts and groups multiple intersections of the multiple joint parts that are determined to have the same joint number, and derives the joint length for each joint number.

An information processing method according to one aspect of the present invention is an information processing method executed by an information processing device having a control unit that performs image processing on reinforcement image data obtained by imaging reinforcing bars in a reinforced concrete structure in which the reinforcing bars are arranged at an intersection, and the information processing device acquires the captured reinforcement image data and stores it in a memory unit, identifies multiple intersections where the reinforcing bars intersect for the reinforcement image data read from the memory unit, and generates intersection data including coordinate information for the identified multiple intersections.

A program according to one aspect of the present invention causes a control unit, which performs image processing on reinforcement image data obtained by imaging reinforcing bars in a reinforced concrete structure in which the reinforcing bars are arranged at different angles, to acquire the captured reinforcement image data and store it in a memory unit, identify multiple intersections where the reinforcing bars intersect for the reinforcement image data read from the memory unit, and generate intersection data including coordinate information for the identified multiple intersections.

The information processing device, information processing method, and program of the present invention can reduce the amount of data to be processed in reinforcement inspection, making it possible to improve reliability while reducing manpower requirements.

FIG. 1 is an overall configuration diagram showing a schematic diagram of a bar arrangement inspection system to which an information processing device according to an embodiment of the present invention is applied. FIG. 2A is a plan view showing overall reinforcement image data of a reinforced concrete structure captured by an imaging device according to one embodiment of the present invention, and FIG. 2B is an enlarged plan view showing intersection image data on which annotation has been performed. FIG. 3 is a flowchart illustrating an information processing method according to an embodiment of the present invention. FIG. 4A is a plan view for explaining correction of an inference result based on a rebar equation in a reinforcing bar arrangement inspection method according to an embodiment of the present invention. FIG. 4B is a plan view for explaining correction of an inference result based on a rebar equation in the reinforcing bar arrangement inspection method according to an embodiment of the present invention. FIG. 5A is a plan view for explaining a method of detecting a joint in the reinforcement inspection method according to one embodiment of the present invention. FIG. 5B is a plan view for explaining a method of detecting a joint in the reinforcement inspection method according to one embodiment of the present invention. FIG. 5C is a plan view for explaining a method of detecting a joint in the reinforcement inspection method according to one embodiment of the present invention. FIG. 6 is a perspective view showing an example of a completed form form of a CIM model outputted from the bar arrangement inspection device according to one embodiment of the present invention.

One embodiment of the present invention will be described below with reference to the drawings. Note that in all the drawings of the embodiment below, the same or corresponding parts are given the same reference numerals. Furthermore, the present invention is not limited to the embodiment described below.

(Reinforcement inspection system)
Fig. 1 is an overall configuration diagram showing a schematic diagram of a reinforcing bar inspection system 1 to which an information processing device according to an embodiment of the present invention is applied. The reinforcing bar inspection system 1 shown in Fig. 1 is a system that takes an image of a deck slab reinforcing bar and measures the reinforcing bar interval, the number of reinforcing bars, joint length, etc., using an information processing device capable of image recognition processing. Note that reinforcing bar arrangement means the arrangement of reinforcing bars in a reinforced concrete structure, and reinforcing bar inspection means inspection of whether reinforcing bars are arranged accurately based on a reinforcing bar arrangement drawing showing the arrangement, dimensions, quantity, type, etc. of each reinforcing bar.

As shown in FIG. 1, the reinforcing bar inspection system 1 includes a reinforcing bar inspection device 10 and an imaging device 20, and is configured to be capable of inspecting a reinforced concrete structure 30 in which vertical reinforcing bars 31, which are reinforcing bars, and horizontal reinforcing bars 32, which are main reinforcing bars, are arranged in a lattice pattern.

The rebar inspection device 10 and the imaging device 20 are connected via a network 2. The network 2 may be one or more combinations of a dedicated line, a short-range wireless communication device, a public communication network such as the Internet, a LAN (Local Area Network), a WAN (Wide Area Network), a telephone communication network such as a mobile phone, a public line, a VPN (Virtual Private Network), etc.

The reinforcing bar inspection device 10, which serves as an information processing device, includes a control unit 11, a memory unit 12, an input unit 13, and an output unit 14.

As shown in FIG. 1, the control unit 11, which serves as a reinforcing bar inspection control unit, specifically includes a processor such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field-Programmable Gate Array), and a main memory unit such as a RAM (Random Access Memory) or a ROM (Read Only Memory) (none of which are shown).

The storage unit 12 is composed of a storage medium selected from a volatile memory such as RAM, a non-volatile memory such as ROM, an erasable programmable ROM (EPROM), a hard disk drive (HDD, Hard Disk Drive), and a removable medium. The removable medium is, for example, a universal serial bus (USB) memory, or a disk recording medium such as a compact disc (CD), a digital versatile disc (DVD), or a Blu-ray (registered trademark) disc (BD). The storage unit 12 may also be composed of a computer-readable recording medium such as an externally attachable memory card.

The storage unit 12 can store an operating system (OS), various programs, various tables, various databases, and the like for executing the operations of the reinforcing bar inspection device 10. Here, the various programs include information processing programs that realize processing based on models such as the learning model and the trained model according to this embodiment. These various programs can also be recorded on computer-readable recording media such as hard disks, flash memories, CD-ROMs, DVD-ROMs, and flexible disks, and widely distributed.

The storage unit 12 can store an OS, various programs, various tables, various databases, and the like for executing the operation of the bar arrangement inspection device 10. Here, the various programs include an information processing program using the intersection identification learning model 121 according to this embodiment and an information processing program for realizing hierarchical clustering. Specifically, the storage unit 12 stores the intersection identification learning model 121, the clustering code 122, the image data 123, the inference data 124, and the intersection data 125. The intersection identification learning model 121 is a model that can be updated as necessary, but if no update is performed, it becomes a trained reinforcement inspection trained model. The storage unit 12 may also be provided in another server that can communicate via the network 2. That is, the storage unit that stores the image data 123 and the storage unit that stores the inference data 124 and the intersection data 125 may be provided in another server that can communicate with the bar arrangement inspection device 10 via the network 2. These various programs and data can also be recorded on computer-readable recording media such as hard disks, flash memory, CD-ROMs, DVD-ROMs, and flexible disks, and widely distributed.

The control unit 11 can realize functions that meet a predetermined purpose by loading a program stored in the memory unit 12 into the working area of the main memory unit and executing the program, and controlling each component unit through the execution of the program. In this embodiment, the control unit 11 executes the functions of the image acquisition unit 111, image processing unit 112, intersection detection unit 113, joint detection unit 114, clustering processing unit 115, and learning unit 116 by executing the program stored in the memory unit 12.

Specifically, for example, the control unit 11 can execute all or at least some of the functions of the image acquisition unit 111, image processing unit 112, intersection detection unit 113, and joint detection unit 114 by reading the intersection identification learning model 121, which is a program, from the storage unit 12. The control unit 11 can realize the function of the clustering processing unit 115 for the acquired data by reading the clustering code 122 from the storage unit 12. It is also possible to generate learning models or trained models corresponding to each of the image processing unit 112, intersection detection unit 113, and joint detection unit 114, and store them in the storage unit 12. Details of the functions of the image acquisition unit 111, image processing unit 112, intersection detection unit 113, joint detection unit 114, clustering processing unit 115, and learning unit 116 will be described later.

The input unit 13 as an input means is configured using a user interface such as a keyboard, input buttons, levers, a touch panel for manual input superimposed on a display such as an LCD, or a microphone for voice recognition. The input unit 13 is configured so that a user can input predetermined information to the control unit 11 by operating the input unit 13. The output unit 14 as an output means is configured so that predetermined information can be notified to the outside. The output unit 14 displays images of waste from the reinforced concrete structure 30 on the display monitor, displays characters and figures on the screen of the touch panel display, and outputs sound from a speaker, according to the control of the control unit 11. The input unit 13 and the output unit 14 may be integrated into an input/output unit, and the input/output unit may be configured from a touch panel display, a speaker microphone, etc.

(Imaging device)
The imaging device 20 as an imaging means is configured to be able to image the vertical reinforcing bars 31 and the horizontal reinforcing bars 32 arranged in a lattice pattern from above the reinforced concrete structure 30. The imaging device 20 is typically configured with the imaging unit 21 mounted on a moving body, but is not necessarily limited to a moving body. The moving body is, for example, an unmanned aerial vehicle (UAV) such as a drone, a manned aircraft such as a helicopter, an airplane, or an airship, which moves within the atmosphere, or a space moving body such as an artificial satellite or a probe, which moves outside the atmosphere. The imaging unit 21 is, for example, configured with a high-resolution imaging camera, but various cameras can be used as long as the multiple vertical reinforcing bars 31 and the multiple horizontal reinforcing bars 32 in the reinforced concrete structure 30 can be distinguished from each other. The imaging device 20 may also be equipped with a distance measuring sensor capable of measuring the distance between the imaging unit 21 and the object to be photographed. When the imaging device 20 is equipped with a distance measurement sensor, even if the reinforced concrete structure 30 has multiple layers of reinforcement bars consisting of vertical reinforcement bars 31 and horizontal reinforcement bars 32 arranged in a lattice pattern, the imaging device 20 can obtain information on the distance to each layer of the multiple reinforcement bars.

(Intersection identification learning model and its generation method)
Next, the intersection identification learning model 121, which is a program stored in the storage unit 12, and a method for generating the same will be described. That is, the intersection identification learning model 121 is a learning model capable of executing a process for extracting intersections (hereinafter, intersections 33) between vertical reinforcing bars 31 and horizontal reinforcing bars 32 from image data (hereinafter, reinforcement image data) of a reinforced concrete structure 30 captured by the imaging device 20. Fig. 2(a) is a plan view showing an example of reinforcement image data of a reinforced concrete structure 30 captured by the imaging unit 21. Fig. 2(b) is a plan view showing an example of a state in which image annotation is performed on the intersections 33 between the vertical reinforcing bars 31 and horizontal reinforcing bars 32 in the reinforcement image data.

As input/output data sets for generating the intersection identification learning model 121, the reinforcement image data captured by the imaging device 20 shown in FIG. 2(a) is used as the learning input parameters, and the image data with the detection frame 33a of the intersection 33 added (hereinafter, intersection image data) shown in FIG. 2(b) is used as the learning output parameters.

Here, as an example of the conditions for imaging by the imaging device 20 to obtain the reinforcement image data, the height of the imaging device 20 at the time of imaging, i.e., the distance from the reinforced concrete structure 30, can be determined so that one pixel of the obtained reinforcement image data is less than 1 mm of the actual dimension in order to reduce measurement errors. In addition, it is preferable to use reinforcement image data with different conditions such as solar radiation conditions, background, or reinforcement state as the reinforcement image data. In addition, in order to connect multiple image data captured by the imaging device 20 to obtain the reinforcement image data of the entire reinforcement structure 30, the captured image data may be resized or the image data may be divided with a predetermined overlap. When dividing the image data, the predetermined overlap is preferably at least half the diameter of the reinforcement bars 31 and 32, taking into account that the reinforcement bars 31 and 32 are divided in the middle of their widths. Furthermore, it is also possible to increase the number of reinforcement image data used as input parameters for learning compared to the captured reinforcement image data by performing image processing such as color standardization, scale conversion, and flip processing (inversion processing) on the reinforcement image data. In this way, by increasing the number of learning input parameters, it becomes possible to extract intersections 33 with high accuracy even for reinforcement image data captured under various environments.

As shown in FIG. 2B, image data in which image annotation is performed on the reinforcement image data to extract the intersection 33 is used as the intersection image data. In the example shown in FIG. 2B, a detection frame 33a, which is called a bounding box surrounding the intersection 33, is drawn on the reinforcement image data as an image annotation. The detection frame 33a is specified by its upper left coordinate (x1, y1) and lower right coordinate (x2, y2). If the upper left coordinate (x1, y1) and lower right coordinate (x2, y2) of the detection frame 33a can be derived, the center coordinates of the detection frame 33a, i.e., the center coordinates (x0, y0) of the intersection 33, can be derived. This makes it possible to derive coordinate information such as the center coordinates of the intersection 33 from the intersection image data, and to obtain intersection data 125 including coordinate information for each intersection 33. Note that the intersection data 125 may or may not include intersection image data. Although not shown, the image annotation may also include specifying the coordinates of the four corners of the joint length of the vertical reinforcing bars 31 and the horizontal reinforcing bars 32. In this embodiment, when calculating the spacing between reinforcing bars in millimeters, it is necessary to measure the calculation in the reinforcement image data in pixel units. Therefore, in the annotation of the intersection 33, it is preferable that the pixel error of the detection frame 33a is within 3 pixels.

The learning unit 116 of the control unit 11 generates an intersection identification learning model 121 by machine learning, such as deep learning using a neural network, using the above-mentioned learning input parameters and learning output parameters as teacher data. The control unit 11 generates intersection image data from the reinforcement image data based on the content learned by the learning unit 116. Furthermore, the learning unit 116 may appropriately update the intersection identification learning model 121 as necessary, using the input reinforcement image data and intersection image data in which the worker has modified the detection frame 33a.

In addition, the detection accuracy of the intersection 33 is improved by performing NMS (Non-Maximum Suppression) on multiple detection frames 33a obtained as multiple inference results (also called recognition results) for the intersection 33 between the same vertical rebar 31 and horizontal rebar 32. That is, the detection frame 33a of the inference result with the highest confidence level, where the IoU value, a value that represents the rate of image overlap, is equal to or greater than a predetermined value, is retained, and other detection frames 33a with lower confidence levels are removed. The predetermined value for the IoU value can be set to, for example, approximately 0.63 to 0.65. This improves the reliability of the detection frame 33a of the intersection 33.

(Reinforcement inspection method)
Next, a method for inspecting a bar arrangement as an information processing method according to the present embodiment will be described. Fig. 3 is a flow chart for explaining the method for inspecting a bar arrangement according to one embodiment. Note that step ST1 is a process performed by the imaging device 20, and steps ST2 to ST11 are a process performed by the bar arrangement inspection device 10.

As shown in FIG. 3, in step ST1, the imaging device 20 captures an image of the reinforced concrete structure 30. The imaging device 20 captures the state of the reinforced concrete structure 30 within its field of view as a video consisting of multiple images. This allows the imaging device 20 to capture the reinforcement image data shown in FIG. 2(a). The image acquisition unit 111 of the control unit 11 acquires the reinforcement image data captured by the imaging device 20 via the network 2. Note that the imaging device 20 may store the reinforcement image data in an external recording medium, and the external recording medium may be used to store the reinforcement image data in the memory unit 12 as image data 123.

Next, the process proceeds to step ST2, where the image processing unit 112 of the control unit 11 performs a predetermined image processing on the acquired reinforcement image data. Specifically, the image processing unit 112 performs necessary processing on the acquired reinforcement image data, such as resizing, splitting, color standardization, scale conversion, and flipping. It is also possible not to perform step ST2 and not to perform image processing on the reinforcement image data.

Next, proceeding to step ST3, the intersection detection unit 113 of the control unit 11 extracts intersections 33 between the vertical reinforcing bars 31 and the horizontal reinforcing bars 32 in the reinforced concrete structure 30 from the acquired reinforcement image data based on the intersection identification learning model 121. Specifically, the intersection detection unit 113 assigns a detection frame 33a to each of the extracted intersections 33 based on the intersection identification learning model 121. Based on the assigned detection frame 33a, the intersection detection unit 113 derives the coordinates (x0, y0) (see FIG. 2) of the center point of each detection frame 33a, and stores them in the memory unit 12 as intersection data 125.

Next, proceeding to step ST4, the clustering processing unit 115 of the control unit 11 reads out the intersection data 125 including the coordinate information of the coordinates (x0, y0) of the center points of the derived multiple detection frames 33a from the storage unit 12. Based on the coordinate information of the read out intersection data 125, the clustering processing unit 115 determines the reinforcing bar number (hereinafter, the reinforcing bar number) to which each intersection 33 belongs, for example, using hierarchical clustering such as agglomerative hierarchical clustering. That is, first, the intersection 33 of the vertical reinforcing bar 31 is projected in the x-axis direction, and the intersection 33 of the horizontal reinforcing bar 32 is projected in the y-axis direction. Note that each of the coordinate data forms an isolated cluster. That is, in the initial state, the number of individual coordinate data is equal to the number of clusters. Next, for example, the distance between the x-coordinates x0 of the intersections 33 along the x-axis direction is calculated by the single link method or the minimum distance method, and the minimum value among them is set as the distance between the clusters. Similarly, the distance between clusters is calculated for the y-coordinate y0 of the intersection 33 along the y-axis direction. Next, clusters that are close to each other are grouped together to create a new cluster. By repeating the above process, a dendrogram can be obtained. After that, the clusters are divided into subsets using the reinforcing bar diameters of the reinforcing bars 31 and 32 as a threshold value. The inference results contained in the final divided clusters are determined to be intersections on the same reinforcing bars 31 and 32, and these intersections 33 are determined to belong to the same reinforcing bar number j. The coordinate information of the detection frame 33a determined to be the intersection 33 of the same reinforcing bars 31 and 32 is associated with the reinforcing bar number j and stored in the memory unit 12 as inference data 124.

Next, proceeding to step ST5, the intersection detection unit 113 derives the rebar formula f(x) based on the center coordinates (xi, yi) (i = 1, 2, ..., n) of each of the multiple intersections 33 associated with the same rebar number j, for example, by the least squares method shown in the following equations (1) and (1-1). Note that the number n is the number of intersections 33 associated with the same rebar number j. Also, the rebar formula f(x) is derived for the number of rebar numbers j. In other words, if the spliced rebars 31, 32 are considered as one rebar 31, 32, the rebar formula f(x) is derived for at least the number of rebars 31, 32.

なお、（ｘ0，ｙ0）は、中心座標の平均値である。 f(x)=ax+b... (1)

Here, (x0, y0) is the average value of the central coordinates.

Next, the process proceeds to step ST6, where the intersection detection unit 113 compares the center coordinates (xji, yji) of the detection frame 33a of the same belonging reinforcing bar number j determined in step ST4 with the reinforcing bar formula f j (x) of the belonging reinforcing bar number j and the reinforcing bar formulas f _j -1 (x) and f _{j+ 1 (x) of the preceding and succeeding belonging reinforcing bar numbers j+1} and j-1 in the reinforcing bars 31 and 32 derived in step ST5. Figures 4A and 4B are plan views for explaining the correction of the inference result based on the reinforcing bar formula in the reinforcing bar arrangement inspection method according to this embodiment.

As shown in FIG. 4A, the intersection detection unit 113 calculates the distance d 0 j,m between the center coordinates (xjm, yjm) of the m-th detection frame 33a of the belonging rebar number j (line L ₁ in FIG. 4A) and the rebar formula f _j (x). The intersection detection unit 113 also calculates the distance d _-1 ^j,m between the center coordinates (xjm, yjm) of the detection frame 33a and the rebar formula f _j-1 (x) of the belonging rebar number j-1. The intersection detection unit 113 also calculates the distance d ₊₁ ^j,m between the center coordinates (xjm, yjm) of the detection frame 33a and the rebar formula f _j+1 (x) of the belonging rebar number ^j ₊ 1 (line L ₂ in FIG. 4A). In FIG. 4A, if the line L ₂ is the rebar formula f _j (x), then the line L ₁ is the rebar formula f _j-1 (x).

Next, the process proceeds to step ST7, where the intersection detection unit 113 judges whether the detection frame 33a is accurate or not, in other words, whether the inference result of the intersection 33 inferred based on the intersection identification learning model 121 is accurate or not, based on the various distances calculated in step ST6. Specifically, the intersection detection unit 113 judges whether the following formula (2) or (3) is satisfied. If formula (2) or (3) is satisfied, the intersection detection unit 113 judges that the inference result of the detection frame 33a of the intersection 33 is not accurate (step ST7: No). If neither formula (2) nor formula (3) is satisfied, the intersection detection unit 113 judges that the inference result of the detection frame 33a of the intersection 33 is accurate (step ST7: Yes). Note that the following formulas (2) and (3) are examples of formulas for judging the inference result, and other formulas and conditions may be used.

If the intersection detection unit 113 determines that the inference result of the detection frame 33a inferred based on the intersection identification learning model 121 is not accurate (step ST7: No), the process proceeds to step ST8. In step ST8, the intersection detection unit 113 corrects the belonging reinforcing bar number j for the inference result of the detection frame 33a. Specifically, for example, the intersection detection unit 113 corrects the belonging reinforcing bar number j based on whether equation (2) or equation (3) holds at the center coordinates (xjm, yjm) of the detection frame 33a that is determined to be inaccurate. That is, if equation (2) holds at the center coordinates (xjm, yjm) of the detection frame 33a, the belonging reinforcing bar number j of the intersection 33 of this detection frame 33a is corrected to the belonging reinforcing bar number j-1. Similarly, if formula (3) is satisfied at the center coordinates (xjm, yjm) of the detection frame 33a, the rebar number j of the intersection 33 of the detection frame 33a is corrected to the rebar number j+1. As a result, the rebar formula f(x) and the detection frame 33a are integrated as shown in FIG. 4B. Note that by comparing the rebar formulas _fj-1 (x), _fj (x), and fj ₊₁ (x) with the center coordinates (xjm, yjm) of the detection frame 33a, it is possible to detect an intersection 33 for which a detection frame 33a has not been generated, i.e., an undetected intersection 33. The inferred coordinate information of the detection frame 33a is stored in the storage unit 12 as part of the intersection data 125. Then, the process proceeds to step ST9.

Furthermore, if the intersection detection unit 113 determines that the inference result of the intersection 33 inferred based on the intersection identification learning model 121 is accurate (step ST7: Yes), it proceeds to step ST9.

When the process proceeds to step ST9, the joint detection unit 114 of the control unit 11 extracts the detection frame 33a of the joint portion from the detection frames 33a clustered by the clustering processing unit 115, and derives the position and length of the joint. Figures 5A, 5B, and 5C are each a plan view for explaining the joint detection method in the reinforcement inspection method according to this embodiment.

That is, as shown in FIG. 5A, at the intersection 33 of the joint, there is a continuous detection frame 33a that is not found at a general intersection. In the example shown in FIG. 5A, a detection frame 33aa is provided at the intersection part between the vertical reinforcing bar 31 and the horizontal reinforcing bar 32a, and a detection frame 33ab is provided at the intersection part between the vertical reinforcing bar 31 and the horizontal reinforcing bar 32b. The joint detection unit 114 calculates the center distance dc for the center coordinates (x0, y0) of each detection frame 33a (33aa, 33ab) inferred as the intersection 33. The center distance dc can be calculated for all inferred detection frames 33a. Next, the joint detection unit 114 extracts at least two detection frames 33aa, 33ab from the multiple detection frames 33a, whose calculated center distance dc is equal to or less than a predetermined distance, specifically, equal to or less than about twice the diameter of the reinforcing bars 31, 32, that is, equal to or less than about two bars.

Next, as shown in FIG. 5B, for the consecutive detection frames 33aa, 33ab of the intersection 33 that have been clustered and inferred by the clustering processing unit 115, the joint detection unit 114 determines the joint numbers to which they belong (hereinafter, the "associated joint numbers"). In the example shown in FIG. 5B, "1" and "2" are set as the associated joint numbers. The coordinate data of each of the inferred detection frames 33aa, 33ab is associated with the associated joint numbers and stored as inference data 124 in the memory unit 12.

Next, as shown in FIG. 5C, the joint detection unit 114 extracts joint parts based on the detection frames 33aa and 33ab associated with the same joint number. In the example shown in FIG. 5C, the area _A1 with the joint number "1" and the area A2 with the joint number "2" are extracted as joint parts. Specifically, the joint detection unit 114 extracts the area _A1 defined by the maximum x coordinate and y coordinate (for example, coordinate (x11, y11)) and the minimum x coordinate and y coordinate (for example, coordinate (x12, y12)) from the coordinates of the detection frame 33a with the joint number "_1" . The joint detection unit 114 infers that the area _A1 is the joint part with the joint number "1". This makes it possible to reduce the detection error of the joint part to less than the interval between the reinforcing bars 31 and 32.

Next, the process proceeds to step ST10, where the intersection detection unit 113 reads out intersection data 125 including coordinate information of the multiple intersection image data obtained by steps ST2 to ST9 from the storage unit 12 and combines them together. The intersection detection unit 113 stores the intersection data 125 including the combined coordinate information in the storage unit 12. The intersection detection unit 113 may also output from the output unit 14 as coordinate information of multiple intersections 33 in the reinforced concrete structure 30.

Then, the process proceeds to step ST11, where the image processing unit 112 assigns an error amount attribute to the CIM (Construction Information Modeling/Management) model as design data based on the coordinate information of the intersection 33. FIG. 6 is a perspective view showing an example of a finished form of a CIM model output from the reinforcing bar inspection device 10 according to this embodiment. As shown in FIG. 6, the image processing unit 112 calculates the amount of error from the design arrangement of the reinforcing bars 31 and 32 in the CIM model based on the coordinate information of the intersection data 125 and the information of the CIM model. The image processing unit 112 draws the reinforcing bars 31 and 32 in the CIM model in a distinguishable manner according to the magnitude of the error amount. In the example shown in FIG. 6, a "thin solid line" is used when the error amount is an extremely small normal value, a "dash line" is used when the error amount is a small slight deviation, a "dash line" is used when the error amount is a slightly large deviation, and a "thick line" is used when the error amount is an extremely large abnormal value. It is also possible to manage the finished form by changing the color of the rebar in the CIM model according to the amount of error, for example by coloring the "thin solid line" of normal values gray, the "dash line" of slight deviation green, the "dash line" of deviation yellow, and the "thick line" of abnormal values red. This allows workers to visually recognize the results of the rebar arrangement inspection using the finished form report for rebars 31 and 32.

Furthermore, if the imaging device 20 is equipped with a distance measurement sensor, and multiple reinforcing bar layers are provided with reinforcing bars 31, 32 arranged in a lattice pattern as one reinforcing bar layer, the image processing unit 112 can also draw multiple reinforcing bar layers along the depth direction when creating a completed form document.

Conventionally, in a reinforcement inspection, a plurality of workers used a scale at the reinforcement position to measure the inspection items of the reinforcing bars 31, 32, such as the reinforcement pitch, one by one. In contrast, in the embodiment described above, the image data of the reinforcement bar captured by the imaging device 20 is acquired, and a detection frame 33a in which the intersections 33 are extracted based on the intersection identification learning model 121 is added to the reinforcement image data to generate intersection image data, and the coordinate information of the intersections 33 in the intersection image data is derived to determine the arrangement information of the reinforcing bars 31, 32. That is, according to one embodiment, the worker does not need to perform measurements manually, and based on the reinforcement image data captured by the imaging device 20, the intersections 33 that intersect in the reinforcing bars 31, 32 are extracted over the entire area to derive coordinate information, and various items related to the reinforcing bars 31, 32 can be measured based on the coordinate information of the intersections 33. Therefore, in a reinforcement inspection, the amount of data to be processed can be reduced, and reliability can be improved while reducing manpower. Furthermore, when the inventors performed a bar arrangement inspection process using the bar arrangement inspection device 10 according to the above-mentioned embodiment, a 75% time reduction and labor saving effect was confirmed.

Although one embodiment of the present invention has been specifically described above, the present invention is not limited to the above-described embodiment, and various modifications based on the technical concept of the present invention are possible. For example, the numerical values given in the above-described embodiment are merely examples, and numerical values different from these may be used as necessary, and the present invention is not limited by the description and drawings that form part of the disclosure of the present invention according to this embodiment.

For example, in the embodiment described above, the intersection identification learning model 121 is stored in the memory unit 12, but it is also possible to store the intersection identification learning model 121 in the memory unit of a reinforcement image server that can communicate with the reinforcement inspection device 10 via the network 2. In this case, the reinforcement image data captured by the imaging device 20 is transmitted to the reinforcement image server via the network 2 and stored in the memory unit. Thereafter, in response to a request from the reinforcement inspection device 10, the control unit of the reinforcement image server adds a detection frame 33a to the reinforcement image data to generate intersection image data and transmits it to the reinforcement inspection device 10. The reinforcement inspection device 10 can then execute the above-mentioned reinforcement inspection process on the received reinforcement image data.

The memory unit 12 can also be stored in a storage server that can communicate via the network 2. In this case, in response to a request from the reinforcement inspection device 10 to the storage server, reinforcement image data, intersection image data, intersection data, and inference data can be transmitted to the reinforcement inspection device 10.

The image acquisition unit 111, image processing unit 112, intersection detection unit 113, joint detection unit 114, clustering processing unit 115, and learning unit 116 in the control unit 11 of the reinforcement inspection device 10 may each be provided in an individual device capable of communicating with each other via network 2, or the necessary processing units may be combined and provided in separate devices via network 2.

In addition, for example, in the above-described embodiment, deep learning using a neural network is used as an example of machine learning for generating the intersection discrimination learning model 121, but machine learning based on other methods may also be performed. For example, other supervised learning such as a support vector machine, decision tree, naive Bayes, or k-nearest neighbor may also be used. Also, semi-supervised learning may be used instead of supervised learning. Also, in addition to hierarchical clustering, known clustering processing methods such as the k-means method may be used for the clustering process.

(recoding media)
In the above-described embodiment, the program for executing the reinforcing bar inspection device 10 or the processing method executed by the reinforcing bar inspection device 10 can be recorded on a recording medium readable by a computer or other machine or device such as a wearable device (hereinafter referred to as a computer, etc.). By having a computer, etc. read and execute the program from the recording medium, the computer, etc. functions as a mobile object control device. Here, a recording medium readable by a computer, etc. refers to a non-transient recording medium that accumulates information such as data and programs by electrical, magnetic, optical, mechanical, or chemical action and can be read by a computer, etc. Among such recording media, those that can be removed from a computer, etc. include, for example, a flexible disk, a magneto-optical disk, a CD-ROM, a CD-R/W, a DVD, a BD, a DAT, a magnetic tape, and a memory card such as a flash memory. In addition, a hard disk, a ROM, etc. are examples of recording media fixed to a computer, etc. Furthermore, an SSD can be used as a recording medium that can be removed from a computer, etc., and as a recording medium fixed to a computer, etc.

In addition, the reinforcing bar inspection device 10 according to one embodiment and the program executed by the reinforcing bar inspection device 10 may be stored on a computer connected to a network such as the Internet and provided by downloading via the network.

Other Embodiments
In one embodiment, the above-mentioned "unit" can be read as a "circuit" etc. For example, the control unit can be read as a control circuit.

Note that in the explanation of the flowcharts in this specification, the order of processing between steps is clearly indicated using expressions such as "first," "then," and "continue." However, the order of processing required to implement this embodiment is not uniquely determined by these expressions. In other words, the order of processing in the flowcharts described in this specification can be changed as long as there are no contradictions.

Further advantages and modifications may be readily derived by those skilled in the art. The broader aspects of the present disclosure are not limited to the specific details and representative embodiments shown and described above. Thus, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof.

REFERENCE SIGNS LIST 1 Reinforcement inspection system 2 Network 10 Reinforcement inspection device 11 Control unit 12 Memory unit 13 Input unit 14 Output unit 20 Imaging device 21 Imaging unit 30 Reinforced concrete structure 31 Vertical reinforcing bars 32, 32a, 32b Horizontal reinforcing bars 33 Intersections 33a, 33aa, 33ab Detection frame 111 Image acquisition unit 112 Image processing unit 113 Intersection detection unit 114 Joint detection unit 115 Clustering processing unit 116 Learning unit 121 Intersection identification learning model 122 Clustering code 123 Image data 124 Inference data 125 Intersection data

Claims (14)

An information processing device including a control unit that performs image processing on reinforcement image data obtained by capturing an image of a plurality of reinforcing bars in a reinforced concrete structure in which the reinforcing bars are arranged crosswise,
The control unit is
The captured reinforcement image data is acquired and stored in a storage unit,
Identifying a plurality of intersections of the reinforcing bars from the reinforcing bar image data read from the storage unit;
generating intersection data including coordinate information of the identified intersections ;
acquiring the reinforcement image data from the storage unit as an input parameter, and inputting the reinforcement image data read from the storage unit into an intersection identification learning model;
Outputting intersection image data to which a detection frame for extracting the intersections is added to the reinforcement image data as an output parameter;
generating the intersection data based on the output intersection image data;
The intersection identification learning model is a learning model generated by machine learning using the reinforcement image data as a learning input parameter and intersection image data in which a detection frame indicating the intersection is added to the reinforcement image data as a learning output parameter . The reinforced concrete structure includes a plurality of continuous reinforcement bars,
A reinforcing bar number unique to the consecutive reinforcing bars is set for each of the consecutive reinforcing bars;
The control unit is
The information processing device according to claim 1 , further comprising: performing clustering based on coordinate information of a plurality of intersections included in the intersection data, and dividing a plurality of intersections determined to have the same reinforcing bar number from the identified plurality of intersections into subsets. An information processing device including a control unit that performs image processing on reinforcement image data obtained by capturing an image of a plurality of reinforcing bars in a reinforced concrete structure in which the reinforcing bars are arranged crosswise,
The reinforced concrete structure includes a plurality of continuous reinforcement bars,
A reinforcing bar number unique to the consecutive reinforcing bars is set for each of the consecutive reinforcing bars;
The control unit is
The captured reinforcement image data is acquired and stored in a storage unit,
Identifying a plurality of intersections of the reinforcing bars from the reinforcing bar image data read from the storage unit;
generating intersection data including coordinate information of the identified intersections;
Clustering is performed based on coordinate information of a plurality of intersections included in the intersection data, and a plurality of intersections determined to have the same reinforcing bar number are divided into subsets from among the identified plurality of intersections.
Information processing device. The reinforced concrete structure includes a plurality of continuous reinforcement bars,
A reinforcing bar number unique to the consecutive reinforcing bars is set for each of the consecutive reinforcing bars;
The control unit is
Among the identified intersections, a plurality of intersections that are determined to have the same reinforcing bar number are extracted, and a reinforcing bar formula that approximates the plurality of intersections that are determined to have the same reinforcing bar number is derived for each of the reinforcing bar numbers;
The information processing device according to claim 2 or 3, further comprising: a processor that determines whether the reinforcing bar number is correct for each of a plurality of intersections that have been determined to have the same reinforcing bar number, based on a plurality of reinforcing bar formulas derived for each reinforcing bar number and the coordinate information of each of the plurality of intersections that have been determined to have the same reinforcing bar number. The control unit is
The information processing device according to claim 4 , wherein, when it is determined that the reinforcing bar number of the intersection is different, the reinforcing bar number of the intersection is corrected based on the reinforcing bar formula of the reinforcing bar number before or after the reinforcing bar number at which the intersection is determined. The control unit is
Calculating the distance between each of the identified intersections;
extracting two intersections whose distance between the intersections is equal to or less than a predetermined distance;
The information processing device according to claim 1 , further comprising: determining that the two intersections are intersections of joint portions of the reinforcing bars. An information processing device including a control unit that performs image processing on reinforcement image data obtained by capturing an image of a plurality of reinforcing bars in a reinforced concrete structure in which the reinforcing bars are arranged crosswise,
The control unit is
The captured reinforcement image data is acquired and stored in a storage unit,
Identifying a plurality of intersections of the reinforcing bars from the reinforcing bar image data read from the storage unit;
generating intersection data including coordinate information of the identified intersections;
Calculating the distance between each of the identified intersections;
extracting two intersections whose distance between the intersections is equal to or less than a predetermined distance;
The two intersections are determined to be intersections of the joints of the reinforcing bars.
Information processing device. The reinforcing steel structure includes a continuous reinforcing steel bar,
The continuous reinforcing bar has a joint portion where at least two reinforcing bars are fixed along a longitudinal direction,
A joint number unique to the joint portion is set for each of the joint portions,
The control unit is
By performing clustering based on coordinate information of the intersections of the plurality of joint parts, a plurality of intersections determined to have the same joint number are extracted and grouped together from the intersections of the plurality of joint parts;
The information processing device according to claim 6 or 7 , wherein a joint length is calculated for each of the joint numbers. An information processing method executed by an information processing device having a control unit that performs image processing on reinforcement image data obtained by capturing an image of a plurality of reinforcing bars in a reinforced concrete structure in which the reinforcing bars are arranged crosswise,
The captured reinforcement image data is acquired and stored in a storage unit,
Identifying a plurality of intersections of the reinforcing bars from the reinforcing bar image data read from the storage unit;
generating intersection data including coordinate information of the identified intersections ;
acquiring the reinforcement image data from the storage unit as an input parameter, and inputting the reinforcement image data read from the storage unit into an intersection identification learning model;
Outputting intersection image data to which a detection frame for extracting the intersections is added to the reinforcement image data as an output parameter;
generating the intersection data based on the output intersection image data;
The intersection identification learning model is a learning model generated by machine learning using the reinforcement image data as a learning input parameter and intersection image data in which a detection frame indicating the intersection is added to the reinforcement image data as a learning output parameter . An information processing method executed by an information processing device having a control unit that performs image processing on reinforcement image data obtained by capturing an image of a plurality of reinforcing bars in a reinforced concrete structure in which the reinforcing bars are arranged crosswise,
The reinforced concrete structure includes a plurality of continuous reinforcement bars,
A reinforcing bar number unique to the consecutive reinforcing bars is set for each of the consecutive reinforcing bars;
The control unit is
The captured reinforcement image data is acquired and stored in a storage unit,
Identifying a plurality of intersections of the reinforcing bars from the reinforcing bar image data read from the storage unit;
generating intersection data including coordinate information of the identified intersections;
Clustering is performed based on coordinate information of a plurality of intersections included in the intersection data, and a plurality of intersections determined to have the same reinforcing bar number are divided into subsets from among the identified plurality of intersections.
Information processing methods. An information processing method executed by an information processing device having a control unit that performs image processing on reinforcement image data obtained by capturing an image of a plurality of reinforcing bars in a reinforced concrete structure in which the reinforcing bars are arranged crossing each other,
The control unit is
The captured reinforcement image data is acquired and stored in a storage unit,
Identifying a plurality of intersections of the reinforcing bars from the reinforcing bar image data read from the storage unit;
generating intersection data including coordinate information of the identified intersections;
Calculating the distance between each of the identified intersections;
extracting two intersections whose distance between the intersections is equal to or less than a predetermined distance;
The two intersections are determined to be intersections of the joints of the reinforcing bars.
Information processing methods. A control unit that performs image processing on reinforcement image data obtained by capturing an image of a reinforced bar in a reinforced concrete structure in which a plurality of reinforced bars are intersected,
The captured reinforcement image data is acquired and stored in a storage unit,
Identifying a plurality of intersections of the reinforcing bars from the reinforcing bar image data read from the storage unit;
generating intersection data including coordinate information of the identified intersections ;
Acquire the reinforcement image data from the storage unit as an input parameter,
The reinforcement image data read from the storage unit is input to an intersection identification learning model, which is a learning model generated by machine learning, using the reinforcement image data as a learning input parameter and intersection image data in which a detection frame indicating the intersection is added to the reinforcement image data as a learning output parameter,
Outputting intersection image data to which a detection frame for extracting the intersections is added to the reinforcement image data as an output parameter;
A program for causing the computer to execute the step of generating the intersection data based on the output intersection image data . A control unit performs image processing on reinforcement image data obtained by capturing an image of a reinforced concrete structure in which a plurality of reinforcing bars are arranged crosswise and continuous, and a unique reinforcing bar number is assigned to each of the continuous reinforcing bars,
The captured reinforcement image data is acquired and stored in a storage unit,
Identifying a plurality of intersections of the reinforcing bars from the reinforcing bar image data read from the storage unit;
generating intersection data including coordinate information of the identified intersections;
Clustering is performed based on coordinate information of a plurality of intersections included in the intersection data, and a plurality of intersections determined to have the same reinforcing bar number are divided into subsets from among the identified plurality of intersections.
A program to make that happen. A control unit that performs image processing on reinforcement image data obtained by capturing an image of a reinforced bar in a reinforced concrete structure in which a plurality of reinforced bars are intersected,
The captured reinforcement image data is acquired and stored in a storage unit,
Identifying a plurality of intersections of the reinforcing bars from the reinforcing bar image data read from the storage unit;
generating intersection data including coordinate information of the identified intersections;
Calculating the distance between each of the identified intersections;
extracting two intersections whose distance between the intersections is equal to or less than a predetermined distance;
The two intersections are determined to be intersections of the joints of the reinforcing bars.
A program to make that happen.

Images