JP7438220B2 - Reinforcing bar determination device and reinforcing bar determination method - Google Patents

Description

The present invention relates to a reinforcing bar determination device and a reinforcing bar determining method.

At construction and civil engineering sites, construction inspections of structures such as buildings and civil engineering structures are performed. For example, in construction work for reinforced concrete buildings, reinforcing bars are inspected according to a reinforcing drawing before concrete is poured. In the reinforcement arrangement inspection, for example, the diameter and number of reinforcing bars used, the spacing between the reinforcing bars, etc. are specified, and an inspection is performed to see if the reinforcing bars are installed according to the reinforcement drawing.

Additionally, in order to support reinforcement inspection, technology has been developed to collect information about reinforcing bars used in construction work from images of the reinforcement arrangement. For example, a technique related to specifying the diameter of reinforcing bars to be inspected is known (for example, Patent Document 1).

Japanese Patent Application Publication No. 2015-001146

However, information on reinforcing bars collected through reinforcement inspection may require high accuracy. For example, when inspecting the strength of a structure, the diameter of reinforcing bars is an important parameter that determines the structural strength, and accurate measurement is required in reinforcing bar arrangement inspections. Therefore, it is desired to provide a technology that can identify information regarding reinforcing bars with high accuracy from images of reinforcing bars.

In one aspect, the present invention aims to improve the accuracy of identifying information regarding reinforcing bars from an image in which reinforcing bars are captured.

A reinforcing bar determination device according to one aspect of the present invention associates at least one of a sample reinforcing bar image showing reinforcing bars and feature information extracted from the sample reinforcing bar image with type information indicating the type of reinforcing bars appearing in the sample reinforcing bar image. a storage unit that stores reinforcing bar information including information; an input unit that receives input of a reinforcing bar image showing the reinforcing bar to be determined; and an evaluation unit that evaluates the similarity between the sample reinforcing bar image and the reinforcing bar image based on the reinforcing bar information. , an output unit that outputs the sample reinforcing bar image and corresponding type information when the reinforcing bar image and the sample reinforcing bar image satisfy a predetermined condition and are similar based on the similarity evaluation result; The information includes the type classified based on the diameter of the reinforcing bars, and the evaluation unit calculates a sample reinforcing bar image and a predetermined frequency of a frequency spectrum obtained by Fourier transforming the pixel values of the reinforcing bars in the longitudinal direction of the reinforcing bars. The method is characterized in that the degree of similarity between the sample reinforcing bar image and the reinforcing bar image is evaluated based on the degree of similarity of waveforms in a range and the degree of similarity of frequencies of a plurality of peaks included in the frequency spectrum .

According to one aspect, it is possible to improve the accuracy of identifying information regarding reinforcing bars from an image showing reinforcing bars.

FIG. 3 is a diagram illustrating an example of photographing a reinforced reinforcing bar that has been constructed according to an embodiment. FIG. 1 is a diagram illustrating a reinforcing bar determination system according to an embodiment. FIG. 1 is a diagram illustrating a block configuration of a reinforcing bar determination device according to an embodiment. FIG. 3 is a diagram illustrating reinforcing bar determination processing according to the embodiment. FIG. 3 is a diagram illustrating collection of information regarding reinforcing bars from images according to an embodiment. FIG. 3 is a diagram illustrating generation of a learned model according to the embodiment. FIG. 3 is a diagram illustrating an exemplary head-to-head transformation. FIG. 3 is a diagram illustrating rotation of an image showing reinforcing bars. FIG. 3 is a diagram illustrating reinforcing bar information according to the first embodiment. FIG. 3 is a diagram illustrating an operation flow of type determination processing according to the first embodiment. FIG. 3 is a diagram illustrating different shapes of reinforcing bars depending on the rotation angle. FIG. 7 is a diagram illustrating reinforcing bar information according to the second embodiment. FIG. 7 is a diagram illustrating reinforcing bar information according to a third embodiment. FIG. 3 is a diagram illustrating a type determination using images in multiple stages. It is a figure which illustrates the test|inspection information based on 5th Embodiment. FIG. 3 is a diagram illustrating noise in a frequency spectrum caused by paint on the surface of a reinforcing bar image. FIG. 7 is a diagram illustrating reinforcing bar type determination according to the seventh embodiment. 1 is a diagram illustrating a hardware configuration of a computer for realizing a reinforcing bar determination device according to an embodiment.

Hereinafter, some embodiments of the present invention will be described in detail with reference to the drawings. Note that the same reference numerals are given to corresponding elements in a plurality of drawings.

As mentioned above, technology is being developed to collect information about reinforcing bars used in construction from images of reinforcing bars.

Here, the JIS (Japanese Standards Association) standard defines the maximum value of the node spacing of reinforcing bars with respect to the diameter of the reinforcing bars. Therefore, one method for identifying information about reinforcing bars used in construction from images of reinforcing bars is to prepare information that associates reinforcing bar diameters with node spacing for multiple reinforcing bars with different diameters. do. It is conceivable that the node spacing of the reinforcing bars is identified from the image, and the diameter of the reinforcing bars associated with the identified node spacing is estimated as the diameter of the reinforcing bars in the image.

However, the node spacing of reinforcing bars varies depending on the lot, or even in images taken of the same reinforcing bar, there may be differences in the node spacing of reinforcing bars detected from the images depending on the imaging conditions.

In addition, as mentioned above, the JIS (Japanese Standards Association) standard specifies the maximum value of the node spacing of reinforcing bars relative to the diameter of the reinforcing bars, and only sets an upper limit. Even if the reinforcing steel is made by different manufacturers, the node spacing may be different. Furthermore, there are various types of deformed reinforcing bars; for example, bamboo-knotted reinforcing bars and threaded-knotted reinforcing bars may have different diameters even if the knot spacing is the same. Even if the spacing between the nodes of the reinforcing bars is similar in this way, the reinforcing bars may be of different types, and it may be difficult to accurately obtain information about the reinforcing bars based on the knot spacing. Therefore, it is desired to provide a technology that can more accurately identify the type of reinforcing bars from images of reinforcing bars.

In the embodiment described below, information in which a sample reinforcing bar image is associated with type information indicating the type of reinforcing bar shown in the sample reinforcing bar image is stored in advance in a storage device. Then, when an image showing the reinforcing bars whose type is to be judged in a reinforcement inspection etc. is input, a reinforcing bar image is created by extracting the area of the reinforcing bars to be judged from that image, and the reinforcing bar image and the sample reinforcing bars are Evaluate the similarity with the image. Based on the similarity evaluation results, if the reinforcing bar image to be determined and the sample reinforcing bar image satisfy a predetermined condition and are similar, the type information associated with the sample reinforcing bar image is added to the reinforcing bar image. Output as the type of reinforcing bar shown in .

In this manner, according to the embodiment, by evaluating the similarity of images, a sample reinforcing bar image similar to the reinforcing bar image to be determined is identified, and thereby the type of reinforcing bar appearing in the reinforcing bar image to be determined is determined. Therefore, various feature information that can be extracted from images can be used to evaluate the degree of similarity, and it is possible to determine the type of reinforcing bars in the reinforcing bar image to be determined with high accuracy. Hereinafter, embodiments will be described in more detail.

FIG. 1 is a diagram illustrating photographing of installed reinforcing bars according to an embodiment. As shown in FIG. 1, a user uses a photographing device 102 to photograph reinforcing bars 101 to be inspected at a construction site. The photographing device 102 may be, for example, a monocular camera or a stereo camera.

FIG. 2 is a diagram illustrating a reinforcing bar determination system 200 according to the embodiment. The reinforcing bar determination system 200 includes, for example, an imaging device 102 and a reinforcing bar determining device 201. The imaging device 102 and the reinforcing bar determination device 201 may be connected via a network 205, for example. The network 205 is, for example, a LAN (Local Area Network) and the Internet. In another embodiment, the imaging device 102 and the reinforcing bar determination device 201 may be connected by short-range wireless communication or by wire, or the imaging device 102 and the reinforcing bar determining device 201 may be integrated as a single device. Good too.

The reinforcing bar determination device 201 is, for example, a computer with a calculation function, and is, for example, a personal computer (PC), a notebook PC, or a tablet terminal. For example, the reinforcing bar determination device 201 may acquire an image of reinforcing bars photographed by the imaging device 102, analyze the reinforcing bars shown in the image, and collect information on reinforcing bars used in the reinforcing bars.

FIG. 3 is a diagram illustrating a block configuration of the reinforcing bar determination device 201 according to the embodiment. The reinforcing bar determination device 201 includes, for example, a control section 301, a storage section 302, a display section 303, and a communication section 304. The control unit 301 controls each part of the reinforcing bar determination device 201, for example. For example, the control unit 301 operates as an input unit 311, an evaluation unit 312, an output unit 313, and a determination unit 314. The storage unit 302 may store, for example, an image photographed by the photographing device 102, reinforcing bar information to be described later, a sample reinforcing bar image, feature information extracted from the sample reinforcing bar image, and the like. The display unit 303 is, for example, a display device such as a liquid crystal display. The display unit 303 displays information on the display screen according to instructions from the control unit 301. The communication unit 304 communicates with the imaging device 102 according to instructions from the control unit 301, for example. The details of each of these units and the information stored in the storage unit 302 will be described later.

FIG. 4 is a diagram illustrating reinforcing bar determination processing according to the embodiment. As shown in FIG. 4, reinforcing bar information 401 is stored in the storage unit 302 of the reinforcing bar determination device 201. In the example of reinforcing bar information 401 in FIG. 4, reinforcing bar types such as D41 and D25, which indicate reinforcing bar diameters as type information, and sample reinforcing bar images taken of reinforcing bars of that type are associated with each other for each of multiple different types of reinforcing bars. is registered.

Further, FIG. 4 shows an image 403 of reinforcing bars obtained by a user photographing installed reinforcing bars during a reinforcing bar arrangement inspection, for example. Then, the control unit 301 creates a reinforcing bar image 404 by, for example, extracting from the image 403 an image area in which the reinforcing bar whose type is to be determined is captured. Then, the control unit 301 evaluates the degree of similarity between the reinforcing bar image 404 and each sample reinforcing bar image registered in the reinforcing bar information 401. Next, based on the similarity evaluation result, the control unit 301 identifies a sample reinforcing bar image similar to the reinforcing bar image 404 to be determined, and sets the type information associated with the sample reinforcing bar image to the reinforcing bar to be determined. Output as the type. For example, in FIG. 4, the reinforcing bar D41 is evaluated as the reinforcing bar with the highest degree of similarity, and it can be specified that the diameter of the reinforcing bar to be determined is D41. The similarity between images can be evaluated using various feature information extracted from the images, so it is possible to evaluate the similarity using feature information with high evaluation accuracy, or to By using this method to evaluate the degree of similarity, it is possible to improve the accuracy of type determination. Some examples of similarity evaluation will be described below.

[Example of similarity evaluation]
(Evaluation example 1)
In evaluation example 1, information related to the node spacing and width of reinforcing bars is collected from images. FIG. 5 is a diagram illustrating collection of information regarding reinforcing bars from images according to the embodiment. As shown in FIG. 5A, the control unit 301 may, for example, perform Fourier transform on the pixel values of the image in the longitudinal direction of the reinforcing bars, and obtain information on the maximum peak frequency by removing the DC component. It is estimated that among the frequency components of the pixel values in the longitudinal direction of the reinforcing bars, the maximum peak frequency excluding the DC component corresponds to the node interval. Therefore, the control unit 301 identifies, for example, the frequency of the maximum peak excluding the DC component in the frequency spectrum obtained by Fourier transform in the longitudinal direction of the reinforcing steel. The frequency of the largest peak may, in one example, be identified as the frequency of the maximum intensity in the frequency spectrum. Note that the pixel value may be a value related to a pixel of an image, such as a brightness value and a color value.

Further, as shown in FIG. 5A, the control unit 301 detects edges of the reinforcing bars that extend substantially parallel to the longitudinal direction of the reinforcing bars that appear in the image. In one example, the control unit 301 applies edge detection processing to the image, searches for edges substantially parallel to the longitudinal direction of the reinforcing bars in a direction perpendicular to the longitudinal direction of the reinforcing bars, and searches for edges of the reinforcing bars in a direction perpendicular to the longitudinal direction of the reinforcing bars. May be detected. Edge detection can be performed using various existing techniques, such as a technique using a Sobel filter, for example. Then, the control unit 301 may specify the distance between the edges of both ends of the detected reinforcing bar that are substantially parallel to the longitudinal direction as the width of the reinforcing bar that appears in the image.

Then, the control unit 301 compares information on the frequency of the identified maximum peak and information on the distance between edges between the reinforcing bar image to be determined and the sample reinforcing bar image, and evaluates the degree of similarity between the two images. do. In one example, the control unit 301 uses the sum value (for example, weighted addition) of the difference in frequency of the maximum peak of the two images and the difference in distance between the edges of the two images as an index for evaluating the degree of similarity. May be used as That is, in this case, the control unit 301 may evaluate that the smaller the total value, the higher the similarity between the sample reinforcing bar image and the reinforcing bar image to be determined.

As described above, in evaluation example 1, the frequency of the maximum peak extracted from the image and the distance between edges are used as feature information to determine the similarity between the reinforcing bar image to be determined and the sample reinforcing bar image. We are evaluating the degree. Therefore, for example, even if reinforcing bars have similar node spacing, they can be distinguished based on information in the width direction, making it possible to more accurately evaluate the degree of similarity and determine the type of reinforcing bars to be determined. Can be done.

(Evaluation example 2)
In evaluation example 2, information on the frequencies of a plurality of peaks is acquired from among the peaks excluding the DC component of the frequency spectrum obtained by Fourier transforming the pixel values of the image in the longitudinal direction of the reinforcing bars. FIG. 5B shows an example in which information on a plurality of frequencies is collected as feature information from the frequency spectrum according to the embodiment. As shown in FIG. 5B, the control unit 301 may also collect frequencies of peaks other than the maximum peak and use them for evaluating the degree of similarity. By performing a comparison using information on a plurality of frequencies, it is possible to compare frequency components based on factors other than the node spacing, such as minute differences in the shape of nodes between manufacturers or lots of the same manufacturer. As a result, it is possible to more accurately evaluate the degree of similarity between the reinforcing bar image to be determined and the sample reinforcing bar image.

Note that the plurality of frequency components to be compared can be determined in various ways. For example, a predetermined number of peaks with higher peak intensities may be extracted from among multiple frequency components of a frequency spectrum obtained by Fourier transforming the pixel values of the image in the longitudinal direction of the reinforcing steel, or a predetermined number of peaks with higher peak intensity may be extracted. A predetermined number of peaks having an intensity equal to or higher than a predetermined threshold value may be extracted from the predetermined number of peaks. Further, the control unit 301 may compare peaks in the same order of peak extraction, such as comparing peaks between the determination target reinforcing bar image and the sample reinforcing bar image in order from the low frequency side. In another embodiment, the comparison may be performed between the closest frequency peaks. Then, the control unit 301 may, for example, integrate the magnitudes of the differences between the frequencies of the compared peaks, and use the total value as an index for evaluating the degree of similarity. That is, in this case, the control unit 301 may evaluate that the smaller the total value is, the higher the similarity between the sample reinforcing bar image and the determination target reinforcing bar image is.

(Evaluation example 3)
The degree of similarity may be evaluated by comparing waveforms obtained by Fourier-transforming pixel values of images in the longitudinal direction of the reinforcing bars between the reinforcing bar image to be determined and the sample reinforcing bar image. As an example, a product moment correlation coefficient can be used to evaluate the similarity of waveforms. For example, suppose there are n sets of data (x ₁ , y ₁ ), (x ₂ , y ₂ ),...,(x _n , y _n ), each consisting of x and y. In this case, the correlation between the pair of x and y can be calculated using the following formula for Pearson's product moment correlation coefficient r _xy .

Here, in the above formula, overlined x and y represent the average values of x and y, respectively. Furthermore, in the formula, s _x represents the standard deviation of x, s _y represents the standard deviation of y, and s _xy represents the covariance of x and y. For example, the frequency and corresponding intensity of the waveform of the frequency spectrum obtained by Fourier transforming the pixel values in the longitudinal direction of the reinforcing bars in a sample reinforcing bar image of a certain reinforcing bar type are expressed as the parameter expressed by x in the above equation. do. Further, the intensity corresponding to the frequency of the waveform of the frequency spectrum obtained by Fourier transforming the pixel values in the longitudinal direction of the reinforcing bars in the reinforcing bar image to be determined is the parameter expressed by y in the above equation. In this case, by inputting the intensities at the same frequency in the frequency spectra obtained from the two images into the above equation as a set, the correlation coefficient r _xy can be calculated, and _the obtained r It can be used as an evaluation index.

In this way, by comparing the Fourier-transformed waveforms, more information can be used, so it is possible to use the Comparisons of frequency components can be performed. As a result, it is possible to more accurately evaluate the degree of similarity between the reinforcing bar image to be determined and the sample reinforcing bar image.

(Evaluation example 4)
In addition to information on the frequency of pixel values in the longitudinal direction of the reinforcing bars in the image, information on the frequencies of pixel values in the width direction of the reinforcing bars perpendicular to the longitudinal direction of the reinforcing bars may be used for evaluating the similarity. FIG. 5C is a diagram illustrating a frequency spectrum of pixel values in the width direction. Note that in FIG. 5C, a plurality of pixel values in the width direction are obtained at different positions in the longitudinal direction of the reinforcing bars in the image, and the pixel values are combined and used to generate the frequency spectrum in the width direction. Thereby, more characteristics in the width direction of the reinforcing bar can be acquired than in collecting characteristics of pixel values in the width direction at one position, and it is possible to improve the accuracy of determining the type of reinforcing bar. Note that compositing can be performed in various ways; for example, it may be performed by joining pixel values in the width direction at multiple positions into one, or by taking the average of the pixel values. May be executed. However, embodiments may collect widthwise pixel value features at one location.

Then, the control unit 301 uses the frequency spectrum of the pixel values in the longitudinal direction of the reinforcing bars and the frequency spectrum of the pixel values in the width direction to evaluate the similarity between the reinforcing bar image to be determined and the sample reinforcing bar image. good. For example, as described above, the similarity of frequency spectra can be evaluated using the sum or average value of the differences in the frequencies of corresponding peaks in the frequency spectra obtained from the two images, the reinforcing bar image to be determined and the sample reinforcing bar image. This can be done by using as an indicator. Alternatively, the degree of similarity may be evaluated using a product-moment correlation coefficient using the intensity with respect to frequency of the frequency spectrum as input.

Further, in one example, the frequency spectrum of pixel values in the longitudinal direction and the frequency spectrum of pixel values in the width direction may be used for one similarity evaluation. In another example, first, a predetermined number of top sample reinforcing bar images are identified by evaluating the degree of similarity using the frequency spectrum of pixel values in the longitudinal direction. After that, similarity evaluation is performed individually using the frequency spectrum of pixel values in the width direction to identify the most similar sample reinforcing bar image, and similar sample reinforcing bars are Images may also be specified.

(Evaluation example 5)
The waveform of pixel values in the longitudinal direction of the reinforcing bars in the image may be used as is for evaluating the degree of similarity between the reinforcing bar image to be determined and the sample reinforcing bar image. In this case, the waveform obtained from the rebar image to be determined and the waveform obtained from the sample rebar image are combined at the position of the peak first detected from one end of the pixel value in the axial direction, such as the top or bottom of the image. After aligning the obtained waveforms, similarity evaluation using the waveforms may be performed. The similarity may be evaluated by, for example, obtaining and comparing the period of the peak in the waveform of the pixel value, or by using a product moment correlation coefficient using the pixel value with respect to the coordinate in the axial direction of the waveform as input. Good too.

(Evaluation example 6)
The degree of similarity may be evaluated by performing local feature matching between the reinforcing bar image to be determined and the sample reinforcing bar image. As the local feature amount, various local feature amounts such as SIFT (Scale-Invariant Feature Transform) and SURF (Speed-Up Robust Features) can be used.

(Evaluation example 7)
Phase information may be collected from the Fourier transform results and used to evaluate similarity. For example, it is conceivable to obtain phase information from the result of performing two-dimensional Fourier transform on a reinforcing bar image to be determined and a sample reinforcing bar image, and generate a phase image. In the phase image, information such as the outline of reinforcing bars is emphasized. For example, in a brightness image, the bright areas of the reinforcing bars may change depending on whether the reinforcing bars are illuminated from above or below. However, since the external shape is emphasized in the phase image, it is less affected by the angle of light irradiation with respect to the reinforcing steel during imaging. Therefore, it is possible to improve the accuracy of similarity evaluation by using phase images for comparison. In one example, the evaluation of the degree of similarity between the phase image obtained from the reinforcing bar image to be determined and the phase image obtained from the sample reinforcing bar image may be performed by matching using local feature amounts.

Furthermore, the phase information may be used to perform, for example, a phase-only correlation method or a rotation-invariant phase-only correlation method to evaluate the degree of similarity between the reinforcing bar image to be determined and the sample reinforcing bar image.

Above, several similarity evaluation methods have been illustrated. In evaluation examples 1 to 7 described above, the similarity is evaluated using, for example, Fourier transform results, frequency spectra, frequencies, distances between edges, waveforms, phase information, local feature amounts, etc. as feature information. are doing. Since all of these feature information are information obtained from images of reinforcing bars, it is possible to evaluate the similarity of images of reinforcing bars by evaluating the similarity using these feature information. However, the evaluation of the degree of similarity according to the embodiment is not limited to the above, and the degree of similarity may be evaluated using a combination of the above-mentioned methods or by other methods.

For example, in another embodiment, it is also possible to evaluate the degree of similarity between images using machine learning such as deep learning, and output type information representing the type of reinforcing steel in the image. In evaluation example 8 below, a case where machine learning is used will be exemplified.

(Evaluation example 8)
A learned model obtained by performing machine learning such as deep learning may be used to identify the type of reinforcing bar appearing in the reinforcing bar image to be determined. FIG. 6 is a diagram illustrating generation of a trained model according to the embodiment. For example, a sample reinforcing bar image showing a reinforcing bar and a plurality of pieces of information associated with type information indicating the type of reinforcing bar in the sample reinforcing bar image are used as training data (FIG. 6(a)). Then, for example, for a sample reinforcing bar image input to the input layer, a trained model is generated by learning the weighting of the neural network so that the type information of the reinforcing bars in the sample reinforcing bar image is output from the output layer. (Fig. 6(b)). By inputting the reinforcing bar image to be determined into the obtained trained model, the control unit 301 can receive the output of reinforcing bar image and corresponding reinforcing bar type information from the learned model (FIG. 6(c)) ). Further, as an example, learning may be performed using a Siamese network. A Siamese network is a network suitable for, for example, inputting two images and calculating the distance between whether the images are similar or dissimilar. Further, the machine learning method is not limited to deep learning, and support vector machine (SVM) or the like may be used.

Note that, for example, when a trained model is used, if a reinforcing bar image is input to the trained model and the type of reinforcing bar appearing in the reinforcing bar image has been determined, the trained model can be used as a teacher by using the reinforcing bar image and the type information of the determination result. As the data further learning may be performed.

For example, as described above, the degree of similarity between the reinforcing bar image to be determined and the sample reinforcing bar image can be evaluated using various methods.

[Normalization of reinforcing bar images]
Next, normalization of reinforcing bar images will be explained. For example, when evaluating the similarity between a reinforcing bar image to be judged in an image taken at a construction inspection site and a sample reinforcing bar image, it is better to The evaluation accuracy will be improved. For example, it is preferable for the reinforcing bars to be determined to have substantially the same orientation in the images as the reinforcing bars in the sample reinforcing bar image in terms of similarity comparison. Furthermore, it is preferable that the size of the reinforcing bars in the images be enlarged or reduced in the same ratio as the actual size of the reinforcing bars, since this allows comparison of the sizes of the reinforcing bars between images.

Therefore, for example, a user who photographs reinforcing bars to be judged at a construction inspection site may want to check if the shooting conditions such as the direction, distance, and focal length for photographing the reinforcing bars to be judged are the same as those used when capturing the sample reinforcing bar images. Photography may be performed to match the conditions within a predetermined error range. Thereby, it is possible to improve the accuracy of evaluating the similarity between the reinforcing bar image to be determined and the sample reinforcing bar image.

However, for example, suppose that there are images taken under different shooting conditions such as shooting direction, shooting distance, and focal length. Even in this case, it is possible to normalize the images, such as by aligning the shooting directions or changing the size of the reinforcing bars in the two images to reflect the ratio of the actual reinforcing bar sizes. Therefore, the control unit 301 determines, for example, the relative position and orientation of the imaging device that captured the image of the reinforcing bars and the reinforcing bars that appear in the reinforcing bar image, based on information regarding the relative position and orientation of the imaging device that captured the image of the reinforcing bars and the reinforcing bars that appear in the reinforcing bar image. The image showing the reinforcing bars may be normalized so that the position and orientation of the reinforcing bars have a predetermined relationship. Information regarding the relative position and orientation of the photographing device that captured the image of the reinforcing bars and the reinforcing bars that appear in the reinforcing bar image may be stored in the storage unit 302, for example. Hereinafter, as an example of image normalization, first, a description will be given of normalization of the photographing direction.

(Front-facing shooting direction)
For example, the control unit 301 may execute a process of converting the orientation of the reinforcing bars in an image obtained by photographing the reinforcing bars used for similarity evaluation into a predetermined orientation. In one example, the control unit 301 may change the orientation of the reinforcing bars in the image so that the reinforcing bars in the image directly face the optical axis of the imaging device. Note that facing directly may mean, for example, a position/posture relationship in which the normal direction of a plane including the reinforcing bars and the photographing direction (for example, the optical axis) of the photographing device are substantially parallel. At construction inspection sites, it may be difficult to take photographs from a position directly facing the reinforcing bars to be determined due to limitations in the photographing environment. In this case, the control unit 301 may perform a process of converting the image so that the orientation of the reinforcing bars to be determined in the image is the same as the orientation of the reinforcing bars when photographed from a position directly facing the image. Note that when there are multiple reinforcing bars to be determined that are arranged approximately parallel to each other in the image, the control unit 301 may determine, for example, the normal direction of the plane including the multiple reinforcing bars to be determined and the imaging direction of the imaging device. A process may be performed in which the image is squarely converted so that the images are substantially parallel to each other.

FIG. 7 is a diagram illustrating an exemplary head-to-head transformation. FIG. 7A illustrates a state in which a plane including a plurality of reinforcing bars to be determined is inclined with respect to the imaging direction of the imaging device 102, and the reinforcing bars are photographed in an oblique direction in the image. Further, FIG. 7(b) shows the state after face-to-face conversion, and the imaging direction of the imaging device 102 is approximately parallel to the normal direction of the plane containing the reinforcing steel group to be determined. The plane containing the reinforcing bars has been rotated.

Note that the head-to-head conversion can be performed, for example, as follows. For example, in an image taken obliquely with respect to a plane including a plurality of reinforcing bars 101 as shown in FIG. It may be accepted from the user. Then, the control unit 301 determines a homography matrix so that the planes of the rectangle indicated by the designated four corner points face each other directly, and uses the determined homography matrix to perform a face-to-face transformation of the entire image. It's fine.

Further, for example, assume that a reinforcing bar image showing a plurality of reinforcing bars to be determined is captured by a stereo camera. In this case, the control unit 301 can generate three-dimensional data corresponding to the pixels by, for example, performing stereo matching between the left viewpoint image and the right viewpoint image of the stereo image. Then, the control unit 301 controls the three-dimensional data of the plane so that the normal vector of the plane containing the reinforcing bars to be determined, represented by the obtained three-dimensional data, is approximately parallel to the imaging direction of the imaging device 102. Identify the matrix to rotate. Then, the control unit 301 specifies a homography matrix for converting the left viewpoint image or right viewpoint image of the stereo image to a facing direction based on the information of the determined matrix, and uses the identified homography matrix to A facing image may also be generated.

(Normalization of longitudinal direction of reinforcing bars)
It is preferable to perform similarity determination in such a manner that the longitudinal direction of the reinforcing bars in the reinforcing bar image to be determined is substantially parallel to the longitudinal direction of the reinforcing bars in the sample reinforcing bar image. Therefore, the control unit 301 may perform normalization by rotating the images such that the reinforcing bars image to be determined and the sample reinforcing bars image are approximately parallel in longitudinal direction.

FIG. 8 is a diagram illustrating the rotation of an image showing reinforcing bars. FIG. 8 illustrates a case where the longitudinal direction of the reinforcing bars in the image is rotated so as to align with the vertical direction. As shown in Figure 8, by rotating the longitudinal direction of the reinforcing bars in the image so that they align with a predetermined orientation such as the vertical direction, the longitudinal direction of the reinforcing bars in the sample reinforcing bar image and the reinforcing bar image to be determined are The longitudinal direction of the reinforcing bars in the image can be made approximately parallel. Thereby, the accuracy of similarity evaluation can be improved.

(Image size normalization)
Regarding the image size, it is also desirable to normalize the predetermined lengths of the reinforcing bars in the two images for which similarity is to be evaluated so that they represent the same length of the reinforcing bars in real space. Note that image size normalization can be performed, for example, as follows.

For example, if the image is taken with a stereo camera, the control unit 301 can acquire three-dimensional data corresponding to pixels by stereo matching. Then, the control unit 301 can specify how much the predetermined length on the reinforcing bar in the image corresponds to the actual length of the reinforcing bar in the three-dimensional data. Therefore, for example, using this information, the control unit 301 normalizes the image size so that a predetermined length on the reinforcing bar in the image corresponds to a predetermined actual length on the reinforcing bar in real space. can do.

For example, even if an image is taken with a monocular camera, if the focal length of the photographing device 102 used for photographing and the distance from the photographing device to the subject are known, the actual size corresponding to the size on the image can be determined. It is possible to obtain Therefore, using this information, the image size may be normalized so that a predetermined length on the reinforcing bar in the image corresponds to a predetermined actual length on the reinforcing bar in real space.

As described above, by normalizing the image size and comparing the sizes of the reinforcing bars in the images, it becomes possible to evaluate the similarity of the actual reinforcing bar sizes.

Note that, for example, the position between the photographing device and the subject used for normalization processing, such as three-dimensional data and information on the focal length of the photographing device 102 used to photograph the image and the distance from the photographing device 102 to the subject. Information regarding the posture may be stored in the storage unit 302. In another embodiment, the control unit 301 may acquire this information by having the user input the information, for example. Alternatively, the control unit 301 may acquire three-dimensional data or information on the photographing distance based on data obtained by scanning the reinforcing bars with a laser scanner. Furthermore, the control unit 301 may take an image by installing a predetermined scale or the like near the reinforcing steel whose type is to be determined, and obtain information about dimensions in real space from the image.

As described above, the accuracy of similarity evaluation can be improved by appropriately normalizing the reinforcing bar images used for similarity evaluation. Note that image normalization may be performed at various timings. For example, the control unit 301 may normalize the sample reinforcing bar image when registering the sample reinforcing bar image in the reinforcing bar information 401, or when evaluating the similarity with the image of the reinforcing bar to be determined. It may be executed at both timings.

Incidentally, in the example shown in FIG. 4 described above, a reinforcing bar image 404 is created by extracting an image area in which a reinforcing bar whose type is to be determined is captured from the image 403. The reinforcing bar image 404 may be created, for example, by having the user select an image area in which the reinforcing bars to be determined are shown, or in another embodiment, the control unit 301 automatically selects an image area in which the reinforcing bars to be determined are shown. may be detected and executed. Furthermore, the control unit 301 automatically executes the above-mentioned face-to-face conversion, normalization of the longitudinal direction of reinforcing bars, and normalization of image size, even if they are executed in response to information input from the user. You can. For example, the technology described in the above-mentioned Patent Document 1 or the technology described in International Publication No. 2018/180442 may be used for these normalization processes.

(First embodiment)
Next, the first embodiment will be described with reference to FIGS. 9 and 10. FIG. 9 is a diagram illustrating reinforcing bar information 900 according to the first embodiment. Reinforcing bar information 900 is an example of the above-mentioned reinforcing bar information 401. In the reinforcing bar information 900, for example, a record in which type information and sample reinforcing bar image information are associated is registered. In the example of FIG. 9, the type information includes information on manufacturer, diameter, and knot type. The manufacturer is information for identifying the manufacturer of the reinforcing bars appearing in the sample reinforcing bar image of the record. The diameter is, for example, information indicating the diameter of the reinforcing bar that appears in the sample reinforcing bar image of the record. The type of knot is, for example, information indicating the type of knot of the reinforcing bar that appears in the sample reinforcing bar image of the record, and in one example, information about bamboo knots or screw knots may be registered. Further, the sample reinforcing bar image information may include, for example, registered image data of the sample reinforcing bar image or information indicating a storage location of the sample reinforcing bar image in the storage unit 302. By referring to the reinforcing bar information 900, the control unit 301 can obtain information on a sample reinforcing bar image used for evaluating the degree of similarity with the reinforcing bar image to be determined.

Next, type determination processing according to the first embodiment will be explained. FIG. 10 is a diagram illustrating an operational flow of type determination processing according to the first embodiment. The control unit 301 of the reinforcing bar determination device 201 may, for example, receive an input of a reinforcing bar image whose type is to be determined, and upon receiving the input, start the operation flow shown in FIG. 10 . Note that the reinforcing bar image to be determined may be input to the control unit 301 from the imaging device 102, or may be input to the control unit 301 by reading out a reinforcing bar image designated by the user from the storage unit 302. .

In step 1001 (hereinafter, step will be referred to as "S", for example, S1001), the control unit 301 of the reinforcing bar determination device 201 refers to the reinforcing bar information 900 and reads out a plurality of sample reinforcing bar images. For example, the control unit 301 may read sample reinforcing bar images corresponding to all records registered in the reinforcing bar information 900.

In S1002, the control unit 301 evaluates the degree of similarity between the input reinforcing bar image to be determined and the plurality of read sample reinforcing bar images. The image similarity evaluation can be performed using various methods, and the control unit 301 may evaluate the similarity using, for example, the methods of evaluation example 1 to evaluation example 7 described above, or in combination. It's fine.

In S1003, the control unit 301 identifies sample reinforcing bar images that satisfy a predetermined condition and are similar, from among the plurality of read sample reinforcing bar images, based on the similarity evaluation result. For example, the predetermined condition may be that the sample reinforcing bar images are evaluated to be the most similar among the read sample reinforcing bar images based on a similarity evaluation result. In another embodiment, for example, in a case where the higher the index used to evaluate the similarity is, the more similar the index is, the predetermined condition is that the index used to evaluate the similarity is greater than or equal to a predetermined threshold. It may be that. On the other hand, if, for example, the lower the index used to evaluate the similarity is, the more similar it is, the predetermined condition is that the index used to evaluate the similarity is less than or equal to a predetermined threshold. good. The indicators used for similar evaluation are, for example, the total value of the differences described in evaluation example 1, the total value of the integrated magnitude of differences described in evaluation example 2, the product-moment correlation coefficient, and the matching of local features. The results may include execution results of the phase-only correlation method and the rotation-invariant phase-only correlation method.

In S1004, the control unit 301 specifies type information that is associated with a sample reinforcing bar image that satisfies a predetermined condition and is similar in the reinforcing bar information 401. Then, in S1005, the control unit 301 outputs the specified type information, and this operation flow ends. For example, assume that there are a plurality of sample reinforcing bar images that satisfy a predetermined condition and are similar to the reinforcing bar image to be determined. In this case, the control unit 301 may output all the type information of the reinforcing bars corresponding to each of the plurality of sample reinforcing bar images, or may output the type information of the reinforcing bar with the smallest diameter among them. Good too. By outputting the type information of the reinforcing bar with the smallest diameter size, even if the determined reinforcing bar type is incorrect, it is possible to estimate the minimum strength of the structure calculated using that type. .

(Second embodiment)
Next, a second embodiment will be described. Deformed reinforcing bars have ribs, knots, etc., and their shapes vary. For example, when the reinforcing bars are rotated around the longitudinal axis of the reinforcing bars and observed from a fixed position perpendicular to the longitudinal axis, the reinforcing bars have different shapes depending on the rotation angle, as shown in Figure 11. There is. For example, in FIG. 11(a), knots are shown throughout the reinforcing bar, but in FIG. 11(b), knots are not formed in a part of the reinforcing bar in the axial direction. In this way, when evaluating the similarity using images taken at different rotation angles with different shapes, the similarity will be low even if the similarity is evaluated between images of reinforcing bars of the same type. Sometimes. As a result, the accuracy of determining the type of reinforcing bar may decrease.

Therefore, in the second embodiment, as illustrated in FIG. 12, information on reinforcing bar images taken from a plurality of angles is registered in the sample reinforcing bar image information of the reinforcing bar information 1200 for one type. For example, for one type of reinforcing bar, a plurality of sample reinforcing bar images are photographed by rotating the longitudinal direction of the reinforcing bar as the rotation axis from a position directly facing the reinforcing bar. Then, in the reinforcing bar information 1200, information on a plurality of reinforcing bar images obtained by photographing reinforcing bars from various angles may be registered in sample reinforcing bar image information of a record identified by one type information.

For example, in the process of S1002 described above, the control unit 301 evaluates the degree of similarity between one record in the reinforcing bar information 1200 and each of a plurality of sample reinforcing bar images corresponding to the reinforcing bar image to be determined. Then, in S1003, the control unit 301 may identify a record of a sample reinforcing bar image that is similar to the reinforcing bar image while satisfying a predetermined condition, based on the plurality of similarity evaluation results. In one example, the control unit 301 selects the value of the index that is evaluated to be most similar to each of the plurality of sample reinforcing bar images evaluated in S1002 as the sample reinforcing bar image of the record type. It may be used as an index of similarity with

For example, by using sample reinforcing bar images taken from various angles to evaluate the similarity with the target reinforcing bar image and determining the type, even if the reinforcing bars have different shapes depending on the viewing angle, they can be similar. be able to appropriately evaluate the degree of Therefore, the accuracy of determining the type of reinforcing bar can be improved.

Further, although an example in which a plurality of sample reinforcing bar images are registered is described above, the embodiment is not limited to this. In another embodiment, the control unit 301 may receive input of a plurality of reinforcing bar images to be determined, and may evaluate the similarity between each of the plurality of reinforcing bar images and the sample reinforcing bar image in S1002. Then, in S1003, the control unit 301 may identify a sample reinforcing bar image that is similar to the plurality of reinforcing bar images and satisfying a predetermined condition, based on the plurality of similarity evaluation results. In one example, the control unit 301 selects the value of the index that is evaluated to be the most similar among the plurality of similarity indices between each of the plurality of reinforcing bar images evaluated in S1002 and the sample reinforcing bar image, according to the type of the record. It may be used as an index of similarity with the sample reinforcing bar image.

Note that the input multiple reinforcing bar images to be determined may be reinforcing bar images taken from different angles as described above, or in another example, multiple reinforcing bar images taken at approximately the same position. There may be. In this case, for example, effects such as camera shake caused by the photographer's movements during shooting may differ depending on the image, and different images may be taken due to fluctuations in lighting. obtain. Therefore, the accuracy of determining the type of reinforcing bar can be improved. Note that the plurality of reinforcing bar images that are input may be, for example, a plurality of images shot by continuous shooting, video shooting, or the like.

(Third embodiment)
In the above embodiment, for example, as shown in FIG. 9, information on a sample reinforcing bar image is registered in the sample reinforcing bar image information of the reinforcing bar information 900 in the storage unit 302, and the sample reinforcing bar image is stored in the storage unit 302. I explained an example of a case where However, embodiments are not limited thereto. For example, in the sample reinforcing bar image information of the reinforcing bar information 900, feature information extracted from the sample reinforcing bar image used for similarity evaluation may be registered instead of information on the sample reinforcing bar image.

FIG. 13 is a diagram illustrating reinforcing bar information 1300 according to the third embodiment. In the example of FIG. 13, the sample reinforcing bar image information of the reinforcing bar information 1300 stores a frequency spectrum obtained by Fourier transforming pixel values in the longitudinal direction of the reinforcing bars appearing in the sample reinforcing bar image.

For example, in the process of S1001 described above, the control unit 301 reads the frequency spectrum from the sample reinforcing bar image information of the reinforcing bar information 1300, and the similarity with the frequency spectrum obtained from the reinforcing bar image to be determined inputted in S1002. may be evaluated. By acquiring feature information used for similarity evaluation from sample reinforcing bar images in advance and registering it in reinforcing bar information 1300, it is possible to reduce the process of acquiring feature information from sample reinforcing bar images each time similarity evaluation is performed. can. Further, there is no need to store sample reinforcing bar images in the storage unit 302.

Note that the feature information registered in the sample reinforcing bar image information of the reinforcing bar information 1300 is not limited to the frequency spectrum, but may include, for example, other information used for evaluating the similarity in the above-mentioned evaluation examples 1 to 7. may be registered. For example, the feature information registered in the sample reinforcing bar image information is the result of Fourier transforming the sample reinforcing bar image, the frequency spectrum obtained from the Fourier transforming result, the frequency of the peak of the frequency spectrum, and the phase information extracted from the Fourier transforming result. , local features extracted from the sample reinforcing bar image or its phase image, and at least one piece of information such as the distance between edges at both ends substantially parallel to the axis of the reinforcing bar may be registered.

(Fourth embodiment)
The type determination using images may be performed in multiple stages. For example, as illustrated in FIG. 14, the control unit 301 first determines the sorting type, such as bamboo knots or screw knots. Subsequently, if it is determined that it is a bamboo knot, in the second step, the control unit 301 executes the operation flow of FIG. You may perform a type determination. Similarly, for example, if it is determined that the joint is a threaded joint, in the second step, the control unit 301 performs the operation shown in FIG. The flow may be executed to perform further type determination.

In this way, by determining the type in multiple stages using the sorting type, it is possible to determine the type using an evaluation method suitable for determining each type. It is also possible to determine the final type by combining them in stages. For example, it is effective for determining the shape of joints, such as extracting local features to determine screw joints and bamboo joints, or using a learning model created to determine these two types using machine learning. It is conceivable to first identify whether it is a screw knot or a bamboo knot using a suitable evaluation method. Then, in the second step, it is possible to perform a determination using, for example, the frequency spectrum of the pixel values in the longitudinal direction of the reinforcing bars in the image, which is effective for determining the knot spacing, and to determine the type of reinforcing bars.

In addition, various types of reinforcing bars can be used as sorting types for pre-sorting before the final type determination of reinforcing bars, and in another example, manufacturer's marks formed at nodes of reinforcing bars can be used. Processing may be performed to evaluate logos and the like and sort them by manufacturer. Furthermore, sorting may be performed multiple times in multiple stages by combining different sorting types.

(Fifth embodiment)
In S1005 of FIG. 10 of the above-described embodiment, the output destination for outputting the type information may be the display screen of the display unit 303 in one example, or a form for recording the inspection results of the reinforcement inspection in another example. There may be inspection information 1500.

FIG. 15 is a diagram illustrating inspection information 1500 such as a form for recording inspection results of reinforcement inspection according to the fifth embodiment. Inspection information 1500 may include, for example, registered information corresponding to a blueprint for construction in a certain area at a construction site, and includes, for example, construction identification information 1511 and inspection target information 1512.

The construction identification information 1511 includes, for example, information indicating the location of the construction and an outline of the construction. In the example of FIG. 15, the construction identification information 1511 includes information such as a number, construction name, construction type, structure number, and member name. The number is, for example, information given to the examination information 1500 for identification. The construction name may be, for example, the name of the construction. The work type is, for example, information indicating the location of the work, and for example, the work type: abutment frame work in FIG. 15 indicates that the work is for the foundation portion that supports the bridge piers. The structure number is, for example, information that specifies a structure that is the target of construction within the construction type. For example, the structure number A01 shown in FIG. 15 indicates which bridge pier the abutment frame work is for. The member name is, for example, information indicating an area targeted for construction within the structure. For example, the component name "Footing" shown in FIG. 15 indicates that the work is for a footing, which is a foundation part within the abutment frame work. For example, by referring to the construction identification information 1511 of the inspection information 1500, the worker can understand which construction the inspection information 1500 describes.

Further, in the inspection object information 1512, for example, a record that associates an inspection object with information on an inspection item is registered. The inspection target is, for example, information specifying the inspection target (for example, reinforcing bars) of the reinforcement inspection performed in the construction work shown in the inspection information 1500. In the example of FIG. 15, the inspection target of the inspection target information 1512 includes information on position 1, position 2, and reinforcing bar number. Position 1 and position 2 may be, for example, information indicating an area where the inspection target is placed in a construction blueprint. Note that position 2 may be information specifying a more detailed area within position 1, for example. The reinforcing bar number is a number assigned to a reinforcing bar or a group of reinforcing bars used within the area specified by the position information.

Further, the inspection item is, for example, information indicating an inspection item to be performed on the inspection target of the inspection target information 1512. In the example of FIG. 15, the test items of the test target information 1512 include test item 1 and test item 2. Furthermore, as shown in inspection item 1, the inspection items may include, for example, a design value and a measured value. The design value is the value on the design drawing for the inspection item. As the measurement value, for example, a measurement result measured in a reinforcement inspection is registered. The inspection items may include, for example, the diameter, position, average spacing, cover thickness, and number of reinforcing bars to be inspected.

In the fifth embodiment, the control unit 301 may output the type information determined in the process of S1005 described above as the measurement value of the examination information 1500. For example, the control unit 301 acquires information on the diameter of the reinforcing bar from the reinforcing bar information 900 based on the type of the reinforcing bar image to be determined, and adds the obtained diameter to the measurement value of the inspection item as shown in a frame 1501 in FIG. You may register this information. Further, the control unit 301 may register, in the inspection information 1500, identification information that identifies the reinforcing bar image to be determined that is used to specify the measurement value, as shown by a frame 1502 in FIG. Note that in another embodiment, the identification information may be information that identifies feature information such as a frequency spectrum extracted from the reinforcing bar image to be determined. For example, the identification information may be a path or a file name used to access the reinforcing bar image to be determined or the feature information extracted from the reinforcing bar image to be determined.

When inspecting reinforcement, it may be necessary to check whether the inspection results have been tampered with by someone. Therefore, for example, as described above, the control unit 301 performs identification for identifying the reinforcing bar image to be determined used in the inspection or the feature information extracted from the reinforcing bar image in association with the inspection result registered in the inspection item. The information is recorded in test information 1500. In this way, by recording the identification information for identifying the reinforcing bar images to be judged used in the inspection or the feature information extracted from the reinforcing bar images in association with the inspection results, the control unit 301 can later perform the steps shown in FIG. It becomes possible to re-execute the operation flow and recalculate the measured value. As a result, it is possible to verify whether or not the test results have been tampered with.

(Sixth embodiment)
In the above-described embodiment, for example, the frequency spectrum obtained by Fourier transformation from a reinforcing bar image to be determined may include low-frequency noise caused by paint applied as a marker on the surface of the reinforcing bar.

FIG. 16 is a diagram illustrating noise in the frequency spectrum caused by paint on the surface of a reinforcing bar image. For example, FIG. 16A shows a state in which a reinforcing bar is coated with paint 1601, and noise 1602 caused by the paint 1601 appears in the frequency spectrum.

Even if the similarity between the frequency spectrum of FIG. 16(a) including the noise 1602 and the frequency spectrum of the sample reinforcing bar image shown in FIG. 16(b) is evaluated in this way, the similarity due to the noise 1602 is may be evaluated low, resulting in a decrease in the accuracy of type determination.

Therefore, in the sixth embodiment, the control unit 301 may extract frequency components in a frequency range to be evaluated from the frequency spectrum, and evaluate the degree of similarity in the extracted range. For example, as shown in FIG. 16, by evaluating the degree of similarity in the frequency range to be evaluated, the influence of noise 1602 can be eliminated, and the degree of similarity can be evaluated low due to noise 1602. can be suppressed.

Note that the frequency range to be evaluated can be determined based on the position of the peak of the frequency spectrum obtained from the sample reinforcing bar image. For example, the control unit 301 may set a predetermined frequency range based on the position of the maximum peak frequency excluding the DC component in the sample reinforcing bar image used for similarity evaluation as the frequency range to be evaluated.

(Seventh embodiment)
In addition, as mentioned above, for example, when evaluating the similarity between a reinforcing bar image whose type is to be determined and a sample reinforcing bar image, it is better to evaluate the similarity when the photographing conditions of the reinforcing bars in the two images are as similar as possible. Improves accuracy. Therefore, in the seventh embodiment, a part of the reinforcing bar image in which the reinforcing bar whose type is to be determined is captured is extracted, and the extracted area is used as a sample reinforcing bar image to perform the reinforcing bar type determination.

FIG. 17 is a diagram illustrating reinforcing bar type determination according to the seventh embodiment. As shown in FIG. 17, the control unit 301 first receives the designation of a part of the image region from the image 403, and extracts the designated region as a sample reinforcing bar image ((1) in FIG. 17).

Subsequently, the control unit 301 stores the sample reinforcing bar image of the extracted region in the storage unit 302 in association with the type information ((2) in FIG. 17). In one example, the control unit 301 may receive an input of type information indicating the type of the sample reinforcing bar image from the user, and may associate the received type information with the sample reinforcing bar image. In the example of FIG. 17, D25 indicating the diameter of the reinforcing bar is associated with the sample reinforcing bar image as type information.

Subsequently, the control unit 301 evaluates the similarity with the sample reinforcing bar image, using an image area in which other reinforcing bars appear in the image 403 as a reinforcing bar image to be determined. Then, based on the similarity evaluation result, when the sample reinforcing bar image and the reinforcing bar image to be determined are similar to each other by satisfying a predetermined condition, the control unit 301 determines the type of the reinforcing bar image to be determined as the type of the reinforcing bar image to be determined. Type information of the sample reinforcing bar image may be output ((3) in FIG. 17). Note that the predetermined condition used here to determine that the types of reinforcing bars are the same is, for example, when the index used to evaluate the degree of similarity indicates that the higher the value, the more similar the degree of similarity is. It may be that the index used for evaluation is greater than or equal to a threshold value. Furthermore, for example, if the index used to evaluate the degree of similarity indicates that the lower the value, the more similar the index is, the predetermined condition is that the index used to evaluate the degree of similarity is less than or equal to the threshold value. It's fine. The threshold value used for determination is set based on empirical rules, etc., so that it can be determined, for example, whether the reinforcing bars shown in the sample reinforcing bar image and the reinforcing bars shown in the judgment target reinforcing bar image are of the same type. good.

As described above, it is possible to determine the type of reinforcing bars by using a reinforcing bar image in a part of the image 403 that shows the reinforcing bars to be determined as a sample reinforcing bar image. In the seventh embodiment, the type of reinforcing bars can be determined under similar shooting conditions between the reinforcing bar image whose type is to be determined and the sample reinforcing bar image, so it is possible to improve the accuracy of similarity evaluation. can. As an example, it is possible to suppress the influence of disturbances such as lighting conditions on similarity evaluation.

Further, according to the seventh embodiment, even if a sample reinforcing bar image is not prepared in advance, a sample reinforcing bar image is generated on the spot from the image 403 of the reinforcing bar to be determined that is taken during type determination. It is possible to generate and determine the type.

Furthermore, a plurality of images 403 may be taken with different angles of view and used for reinforcing reinforcement inspection, etc. In this case, a sample reinforcing bar image may be extracted from a part of one image 403, and the extracted sample reinforcing bar image may be used to determine the type of reinforcing bar appearing in another image 403. In this case as well, the degree of similarity can be evaluated under similar photographing conditions such as illumination conditions, so the accuracy of evaluation of the degree of similarity can be improved. By improving the accuracy of similarity evaluation, it is possible to improve the accuracy of determining the type of reinforcing bar.

Note that although the example in FIG. 17 shows an example in which the user specifies a region in which reinforcing bars are shown in the image 403, the embodiment is not limited to this. For example, in another embodiment, the control unit 301 may detect a plurality of reinforcing bars appearing in the image 403 by executing an algorithm for detecting reinforcing bars from the image 403. Then, the control unit 301 accepts the user's selection of reinforcing bars from among the plurality of detected reinforcing bars, associates the reinforcing bars selected by the user with the type information input by the user, and uses them as a sample reinforcing bar image. You can.

In another embodiment, a plurality of locations may be selected from the captured image for one type of reinforcing bar and used as sample reinforcing bar images. For example, the control unit 301 may extract a plurality of areas where reinforcing bars are visible from the photographed image 403 for one type of reinforcing bars, and register the extracted regions as a plurality of sample reinforcing bar images. As mentioned above, with deformed reinforcing bars, even the same type of reinforcing bars may have different shapes depending on their orientation, and such reinforcing bars may be installed with ribs and knots in different orientations during construction. . Therefore, by extracting multiple areas where reinforcing bars are visible from the photographed image 403 and obtaining sample reinforcing bar images showing the states of multiple reinforcing bars with different rib and knot orientations, it is possible to It is possible to suppress the influence of disturbances such as lighting conditions on similarity evaluation. Note that the plurality of sample reinforcing bar images may be extracted from, for example, a plurality of images 403 that are photographed with shifted angles of view with respect to reinforcing bars installed in a certain area.

Although the embodiments have been illustrated above, the embodiments are not limited thereto. For example, the above-described operational flow is an example, and the embodiments are not limited thereto. If possible, the operational flow may be executed by changing the order of processing, may include additional processing, or may omit some processing.

Further, in the above embodiment, an example was described in which the reinforcing bar image 404 to be determined is extracted from the image 403 and processed, but the embodiment is not limited to this. For example, in another embodiment, the image 404 is extracted from the image 403 and processed. An evaluation of similarity with the sample reinforcing bar image may be performed. As an example, in the rotation-invariant phase-only correlation method, similarity can be evaluated by handling rotation around the normal direction of the image plane. Therefore, for example, after performing the above-described facing process on the image 403, the control unit 301 calculates the degree of similarity between the image 403 and the sample reinforcing bar image using a rotationally invariant phase without generating the reinforcing bar image 404 to be determined. Evaluation may be performed using the limited correlation method. In this case, the control unit 301 can not only evaluate the degree of similarity but also specify the position of a reinforcing bar on the image 403 that is evaluated to have a high degree of similarity to the sample reinforcing bar image.

Further, in the above-described embodiment, an example is described in which diameter information is determined as an example of the type, but the embodiment is not limited to this. For example, by registering various information regarding the type of reinforcing bars (for example, manufacturer, knot spacing, knot type, etc.) in the reinforcing bar information 401, that information can be used to match input reinforcing bar images to sample reinforcing bar images. It can be obtained by evaluating the degree of similarity.

Further, in the above embodiment, an example is described in which the reinforcing bar determination device 201 stores reinforcing bar information such as reinforcing bar information 900, 1200, 1300, 1401, and 1402, but the embodiment is not limited to this. It's not something you can do. For example, in another embodiment, reinforcing bar information may be stored in a database server separate from reinforcing bar determining device 201. In this case, the reinforcing bar determination device 201 may access the database server and acquire information from the reinforcing bar information.

Note that in the embodiment described above, for example, at the start of the operation flow in FIG. 10, the control unit 301 operates as the input unit 311. Further, for example, in S1002 of FIG. 10, the control unit 301 operates as the evaluation unit 312. For example, in S1003 to S1005 in FIG. 10, the control unit 301 operates as the output unit 313. For example, in the sorting process shown in FIG. 14, the control unit 301 operates as the determination unit 314.

FIG. 18 is a diagram illustrating the hardware configuration of a computer 1800 for realizing the reinforcing bar determination device 201 according to the embodiment. The hardware configuration in FIG. 18 includes, for example, a processor 1801, a memory 1802, a storage device 1803, a communication interface 1804, an external interface 1805, a display device 1806, and an input device 1807. Processor 1801 may be communicatively connected to memory 1802, storage device 1803, communication interface 1804, external interface 1805, display device 1806, and input device 1807, for example, via a bus or the like.

Processor 1801 may be, for example, a single processor or a multiprocessor and multicore. The processor 1801 provides some or all of the functions of the control unit 301 described above by using the memory 1802 to execute, for example, a program that describes the procedure of the operation flow described above. For example, the processor 1801 operates as the input unit 311, the evaluation unit 312, the output unit 313, and the determination unit 314 by reading a program stored in the storage device 1803 into the memory 1802 and executing it.

Memory 1802 is, for example, a semiconductor memory and may include a RAM area and a ROM area. The storage device 1803 is, for example, a hard disk, a semiconductor memory such as a flash memory, or an external storage device. Note that RAM is an abbreviation for Random Access Memory. Further, ROM is an abbreviation for Read Only Memory. The storage unit 302 described above may include, for example, a memory 1802 and a storage device 1803.

The communication interface 1804 is, for example, a communication device that connects to a network and transmits and receives data according to instructions from the processor 1801. External interface 1805 may be, for example, an interface with an external device. In one embodiment, the reinforcing bar determination device 201 may be connected to the imaging device 102 via an external interface 1805. Note that in another embodiment, the reinforcing bar determination device 201 may include a communication interface 1804 such as a Wi-Fi (registered trademark) communication device and a Bluetooth (registered trademark) communication device, and may be connected via a network or over a short distance. It may be connected to the photographing device 102 by wireless communication. Communication interface 1804 and external interface 1805 are examples of the communication unit 304 described above.

The display device 1806 is, for example, a device with a display function such as a liquid crystal display. The display device 1806 is an example of the display section 303 described above. The input device 1807 is, for example, a device such as a keyboard and a touch panel that receives input from a user.

Each program according to the above-described embodiments may be provided to the reinforcing bar determination device 201 in the following format, for example.
(1) Installed in the storage unit 302 in advance.
(2) Provided by a server such as a program server.

Further, the hardware configuration for realizing the reinforcing bar determination device 201 described with reference to FIG. 18 is an example, and the embodiment is not limited to this. For example, some or all of the functions of the control unit 301 described above may be implemented as hardware using dedicated circuits such as FPGA and SoC. Note that FPGA is an abbreviation for Field Programmable Gate Array. SoC is an abbreviation for System-on-a-chip. As an example, the above-mentioned control unit 301 may be a circuit that outputs type information with respect to an input image according to the similarity evaluation according to the above-described embodiment. Moreover, the above-described input section 311, evaluation section 312, output section 313, and determination section 314 may be individually implemented as circuits such as an input circuit, an evaluation circuit, an output circuit, and a determination circuit. Further, all or part of these may be implemented as an integrated circuit.

Above, several embodiments are described. However, the embodiments are not limited to the embodiments described above, but should be understood to include various modifications and alternative forms of the embodiments described above. For example, it will be understood that the various embodiments can be embodied by changing the components without departing from the spirit and scope thereof. Furthermore, it will be understood that various embodiments can be implemented by appropriately combining the plurality of components disclosed in the embodiments described above. Furthermore, various embodiments can be implemented by deleting or replacing some components from all the components shown in the embodiments, or adding some components to the components shown in the embodiments. Those skilled in the art will understand that this can be done.

101: Reinforcing bars 102: Imaging device 200: Reinforcing bar determination system 201: Reinforcing bar determining device 205: Network 301: Control unit 302: Storage unit 303: Display unit 304: Communication unit 311: Input unit 312: Evaluation unit 313: Output unit 314: Judgment unit 1800: Computer 1801: Processor 1802: Memory 1803: Storage device 1804: Communication interface 1805: External interface 1806: Display device 1807: Input device

Claims (8)

a storage unit that stores reinforcing bar information including information that associates at least one of a sample reinforcing bar image showing reinforcing bars and characteristic information extracted from the sample reinforcing bar image with type information indicating the type of reinforcing bars appearing in the sample reinforcing bar image; and,
an input unit that receives an input of a reinforcing bar image showing the reinforcing bar to be determined;
an evaluation unit that evaluates the similarity between the sample reinforcing bar image and the reinforcing bar image based on the reinforcing bar information;
an output unit that outputs the type information corresponding to the sample reinforcing bar image when the reinforcing bar image and the sample reinforcing bar image satisfy a predetermined condition and are similar based on the similarity evaluation result;
Equipped with
The type information includes types classified based on the diameter of the reinforcing bars,
The evaluation unit calculates the degree of waveform similarity in a predetermined frequency range of a frequency spectrum obtained by Fourier transforming pixel values of the reinforcing bars in the longitudinal direction of the reinforcing bars of the sample reinforcing bar image and the reinforcing bar image, and the plurality of waves included in the frequency spectrum. A reinforcing bar determination device, characterized in that the degree of similarity between the sample reinforcing bar image and the reinforcing bar image is evaluated based on the similarity of peak frequencies . The evaluation unit calculates the degree of waveform similarity in a predetermined frequency range of a frequency spectrum obtained by Fourier transforming the pixel values of the reinforcing bars in the longitudinal direction of the reinforcing bars in the sample reinforcing bar image and the reinforcing bar image, and the frequency range of the frequency spectrum. The reinforcing bar determination device according to claim 1, wherein the similarity between the sample reinforcing bar image and the reinforcing bar image is evaluated based on the similarity of frequencies of a plurality of peaks included in the reinforcing bar image. The feature information extracted from the sample reinforcing bar image includes a frequency spectrum obtained by Fourier transforming the sample reinforcing bar image, a peak frequency of the frequency spectrum, phase information extracted from the result of the Fourier transformation, and a local area extracted from the sample reinforcing bar image. The reinforcing bar determination device according to claim 1, comprising at least one of a feature amount and a distance between edges of reinforcing bars that are substantially parallel to the longitudinal direction of the reinforcing bars in the sample reinforcing bar image. The evaluation unit further determines the relative position and orientation of the photographing device that photographed the reinforcing bar image and the reinforcing bars that appear in the reinforcing bar image, based on information regarding the relative position and orientation of the photographing device that photographed the reinforcing bar image and the reinforcing bars that appear in the reinforcing bar image. The reinforcing bar determination device according to claim 1, wherein the similarity evaluation is performed after the reinforcing bar images are normalized so that the positions and orientations of the reinforcing bars have a predetermined relationship. The evaluation unit further determines the relative position and orientation of the photographing device that photographed the sample reinforcing bar image and the reinforcing bars that appear in the sample reinforcing bar image, based on the information regarding the relative position and orientation of the photographing device and the reinforcing bars that appear in the sample reinforcing bar image. The reinforcing bar determination device according to claim 1, wherein the similarity evaluation is performed after the sample reinforcing bar images are normalized so that relative positions and orientations of the reinforcing bars have a predetermined relationship. The storage unit stores a plurality of pieces of reinforcing bar information for each of a plurality of different types,
The reinforcing bar determination device according to claim 1, wherein the predetermined condition is that the reinforcing bar image is evaluated to be most similar to the reinforcing bar image based on the similarity evaluation result. Accepts the input of a reinforcing bar image that shows the reinforcing bar to be judged,
Based on reinforcing bar information that includes information that associates at least one of a sample reinforcing bar image showing reinforcing bars and characteristic information extracted from the sample reinforcing bar image with type information indicating the type of reinforcing bars appearing in the sample reinforcing bar image, Evaluating the similarity between the sample reinforcing bar image and the reinforcing bar image,
Based on the similarity evaluation result, if the reinforcing bar image and the sample reinforcing bar image satisfy a predetermined condition and are similar, outputting the type information corresponding to the sample reinforcing bar image. ,
The type information includes types classified based on the diameter of the reinforcing bars ,
The sample reinforcing bar image and the reinforcing bar image are subjected to Fourier transform of the pixel values of the reinforcing bars in the longitudinal direction of the reinforcing bars, and the waveform similarity in a predetermined frequency range of the frequency spectrum and the frequency of a plurality of peaks included in the frequency spectrum are determined. A reinforcing bar determination method that evaluates the degree of similarity between the sample reinforcing bar image and the reinforcing bar image based on the degree of similarity . The similarity of waveforms in a predetermined frequency range of the frequency spectrum obtained by Fourier transforming the pixel values of the reinforcing bars in the longitudinal direction of the reinforcing bars of the sample reinforcing bar images and the reinforcing bar images, and the similarity of the waveforms of the frequency spectrum included in the frequency range. 8. The reinforcing bar determination method according to claim 7, wherein the similarity between the sample reinforcing bar image and the reinforcing bar image is evaluated based on the similarity of peak frequencies.

Images