JP2019120693A - Method for detecting damage to exterior materials - Google Patents

Abstract

To provide a damage detection method of exterior materials that can detect damages to accurately identify the damage site of exterior materials in a simpler work.SOLUTION: Frequency analysis of color values is performed, the cycle and phase of the change in color values along the intersecting direction is calculated, an occurrence pattern of a first linear portion 11 is estimated based on the period and phase, the first linear portion 11 is removed from the image A4 on the basis of an estimated occurrence pattern with respect to the image A4 which has been differentially processed, and the damage site 13 of the roof 10 is identified from the image A5 that removes the first linear portion 11 by imaging the exterior material, and differentially processing it with respect to the image A3 converted to grayscale for each pixel in accordance with the change in white color from black in the quantified color value for the color values of the pixel group arranged along the direction intersecting the first linear portion 11, by performing the Fourier transformation.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a damage detection method for detecting a damaged portion of an outer covering material, and more particularly to a damage detection method for an outer covering material having a plurality of linear portions oriented at equal intervals.

  2. Description of the Related Art Conventionally, the exterior material is imaged by an imaging device to detect damage to the exterior material. For example, Patent Document 1 proposes a method of detecting damage to a paved road paving the road as an exterior material. In this damage detection method, size-dependent crossover is introduced into a parallel image filter automatic generation system by genetic programming, and a place where extraction of a crack seems to be difficult from multiple actual pavement images is selected and adopted as training data for filter construction By doing this, an image filter for extracting cracks is automatically constructed from various types of images. This extraction image filter is applied to individual blocks obtained by dividing the entire image of the evaluation target pavement area into grids, to evaluate the evaluation target pavement area.

JP, 2011-179874, A

  However, for example, when it is intended to damage all the exterior materials by the method shown in Patent Document 1, training data for constructing an image filter for extraction must be acquired as various types of damage data. And in order to detect the damage of the exterior material correctly, it was necessary to acquire a huge amount of damage data, and it had to carry out an excessive work.

  The present invention has been made in view of such a point, and provides a method of detecting damage to an exterior material which can accurately identify and detect a damaged part of the exterior material with a simpler operation. It is to do.

  As a result of intensive investigations, the inventors believe that if the exterior material having a specific pattern is removed from the image obtained by imaging this pattern, the damaged portion of the exterior material can be easily detected. The

  The present invention is based on such an idea, and the method of detecting damage to the package according to the first invention detects a damaged portion of the package having a plurality of linear portions oriented at equal intervals. A damage detection method comprising: imaging the exterior material; converting an imaged image to grayscale; and emphasizing an edge in a direction along the linear portion with respect to the image converted to grayscale. The step of performing differential processing, and the linear portion of the numerical value represented by the numerical value corresponding to the change in color from black to white for each pixel with respect to the image subjected to the differential processing. Frequency analysis of the color values is performed by performing Fourier transform on the color values of the pixel group arranged along the crossing direction, and a period of change of the color values along the crossing direction Step of calculating phase A step of constructing a filter image according to the image subjected to the differential processing by performing an inverse Fourier transform based on the period and the phase for each of the pixel groups; and performing the differential processing on the filter image And C. identifying the damaged portion of the exterior material by superimposing on the image.

  According to the first aspect, first, the exterior material is imaged by the imaging device, the imaged image is converted into gray scale, and an edge in a direction along the linear portion of the image converted into gray scale is Perform differential processing as emphasized. As a result, with respect to the image of the exterior material subjected to differentiation processing after conversion to gray scale, the image of the linear portion, the image of the damaged portion extending along the linear portion, and the predetermined direction in this direction And an image of the injury site having a width of

  Next, with respect to the image subjected to the differential processing, the image of the highlighted line portion and the image of the damaged portion in the color value quantified according to the change of the color from black to white for each pixel The color values assigned to the pixels that make up T.sub.1 become significantly different values compared to the color values assigned to the pixels corresponding to the other parts. Note that binarization processing may be performed on the image subjected to the differentiation processing as necessary.

  Next, frequency analysis of the color values is performed by performing Fourier transform on the color values of each pixel group arranged along the direction crossing the linear portion, and the color along the direction of crossing Calculate the period and phase of change of numerical value. The period and the phase of the change of the color value subjected to the frequency analysis by the Fourier transform correspond to the information of the arrangement period of the plurality of linear portions oriented at equal intervals and the information of the position of the linear portion.

  Next, the inverse Fourier transform is performed based on the period and the phase of the change of the color numerical value for each group of pixels to construct a filter image according to the image subjected to the differential processing. In addition, you may perform a binarization process with respect to this filter image as needed.

  The obtained filter image is an image including information on the arrangement period of the linear portions constituting the group of linear portions and information on the position of the group of each linear portion. Then, the image subjected to differential processing is superimposed, and the linear portion is removed from the image subjected to differential processing. In the image from which the linear portion has been removed, since the damaged portion of the exterior material remains, the damaged portion can be identified from this image.

  As a more preferable aspect, the linear portions are each formed between the plurality of first linear portions oriented at equal intervals and the adjacent first linear portions. And a group of a plurality of second linear portions arranged along the first linear portion and having a plurality of second linear portions at equal intervals so as to connect them; Adjacent groups of second linear portions are arranged in a staggered manner offset in a direction along the first linear portions, and in the step of performing the differential processing, the gray scale is used. The converted image is differentiated so that an edge in a direction along one of the first and second linear portions is emphasized and the other linear portion disappears from the image. In the step of calculating the period and phase of the change of the color value, the second linear portion Calculating the period and the phase change of said color numerical values.

  According to this aspect, in the differential processing, an edge in a direction along any one of the first and second linear portions is enhanced with respect to the image converted into gray scale, Differential processing is performed so that the linear portion disappears from the image. For example, when one linear portion is a second linear portion and the other linear portion is a first linear portion, an edge in a direction along the second linear portion is emphasized, and the first linear portion is emphasized. The differentiation process is performed so that the linear portion of the image disappears from the image. As a result, with respect to the image of the exterior material subjected to differentiation processing after conversion to gray scale (first differentially processed image), along the portion of the image of the second linear portion and the second linear portion It is possible to highlight the part of the image of the damaged site which has been extended and the part of the image of the damaged site which has a predetermined width in this direction.

  Next, frequency analysis of the color values is performed by performing Fourier transform on the color values of the pixel groups arranged along the first linear portion of the captured image, and the first line is generated. Calculate the period and phase of the change of the color value along the shape part. The period and phase of the change in color value subjected to frequency analysis by Fourier transform are information of the arrangement period of the plurality of second linear portions constituting the group of the plurality of linear portions oriented at equal intervals, and each It corresponds to the information of the position of the group of the second linear portion. In addition, since the group of 2nd linear parts is offset, these exact positional information can not be known from the phase mentioned above.

  Therefore, since the appearance pattern of the second linear portion can not be easily estimated on the basis of the calculated period and phase, in the next step, it is preferable to By performing inverse Fourier transform on the information of the part, a filter image (first filter image) corresponding to the image subjected to the differentiation process is constructed. In addition, you may perform a binarization process with respect to this filter image as needed.

  The obtained filter image is an image including information on the arrangement period of the second linear portions constituting the group of the second linear portions and information on the position of the groups of the respective second linear portions. In the next step, an image (first differentially-processed image) obtained by differentially processing the filter image is superimposed, and an image (first differentially-processed image) obtained by differentially processing the second linear portion Remove from In the image from which the second linear portion has been removed, since the damaged portion of the exterior material remains, the damaged portion can be identified from this image.

  Similarly, one linear portion is a first linear portion, and the other linear portion is a second linear portion, and the above-described differentiation process is performed, and an edge in a direction along the first linear portion. , And the second linear portion may be eliminated from the image. A period and a phase of change in color value are calculated by Fourier transform from the image subjected to differential processing (second differentially processed image), and then inverse Fourier transform is performed to obtain a filtered image according to the image subjected to the differential processing. A (second filter image) may be constructed, and the damaged portion of the exterior material may be identified from this filter image.

  Furthermore, a synthetic differentially processed image obtained by synthesizing the first differentially processed image and the second differentially processed image, and a synthetic filter image obtained by synthesizing the first filter image and the second filter image described above are created, and the synthetic filtered image is generated. The synthetic differentially processed image may be superimposed to identify the damaged portion of the exterior material.

  Furthermore, the filter image described above may be corrected. As a specific preferable aspect, a straight line along the one linear portion is detected by Hough transformation with respect to the captured image of the exterior material, and an image composed of the detected straight line is generated; And correcting the constructed filter image based on the image of the straight line.

  The filter image calculated by the frequency analysis can accurately represent the number of lines along one of the linear portions and the spacing of the lines, and in straight line detection by the Hough transform, the straight lines along one of the linear portions The position can be detected accurately. Therefore, if the filter image is corrected so that one linear portion of the differentially processed image disappears from the image of the detected straight line, the damaged portion of the exterior material can be identified more accurately.

  As a more preferable aspect, before the step of performing the differentiation process, an image is extracted from the captured image of the exterior material taken, and in the step of performing the differentiation process, the extracted image and the captured image are extracted with respect to the extracted image The extended image obtained by extending the extracted image is created by performing the differential process and copying the extracted image subjected to the differential process in a region adjacent to the extracted image subjected to the differential process, and the created extended image created And the step of calculating the period and phase of change of the color value and the step of constructing the filter image to construct an extended filter image corresponding to the expanded image as the filter image; In the step of identifying the damaged part of the material, the area from which the extracted image is extracted by superimposing the expanded filter image on the captured image subjected to the differentiation process, and Specifying a damaged site of the outer member adjacent areas Les.

  According to this aspect, it is possible to more easily identify the damaged area of the area from which the extracted image is extracted and the area adjacent thereto by the expansion filter image without constructing a new filter image. Here, in the case where the extracted image is rectangular and the image adjacent thereto includes linear portions at the same intervals as the image extracted in a shape different from the rectangular shape, the filter image including the adjacent image Construction may reduce the accuracy of identifying the site of damage to the sheathing material. However, according to this aspect, since the expansion filter image is constructed based on the image with high accuracy of identification of the damaged site, the area of the area different from the rectangular shape in which the linear portion is formed can be obtained. It is possible to more accurately identify the damaged site of the exterior material.

  As a more preferable aspect, after performing the differentiation process and before creating the expanded image, an extracted image subjected to the differentiation process and a captured image subjected to the differentiation process are included in the exterior material. The expansion filter image is constructed from the extracted image corrected by imaging from the normal direction of the surface to be an image subjected to the differentiation process, and the expansion filter image is superimposed on the corrected captured image By combining, the damage site | part of the said exterior material of the area | region which extracted the said extraction image, and the area | region adjacent to this is specified.

  According to this aspect, as the image is taken from the normal direction of the surface of the exterior material (the front of the surface of the exterior material), the extracted image subjected to the differentiation process and the imaged image subjected to the differentiation process are corrected in advance . Since the damaged part of the exterior material is identified using the image after this correction, it is possible to avoid the erroneous detection of the damaged part due to the distortion of the image due to the distance from the imaging condition.

  In these embodiments, the expansion filter image is created by copying the extraction image before constructing the filter image. For example, after the filter image is constructed, this filter image is copied and expanded A filtered image may be constructed.

  As such a preferable aspect, before the step of performing the differentiation process, an image is extracted from the captured image of the exterior material taken, and the filter image is extracted from the step of performing the differentiation process on the extracted extracted image. The step of constructing up to the step of constructing, the step of constructing the filter image, and the step of identifying the damaged part of the exterior material, the extracted image subjected to the differentiation processing and the differentiation processing other than that are performed The damaged portion of the exterior material may be specified by applying it to the area of the taken image (for example, the area adjacent to the extracted image).

  In this case, the extracted image subjected to the differentiation process and the captured image subjected to the differentiation process are imaged from the normal direction of the surface of the outer covering material to obtain the image subjected to the differentiation process, The filter image is constructed with respect to the corrected and corrected extracted image, and the constructed filter image is applied to the region of the captured image which has been subjected to the correction process and the other differential processing. It is preferable to identify the damaged part of the exterior material.

  A method of detecting damage to an exterior material according to a second aspect of the present invention is a damage detection method for detecting a damaged portion of the exterior material having a plurality of linear portions oriented at equal intervals, and using the imaging material as the exterior material. Performing imaging, converting the imaged image into grayscale, and performing differential processing on the image converted into grayscale so that an edge in a direction along the linear portion is emphasized. Pixels arranged in a direction intersecting with the linear portion in a numerical color value digitized according to a change in color from black to white for the image subjected to the differential processing Performing a frequency analysis of the color values by performing Fourier transform on the color values of the group, and calculating a period and a phase of change of the color values along the intersecting direction; the period and the step Based on the phase A step of removing the linear portion from the image, wherein an appearance pattern of the linear portion is estimated, and the image subjected to the differential processing based on the appearance pattern; an image obtained by removing the linear portion And a step of identifying a damaged portion of the outer covering material.

  According to the second aspect, first, the exterior material is imaged by the imaging device, the imaged image is converted to grayscale, and an edge in a direction along the linear portion is converted to the grayscale converted image. Perform differential processing as emphasized. As a result, with respect to the image of the exterior material subjected to differentiation processing after conversion to gray scale, the image of the linear portion, the image of the damaged portion extending along the linear portion, and the predetermined direction in this direction And an image of the injury site having a width of

  Next, with respect to the image subjected to the differential processing, the image of the highlighted line portion and the image of the damaged portion in the color value quantified according to the change of the color from black to white for each pixel The color values assigned to the pixels that make up T.sub.1 become significantly different values compared to the color values assigned to the pixels corresponding to the other parts. Note that binarization processing may be performed on the image subjected to the differentiation processing as necessary.

  Next, frequency analysis of the color values is performed by performing Fourier transform on the color values of the pixel group arranged along the direction crossing the linear portion, and the color values along the direction of crossing Calculate the period and phase of change of The period and the phase of the change of the color value subjected to the frequency analysis by the Fourier transform correspond to the information of the arrangement period of the plurality of linear portions oriented at equal intervals and the information of the position of the linear portion.

  Therefore, in the next step, the appearance pattern of the linear portion is estimated based on the calculated period and phase, and the linear portion is removed from the image on which the differential processing has been performed based on the estimated pattern. can do. Specifically, the image of the linear portion is given the same color as the image around it, and the image of the linear portion is made inconspicuous to the image subjected to the differential processing.

  Thus, in the next step, it is possible to leave an image corresponding to the damaged area of the covering material from the image from which the linear portion has been removed, and to identify the damaged area of the covering material simply and accurately from this image. it can. As a result of this, it is possible to accurately identify and detect the damaged site of the exterior material by a simpler operation.

  As a more preferable aspect, in the step of identifying the damaged site, the damaged site is identified by superimposing the image obtained in the removing step on the image captured in the imaging step. Specifically, in the first invention, the damaged site is identified by superimposing the image from which the linear portion is removed on the captured image. In the second invention, the damaged site is identified by superimposing the image from which the second linear portion has been removed on the captured image. This makes it possible to more accurately specify the position of the damaged site with respect to the exterior material that is the captured image.

  Although the exterior material mentioned above can mention the exterior part of a building comprised with a tile or block etc. which were periodically arranged, a paved part such as a road or a square, etc., for example, the exterior material is more preferable as a preferred embodiment. Is a roof of a building on which slate tiles are laid, and the first linear portion and the second linear portion are boundary portions between the slate tiles disposed on the roof.

  According to this aspect, since the roof of the building on which the slate tile is laid has the first linear portion and the second linear portion periodically because the slate tile is laid regularly. More accurately identify the damaged site of the roof, it can be detected.

  As a further preferable aspect, the imaging of the exterior material is performed from an unmanned aerial vehicle equipped with an imaging device. According to this aspect, the exterior material can be easily imaged at a desired angle from above. In particular, if the exterior material is a roof, the roofs of the buildings can be easily imaged from desired locations above these from unmanned aerial vehicles without making a footing for the buildings for imaging .

  According to the present invention, it is possible to accurately identify and detect a damaged site of the exterior material by a simpler operation.

It is the apparatus schematic for enforcing the damage detection method of the exterior material which concerns on embodiment of 1st and 2nd invention. It is a part of flowchart of the damage detection method of the exterior material which concerns on 1st Embodiment. It is a schematic diagram of the image imaged at the imaging process shown in FIG. (A) is a schematic diagram of the image which extracted a part of image of FIG. 3, (b) is a schematic diagram of the image after a pre-processing process. (A) is a schematic diagram of the image after gray scale conversion in the differentiation process shown in FIG. 2, (b) is a schematic diagram of each pixel for demonstrating a differentiation process. (A) is a schematic diagram of the image after the differentiation process shown in FIG. 2, (b) is a figure which shows the color numerical value allocated to the N-th pixel row in the image after the differentiation process. is there. (A) is a graph showing the relationship between the frequency and the amplitude obtained in the frequency analysis step shown in FIG. 2, and (b) shows the relationship between the frequency and the phase obtained in the frequency analysis step shown in FIG. It is a graph shown. (A) is a schematic diagram of the image after the 1st linear partial removal process shown in FIG. 2, (b) is a schematic diagram of the image after the damage site identification process shown in FIG. It is a part of flowchart of the damage detection method of the exterior material which concerns on 2nd Embodiment. (A) is a schematic diagram of the image which extracted a part of the image of FIG. 3, (b) is a schematic diagram of the image after a pre-processing process, (c) is an image after gray scale conversion (D) is a schematic diagram of the image after the differential treatment process shown in FIG. (A) is a graph showing the relationship between frequency and amplitude obtained in the frequency analysis step shown in FIG. 9, (b) shows the relationship between frequency and phase obtained in the frequency analysis step shown in FIG. It is a graph shown. (A) is an image after the pre-processing step in a part of the image of the actual roof, (b) is an image after the differential processing step with respect to the image shown in (a), (c) Is a filter image created from the image shown in (b). (A) is an image after the 2nd linear partial removal process shown in FIG. 9, (b) is a schematic diagram of the image after the damage site identification process shown in FIG. It is a flowchart of the damage detection method of the exterior material which concerns on 3rd Embodiment. (A)-(d) is an example of the image in each process shown in FIG. It is a flowchart of the damage detection method of the exterior material which concerns on 4th Embodiment. (A) is a schematic diagram of the image of the roof of the ridge which was imaged by the imaging process, (b) is an example of an extended filter image. (A) And (b) is a schematic diagram for demonstrating the process to the area | region shape correction process shown in FIG. It is a schematic diagram for demonstrating the area | region expansion process shown in FIG.

  Hereinafter, a method of detecting damage to the exterior material according to the embodiments of the first and second inventions will be described with reference to FIGS.

1. Damage Detection Device 100 and Roof 10 The damage detection method according to the present embodiment detects damage to the roof 10 of the building 1 as an exterior material. As shown in FIG. 1, in this diagnostic method, damage to the roof 10 is detected using an unmanned aerial vehicle (UAV) 5 equipped with the imaging device 6 shown in FIG. 1 and a processing device 7.

1-1. Unmanned Aerial Vehicle (UAV) 5 Equipped with the Imaging Device 6 The unmanned aerial vehicle (UAV) 5 is a so-called drone, and the unmanned aerial vehicle 5 is operated by the wireless controller (shown) to the upper sky of the building 1 or the like. The imaging device 6 is mounted on the unmanned aerial vehicle 5, and an object can be imaged by the operation of the wireless controller.

  The imaging device 6 mounted on the unmanned aerial vehicle 5 is a device that can capture color or monochrome digital images, and is a device that captures an image of the roof 10 of the building 1 in a process described later. Examples of the imaging device 6 include a digital camera (so-called digital camera), a communication terminal having a camera function, and the like.

1-2. The Processing Device 7 The processing device 7 includes an input device 71, a display device 72, an arithmetic device (CPU) 73, and a storage device (for example, a RAM or the like) 74. The processing device 7 is a portable terminal such as a personal computer or a smartphone or a tablet terminal. The storage device 74 stores an image processing program 74A, an analysis program 74B, and the like which are performed in steps to be described later. The storage device 74 stores an OS (Operation Program) (not shown), and the image processing program 74A is installed as an application program.

  The input device 71 is an input device such as a keyboard, a mouse, and a touch panel, and is connected to the arithmetic device 73 and the storage device 74 via an input / output port (not shown) such as an interface conforming to the USB standard. For example, processing conditions of each process are input by the input device 71. The image captured by the imaging device 6 is input to the storage device 74, for example, wirelessly via the input / output port. The display device 72 is an output device capable of displaying an image such as a liquid crystal display device, and outputs an image or the like of each process.

1-3. About the Roof 10 As shown in FIG. 3, the exterior material to be detected for damage is the roof 10, and in the present embodiment, the roof 10 is a roof made of, for example, rectangular slate tiles. The roof 10 is equally spaced so as to connect the respective first linear portions 11 between the plurality of first linear portions 11 oriented at equal intervals and the adjacent first linear portions 11. And a plurality of second linear portions 12, and a group of the plurality of second linear portions 12 arranged along the first linear portion 11, and a plurality of groups adjacent to the group. The groups of second linear portions 12 are arranged in a staggered manner offset in the direction along the first linear portions 11.

2. About the damage detection method concerning a 1st embodiment By performing the following series of processes shown in Drawing 2 to such a roof 10, a damaged part is detected. The damage detection method according to the first embodiment will be described with reference to FIGS. The first embodiment corresponds to an embodiment of the second invention.

2-1. Regarding Imaging Step S21 First, in the present embodiment, the imaging step S21 shown in FIG. 2 is performed. As shown in FIG. 1, the roof 10 of the building 1 is imaged by the imaging device 6. At this time, the unmanned aerial vehicle 5 is made to fly above the building 1 and the roof 10 is imaged by the imaging device 6 from above. Thereby, the image (captured image) 10A of the roof 10 as shown in FIG. 3 can be acquired. The captured image 10A is transmitted to the processing device 7, and the processing device 7 performs the following steps on the captured image.

2-2. Pretreatment Step S22 Next, a pretreatment step S22 is performed. In this process, the image A1 is extracted from the image of the roof 10 shown in FIG. 3 and correction such as distortion by the lens of the imaging device 6 is performed. In addition, with respect to the extracted image A1, the image is inclined in accordance with the slope of the roof 10 by projective transformation or the like. Thus, from the extracted image A1 shown in FIG. 4A, an image A2 after the pre-processing step S22 shown in FIG. 4B can be obtained. In the imaging step S21, projective transformation can be omitted if the direction of the imaging device 6 is directed perpendicularly to the roof surface during imaging with the unmanned aerial vehicle 5.

2-3. Regarding the differential processing step S23 Next, the differential processing step S23 is performed. In this step, the image A2 is converted to gray scale to obtain the image A3 shown in FIG. 5A, and then the direction along the first linear portion 11 with respect to the image A3 converted to gray scale Perform differentiation processing so that the edge of is emphasized. Thereby, the second linear portion 12 is extinguished.

  Specifically, as shown in FIG. 5B, a filter matrix (sobel filter) is applied to each pixel according to the change in color from black to white. . At this time, the converted color value is converted for each pixel in accordance with the change in color from black to white. Specifically, in the filter matrix, negative values are set to zero so that black is 0 and white is 255, and values exceeding 255 are all 255.

  Thereby, as shown to Fig.6 (a), the 2nd linear part 12 which is a vertical pattern of the roof 10 is erase | eliminated, The 1st linear part 11 which is a horizontal pattern of the roof 10, and others An image A4 in which information such as the damaged site 13 is emphasized can be obtained.

2-4. Frequency Analysis Step S24 Next, the frequency analysis step S24 is performed. Here, for the image A4 subjected to the differentiation process, color values are assigned for each pixel according to the change in color from black to white. For this image A4, frequency analysis of the color values is performed by performing Fourier transform on the color values of the pixel group arranged along the direction crossing the first linear portion 11, and the crossing is performed. Calculate the period and phase of the change of the color value along the direction.

  Specifically, as shown in FIG. 6 (a), Fourier transformation is performed on the change of the assigned color value as shown in FIG. 6 (b) for each row of pixels of the image A4. Perform frequency analysis of color values. The horizontal axis in FIG. 6B is the position of the pixels of the pixel group arranged along the direction (specifically, the vertical direction) intersecting the first linear portion 11, and one vertical row The data of B.sub.1 and B.sub.2 can be combined to obtain the period and the phase of the change of the color value according to the pixel. Such frequency analysis is performed for all the columns.

  Thus, as shown in FIGS. 7A and 7B, the relationship between the amplitude and the phase can be obtained with respect to the frequency. Here, the frequency refers to the number of cycles / the number of column pixels. As shown in FIG. 7A, since the frequency at which the first peak of the amplitude appears changes depending on the number of first linear portions 11, from FIG. 7A, the plurality of first linear portions 11 can be obtained. It is possible to obtain sequence cycle information. Specifically, regarding the result of averaging the results of Fourier transform of each column for each frequency, the frequency at which the first peak appears changes depending on the number of first linear portions 11. Therefore, the frequency at which the first peak appears corresponds to the information of the period of the first linear portion. On the other hand, as shown in FIG. 7B, the change of the phase changes according to the position of the first linear portion 11, so that the information of the position of the first linear portion 11 can be specified.

2-5. First Linear Portion Removal Step S25 Next, a first linear portion removal step S25 is performed. In this step, the appearance pattern of the first linear portion 11 is estimated based on the calculated period and phase, and the first line is generated for the image A4 subjected to differentiation processing based on the estimated appearance pattern. Like portion 11 is removed from the image A4.

  When estimating the appearance pattern, as shown in FIG. 7A, if the first peak is one peak of the waveform, the amplitude and phase of a plurality of peaks including this peak are used to The number of occurrences of the line corresponding to the linear portion 11 and the position thereof are estimated. The appearance pattern can be estimated from the number of occurrences of the estimated line and the position thereof. On the other hand, as shown in FIG. 11A, which will be described later, when the first peak spans two frequencies, the first linear portion is used for each row using the amplitude and phase of one of the frequencies having the larger amplitude. Estimate the number of occurrences of the line corresponding to 11 and its position.

  Specifically, the image of the first linear portion 11 in FIG. 6 (a) is given the same color (black) as the image around it, and the image subjected to the differential processing is Make the image of the linear portion 11 of 1 less noticeable. Thereby, as shown to Fig.8 (a), the image A5 of the damage site 13 can be obtained. For example, this image A5 may be binarized with an appropriate threshold.

2-6. About Damaged Part Identification Step S26 Next, the damaged part identification step S26 is performed. The damaged portion 13 of the roof 10 is identified from the image A5 from which the first linear portion 11 has been removed. Specifically, on the image A2 shown in FIG. 4A, in which the state of the roof 10 after the pretreatment step S2 is known, an image in which the white color and the black color of the image A5 are reversed is superimposed. Thus, as shown in FIG. 8 (b), an image A6 in which the damaged portion 13 of the roof 10 is accurately identified can be obtained by a simpler operation.

3. Regarding a Damage Detection Method According to the Second Embodiment Furthermore, as a damage detection method different from the above-described method, a damaged portion is detected by performing a series of steps shown in FIG. 9 on the roof 10. The damage detection method according to the second embodiment will be described below with reference to FIGS. The second embodiment corresponds to an embodiment of the first invention.

3-1. About imaging process S91 and pre-processing process S92 By the method similar to the method mentioned above, imaging process S91 and pre-processing process S92 are performed. In the imaging step S91, the image 11A of the roof 10 is imaged (see FIG. 3), and as shown in FIG. 10A, a part of the image B1 is extracted. Next, as shown in FIG. 10B, an image B2 without distortion or the like can be obtained by the pre-processing step S92.

3-2. Regarding the differential processing step S93 Next, the differential processing step S93 is performed. In this step, the image B2 is converted to gray scale to obtain an image B3 shown in FIG. 10C, and then the direction along the second linear portion 12 with respect to the image B3 converted to gray scale Perform differentiation processing so that the edge of is emphasized. Thereby, the first linear portion 11 is erased from the image B3.

  Specifically, as shown in FIG. 10C, a filter matrix (sobel filter) is applied to each pixel according to the change in color from black to white. . At this time, the converted color value is converted for each pixel in accordance with the change in color from black to white. Specifically, in the filter matrix, negative values are set to zero so that black is 0 and white is 255, and values exceeding 255 are all 255.

  As a result, as shown in FIG. 10C, the first linear portion 11 which is the horizontal pattern of the roof 10 is erased, and the second linear portion 12 which is the vertical pattern of the roof 10 and the other An image B4 in which information such as the damaged site 13 is emphasized can be obtained.

3-3. Frequency Analysis Step S94 Next, the frequency analysis step S94 is performed. Here, a color value is assigned to the image B4 subjected to the differentiation processing, in accordance with the change in color from black to white, for each pixel. The image B4 is subjected to Fourier transform on the color values of the pixel group arranged along the direction (specifically, the lateral direction) intersecting the second linear portion 12 to obtain a color. Perform frequency analysis of numerical values and calculate the period and phase of change of color values along the crossing direction.

  Specifically, frequency analysis of color values is performed by performing Fourier transform on the change of the assigned color values for each column of pixels of the image B4. One row of data can be combined to obtain the period and phase of change of the color value of the pixel. Such frequency analysis is performed for all the rows.

  Thus, as shown in FIGS. 11A and 11B, the relationship between the amplitude and the phase can be obtained with respect to the frequency. However, since the groups of the second linear portions 12 have offsets, the number of groups of the adjacent second linear portions 12 in the longitudinal direction may differ, and from the obtained phases, these accurate I do not know the location information.

  Therefore, since the appearance pattern of the second linear portion can not be easily estimated based on the calculated period and phase, a filter image constructing step S95 described below is performed.

3-4. Filter Image Construction Step S95 In this step, the inverse Fourier transform is performed based on the period and the phase to construct a filter image according to the image B4 subjected to the differential processing. FIG. 12 (a) is an image after the pre-processing step in a partial image of the actual roof 10. As shown in FIG. On the other hand, FIG. 12 (b) is an image after the differentiation process step S93 of FIG. 12 (a). In this step, a filter image shown in FIG. 12 (c) is further created.

  Specifically, inverse Fourier transform is performed using the amplitude and phase information of the frequency of the first peak (the frequencies of 0.05 and 0.06 in FIG. 11A). The image obtained by the inverse Fourier transform is binarized and subjected to expansion / contraction processing. Thereby, the filter image according to FIG. 12C can be obtained.

3-5. Regarding Second Linear Portion Removal Step S96 Next, a second linear portion removal step S96 is performed. In this step, the second linear portion 12 is removed from the image B4 of differential processing by superimposing the image B4 subjected to the differential processing shown in FIG. 10D on the filter image. Specifically, when the color value of the filter image is equal to or greater than a predetermined value, the pixel of the corresponding position of the image B4 is assigned a color (black) equivalent to the pixels around it, for the pixels of the two images. . Thereby, the image of the second linear portion 12 is made less noticeable to the image B4 subjected to the differentiation process. In this way, as shown in FIG. 13A, an image B5 of the damaged site 13 can be obtained. For example, this image B5 may be binarized with an appropriate threshold.

3-6. About Damaged Part Identification Step S97 Next, the damaged part identification step S97 is performed. The damaged portion of the roof 10 is identified from the image B5 from which the second linear portion 12 has been removed. Specifically, on the image B2 shown in FIG. 10 (b), where the state of the roof 10 after the pre-processing step S2 can be determined, the image in which the white and the black are reversed in the image B5 of FIG. 13 (a) is superimposed. . In this way, as shown in FIG. 13B, it is possible to obtain an image B6 in which the damaged portion 13 of the roof 10 is accurately identified by a simpler operation.

  In this embodiment, after obtaining an image B4 (first differentially processed image) in which the first linear portion 11 is erased from the image B3 by differential processing, the first filter image (first filter image) is constructed. Two linear portions 12 were removed from image B4. Besides this, for example, the differential process may be performed such that the edge in the direction along the first linear portion 11 is emphasized and the second linear portion 12 disappears from the image B2. From the image subjected to differential processing (second differentially processed image), after calculating the period and phase of change in color numerical value by Fourier transform, by performing inverse Fourier transform, a filter according to image B2 subjected to this differential processing An image (second filter image) may be constructed, and the damaged area of the roof 10 may be identified from this filter image.

  Furthermore, a synthetic differentially processed image obtained by synthesizing the first differentially processed image and the second differentially processed image, and a synthetic filter image obtained by synthesizing the first filter image and the second filter image described above are created, and the synthetic filtered image is generated. The synthetic differentially processed image may be superimposed to identify the damaged portion of the roof 10. For example, in the combination of the first differentially-processed image and the second differentially-processed image, it is preferable to combine so that the first linear portion 11, the second linear portion 12, and the damaged portion 13 such as a crack remain. Furthermore, in the combination of the first filter image and the second filter image, it is preferable to combine so that the first linear portion 11 and the portion to be eliminated as the second linear portion 12 remain in both filter images. .

4. Regarding Damage Detection Method According to Third Embodiment In the third embodiment, a damaged portion is detected by performing a series of steps shown in FIG. 14 on the roof 10. The damage detection method according to the third embodiment will be described below with reference to FIGS. 14 and 15. The third embodiment corresponds to an embodiment of the first invention.

  The damage detection method of the roof 10 according to the third embodiment is different from the second embodiment in that the constructed filter image is further corrected. A filter image construction step S95 and a second linear portion removal step S96 In addition, the following steps are further performed. Therefore, in the present embodiment, the above-described imaging step S91 to the filter image constructing step S95 are performed. FIG. 15 (a) is an image obtained in the differentiation processing step S94, and FIG. 15 (c) is an image obtained in the filter image construction step S95.

  Next, a straight line detection process step S141 is performed. A straight line along the second linear portion 12 is detected by image processing on the captured image of the roof 10, and an image composed of the detected straight line is created. In this embodiment, detection of a straight line can be performed by generally known image processing, and is performed by Hough transform (Hough transform).

  Next, a detection determination step S142 is performed. Specifically, when the detected straight line and the second linear portion 12 do not match (for example, when a straight line is detected at a position other than the second linear portion), the straight line detection process step It returns to S141. In the straight line detection process step S141, the straight line detection condition of the Hough transform is changed, and the straight line is detected again.

  Such an operation is repeated, and when the detected straight line and the second linear portion 21 have consistency, for example, an image as shown in FIG. 15B is obtained. In this case, the process proceeds to the filter image correction step S143. In this step, the filter image constructed by the logical sum is corrected based on the detected straight line image. Specifically, the filter image is corrected so that the second linear portion 12 of the differentially processed image disappears from the detected straight line image. A second linear portion removal step S96 and a damaged site identification step S97 are performed using the corrected filter image.

  In the present embodiment, the filter image calculated by the frequency analysis can accurately represent the number of lines along one of the linear portions and the intervals of the lines, and in the straight line detection by the Hough transform, the one linear portion Because the position of the straight line along the line can be accurately detected, the damaged image of the exterior material can be identified more accurately by the corrected filter image.

5. Regarding Damage Detection Method According to Fourth Embodiment In the fourth embodiment, a damaged portion is detected by performing a series of steps shown in FIG. The damage detection method according to the fourth embodiment will be described below with reference to FIGS. 17 and 18. The fourth embodiment corresponds to an embodiment of the first invention.

  The damage detection method of the roof 10 according to the fourth embodiment is different from the second embodiment in that the constructed filter image is expanded and applied to other parts, and the roof of the ridge shown in FIG. 10 is a method for suitably detecting a damaged site. The present embodiment further performs the following steps between the differential processing step S93 and the frequency analysis step S94.

  Specifically, first, in the pre-processing step S92, an image (extracted image) E1 is extracted from the captured image 10B of the roof 10 captured. The extraction image E1 is an image of the largest range in which a rectangular image can be extracted within one roof surface among the four roof surfaces constituting the roof 10. Therefore, the triangle-shaped image E2 of the part adjacent to the extraction image E1 among the roof surfaces is not extracted. At this time, instead of the captured image 10B, a rectangular image including the image E2 and the images B2 and B2 adjacent to the image E2 may be extracted and used instead of the captured image shown below.

  Next, differentiation processing is performed on the extracted image E1 and the captured image 10B extracted in the differentiation processing step S93. A differential process is performed such that an edge in a direction along the first linear portion 11 is emphasized and the second linear portion 12 disappears, and a first differentially processed image is calculated. Similarly, a differential process is performed such that the edge in the direction along the second linear portion 12 is emphasized and the first linear portion 11 disappears, and a second differentially processed image is calculated.

  Next, a linear portion estimation step S161 is performed. In this step, straight lines along the first and second linear portions 11 and 12 are estimated for each of the first and second differentially processed images. The estimation of the straight line is performed by the Hough transform (Hough transform) as in the straight line detection processing step S141. Thus, straight lines are detected in a lattice.

  Next, an intersection point calculation step S162 is performed. In this step, using the straight line estimated in the linear portion estimation step S161, as shown in the left diagram of FIG. 18A, the intersection of the four corners of the extracted image E1 is calculated. The extracted steps are four points in the case where the quadrangle formed by the straight line has the largest area.

  Next, a correction coefficient calculation step S163 is performed. In this process, as shown in the right figure of FIG. 18A, the parameters (correction coefficient) of the projective transformation are set so that the quadrangle connecting four points becomes a rectangle (the angle of the four corners is 90 °). calculate.

  Next, a pattern synthesis step S164 is performed as necessary. In this step, the first and second differentially processed images are combined. In the present embodiment, the following steps are performed using the second differentially processed image in which the edge in the direction along the second linear portion 12 is emphasized without performing synthesis.

  Next, an area shape correction step S165 is performed. In this step, as shown in the left and right views of FIG. 18B, the second differentially processed image is projected by projective transformation using the parameter (correction factor) of the projective transformation calculated in the correction coefficient calculation step S163. to correct. Thus, the second differentially-processed image is captured from the normal direction of the roof surface of the roof 10 and corrected to the second differentially-processed image. The differential process of the captured image 10B is also performed by the same series of operations.

  Next, the region expansion step S166 is performed. In this step, as shown in FIG. 19, the second differentially-processed image is copied in the area adjacent to the second differentially-processed image to create an extended image obtained by extending the second differentially-processed image. This extended image is an image used for filter construction. When creating the expanded image, the second differentially processed image is duplicated such that the pitch of the second linear portions 11 of the expanded image is the same. The duplication may be either a duplicate of the second differentially-processed image or a duplicate of the second differentially-processed image.

  Next, a frequency analysis step S94 and a filter image construction step S95 are performed on the created expanded image. At this time, the steps from the straight line detection step S141 to the filter image correction step S143 of the third embodiment may be performed. In this way, an extended filter image corresponding to the extended image is constructed as a filtered image. In the present embodiment, the expanded filter image is constructed by the second differentially-processed image. However, the expanded filter image may be constructed by the first differentially-processed image, or these expanded filter images may be synthesized. FIG. 17 (b) is an expanded filter image obtained by this combination.

  Finally, in the damaged part specifying step S97, the expansion filter image is superimposed on the corrected captured image to form an area corresponding to the extracted image E1 and an area adjacent to the area (an area corresponding to the image E2). Identify the site of injury.

  In the fourth embodiment, it is possible to more easily identify the damaged area of the area from which the extracted image is extracted and the area adjacent thereto by the expansion filter image without constructing a new filter image. Here, as shown in FIG. 17A, the extracted image E1 has a rectangular shape, and an image adjacent thereto has the same interval as an image extracted in a shape different from the rectangular shape (specifically, a triangular shape). In the case where the first and second linear portions 11 and 12 are included, constructing a rectangular-shaped filter image including adjacent areas may reduce the accuracy of identifying the damaged site of the roof 10. However, in the present embodiment, since the image of the portion with high detection accuracy is expanded to create the expanded filter image, a region having a shape different from the rectangular shape in which the first and second linear portions 12 are formed (triangular For the image E2) of the shape, the damaged part of the roof 10 in the area can be identified more accurately.

  Furthermore, in the region shape correction step S165, as imaged from the normal direction of the roof surface of the roof 10 (the front of the roof surface of the roof 10), the second differentially processed image and the imaged image subjected to differential processing are It corrected beforehand. By performing a series of steps using these corrected images, the damaged area of the roof 10 is identified, so that distortion of the image due to perspective caused by the imaging conditions can be corrected, so that erroneous detection of the damaged area of the roof 10 can be made. It can be avoided.

  In the fourth embodiment, the extension filter image is created by copying the extraction image before constructing the filter image, but for example, after the filter image is constructed, the filter image is duplicated and expanded. An extended filter image may be constructed.

  As mentioned above, although the embodiment of the present invention was explained in full detail, the present invention is not limited to the above-mentioned embodiment, and various design is possible in the range which does not deviate from the spirit of the present invention described in the claim. It is possible to make changes. For example, in the present embodiment, the roof is exemplified as the exterior material, but the exterior material may be, for example, an outer wall material of a building or a paved road.

  Furthermore, in the present embodiment, the first invention and the second invention are individually implemented for different areas of the captured image, but for example, the same area of the captured image is described with reference to FIG. After performing the series of processes described above, the series of processes described in FIG. 9 may be performed, or after the series of processes described in FIG. 9 may be performed, the series of processes described in FIG. This makes it possible to more accurately identify the damaged site of the exterior material.

1: Building, 5: Unmanned Aerial Vehicle (UAV), 6: Imaging device, 7: Processing device, 10: Roof (exterior material), 11: first linear portion, 12: second linear portion, 13: Damaged part, S21, S91: imaging step, S23, S93: differentiation processing step, S24, S94: frequency analysis step, S25: first linear portion removal step, S26, S97: damaged part identification step, S95: filter image Construction process, S96: second linear portion removal process, S141: straight line detection process, S165: area shape correction process, S166: area expansion process, A1 to A6, B1 to B6, E1, E2: image

Claims (9)

What is claimed is:
Imaging the exterior material;
Performing a differential process on the captured image so as to convert the image into gray scale so that an edge in a direction along the linear portion is emphasized in the image converted into the gray scale;
Pixels arranged in a direction intersecting with the linear portion in a numerical color value digitized according to a change in color from black to white for the image subjected to the differential processing Performing a frequency analysis of the color values by performing Fourier transform on the color values of the group, and calculating a period and a phase of change of the color values along the intersecting direction;
Constructing a filter image according to the image subjected to the differential processing by performing inverse Fourier transform based on the period and the phase for each pixel group;
And d. Identifying the damaged portion of the outer covering material by superimposing the filter image on the image subjected to the differential processing. The linear portions are equally spaced so as to connect the respective first linear portions between a plurality of first linear portions oriented at equal intervals and an adjacent first linear portion. A plurality of second linear portions, and a group of a plurality of second linear portions arranged along the first linear portion, and a plurality of second portions adjacent thereto Groups of linear portions are arranged in a staggered manner offset in a direction along the first linear portion,
In the step of performing the differential processing, an edge in a direction along one of the first and second linear portions is emphasized in the image converted into the gray scale, and the other is Perform differentiation processing so that the linear part disappears from the image,
2. The package according to claim 1, wherein, in the step of calculating the period and the phase of the change of the color value, the period and the phase of the change of the color value along the other linear portion are calculated. Damage detection method. Detecting a straight line along the one of the linear portions of the captured image of the exterior material by Hough transformation, and creating an image composed of the detected straight line;
3. The method according to claim 2, further comprising the step of: correcting the constructed filter image based on the detected straight line image. Before the step of performing the differential processing, an image is extracted from a captured image of the captured exterior material,
Performing the differentiation process on the extracted image extracted and the captured image in the step of performing the differentiation process;
By extracting the extracted image subjected to the differentiation process in a region adjacent to the extracted image subjected to the differentiation process, an extended image obtained by extending the extracted image is created.
The step of calculating the change period and phase of the color value and the step of constructing the filter image are performed on the generated extended image, and the expanded filter image corresponding to the expanded image is processed as the filter image. Build
In the step of identifying the damaged portion of the outer covering material, the extended filter image is superimposed on the captured image subjected to the differentiation processing to obtain an area from which the extracted image is extracted and the outer covering material in an area adjacent thereto. The damage detection method of the exterior material as described in any one of Claims 1-3 characterized by identifying a damage site | part. After performing the differentiation process and before creating the expanded image, the extracted image subjected to the differentiation process and the captured image subjected to the differentiation process are viewed from the normal direction of the surface of the exterior material Correct the image so as to be an image that has been imaged and subjected to the differential processing,
Constructing the expanded filter image from the corrected extracted image;
The damaged region of the exterior material in the area where the extracted image is extracted and the area adjacent thereto is identified by superimposing the expanded filter image on the corrected captured image. How to detect damage to exterior packaging materials. What is claimed is:
Imaging the exterior material;
Performing a differential process on the captured image so as to convert the image into gray scale so that an edge in a direction along the linear portion is emphasized in the image converted into the gray scale;
Pixels arranged in a direction intersecting with the linear portion in a numerical color value digitized according to a change in color from black to white for the image subjected to the differential processing Performing a frequency analysis of the color values by performing Fourier transform on the color values of the group, and calculating a period and a phase of change of the color values along the intersecting direction;
A step of estimating an appearance pattern of the linear portion based on the period and the phase, and removing the linear portion from the image with respect to the image subjected to the differential processing based on the appearance pattern; ,
And d. Identifying the damaged portion of the sheathing material from the image from which the linear portion has been removed.   In the step of identifying the damaged site, the damaged site is identified by superimposing the image obtained in the removing step on the image captured in the imaging step. How to detect damage to exterior packaging materials.   The said exterior material is a roof of the building in which the slate tile was laid, The damage detection method of the exterior material as described in any one of Claims 1-7 characterized by the above-mentioned.   The damage detection method of the exterior material as described in any one of Claims 1-8 which image-captures the said exterior material from an unmanned aerial vehicle carrying an imaging device.

Images