KR102416146B1 - Apparatus and method for evaluating state of road pavement - Google Patents

Abstract

A road pavement state determination device and method are presented. The road pavement state determination method is performed by a road pavement state determination device. The method includes the steps of: constructing machine learning data including a training image, a characteristic value associated with the training image, and type and severity information of a defect corresponding to the training image; training a defect classification model using the constructed learning data; and extracting a defect group from a defect image obtained from a road pavement image, inputting the extracted defect group image and feature values into a trained defect classification model, and calculating the type and severity of defects included in the defect image.

Description

Apparatus and method for determining road pavement conditions using machine learning and image processing techniques

Embodiments disclosed herein relate to an apparatus and method for determining a road pavement state for analyzing a pavement image of a road to evaluate the type and severity of a defect included in the road pavement.

The durability of the pavement depends on the pavement environment, the structure of the road, the condition of the ground, the amount of traffic, and the partial location of the pavement. Therefore, even if the pavement of the same road constructed within the same construction period, the degree of damage is different for each part, and the pattern and type of defects caused by the damage are also different.

Therefore, after the pavement is constructed, repairs must be performed by partially applying different construction methods depending on the degree of damage, the extent of the damage, and the method of damage. At this time, in order to decide whether to repair or not, and to determine the repair method, it is necessary to clearly detect the damaged part of the road and to determine the degree or degree of damage in detail.

For this purpose, recently, a method of photographing an image of a road using an automatic pavement condition survey device and analyzing the photographed image to detect damage has been widely used. In particular, in analyzing cracks in road pavement, various methods have been developed to automatically detect cracks using an image processing method and to quantify cracks by automatically quantifying crack conditions.

However, the conventional image analysis and crack quantification method has a problem in that accuracy is poor, so it is inconvenient for a person to perform actual measurement analysis while viewing the image.

In addition, although conventional crack analysis systems were able to significantly calculate the amount of cracks, there was a problem in that they did not support the qualification of cracks that determine the nature of cracks. That is, since the type of crack could not be determined using the conventional systems, there was a problem that human judgment was required to determine the repair method.

In relation to this, Korean Patent Registration No. 10-1456565 is about a 'road pavement damage investigation and analysis system, and road pavement damage investigation and analysis method', and describes a method for collecting and analyzing planar image data. However, even with this prior art, since the type of damage cannot be clearly identified, there is a hassle in that the manager has to visually check the image in order to determine the need for repair or the method to be used for repair.

Therefore, there is a need for a technique for solving the above-mentioned problems.

On the other hand, the above-mentioned background art is technical information that the inventor possessed for the purpose of derivation of the present invention or acquired during the derivation process of the present invention, and it cannot be said that it is necessarily known technology disclosed to the general public before the filing of the present invention. .

SUMMARY Embodiments disclosed in the present specification provide an apparatus and method for determining a road pavement state that can detect a defect from a photographed image of a road pavement, and accurately determine the type and severity of the detected defect.

Embodiments disclosed in the present specification, without the human eye to confirm the image of the road pavement, it is an object to provide a road pavement condition determination apparatus and method that can reliably determine the need for repair of the road pavement and the method to be used for the repair have.

Further, the embodiments disclosed in the present specification, in calculating the type and severity of the defect by analyzing the photographed image of the road pavement, can improve the accuracy of determination by using an artificial intelligence model that learned the defect included in the photographed image. An object of the present invention is to provide an apparatus and method for determining the state of a road pavement.

As a technical means for achieving the above-mentioned technical problem, according to an embodiment, the road pavement determination method performed by the road pavement state determination apparatus, a training image and a characteristic value associated with the training image, and the training image Constructing a machine learning learning material including the type and severity information of the corresponding defect; training a defect classification model using the constructed learning data; and grouping the connected defects in the defect image from the defect image obtained from the road pavement image, expanding the shape of the defect outwardly to merge the overlapping defects to extract the defect group, and the image and characteristics of the extracted defect group inputting the value into the trained defect classification model, and calculating the type and severity of the defect included in the defect image, wherein the calculating includes: obtaining a defect image from the pavement image; forming a pixel group by grouping connected defective pixels in the defective image; generating a defect group by expanding the shape of each pixel group to the outside; calculating a characteristic value of the defect group; and inputting a defect group image and characteristic value corresponding to the defect group into the trained defect classification model, and calculating a defect type and severity for each defect group included in the defect image, wherein the defect group The calculating of the characteristic value of may include: generating the defect group image by cropping the defect image in the form of a minimum rectangle including polygons of the defect group; and calculating a characteristic value for a defect in the defect group image by using the previously calculated characteristic values of the line segment.

According to another embodiment, the apparatus for determining the state of a road pavement includes: a communication unit for receiving an image of a road pavement that is a target for which the type and severity of a defect are determined; A training image and a feature value related to the training image, and a defect classification model trained using machine learning learning data including information on the type and severity of a defect corresponding to the training image, a storage unit for storing the road pavement image ; Obtaining a defect image from the road pavement image, grouping the connected defects in the defect image, expanding the shape of the defect in the outward direction to merge overlapping defects to extract a defect group, which corresponds to the extracted defect group By inputting the defect group image and the characteristic value into the defect classification model, the type and severity of the defect included in the defect image are calculated. a control unit generating the defect group by merging overlapping defects; and an input/output unit for outputting data on the type and severity of the defect calculated by the control unit, wherein the control unit groups the defective pixels connected in the defect image to form a pixel group, and determines the shape of each pixel group. Each defect group included in the defect image is generated by expanding it to the outside, calculating the characteristic value of the defect group, and inputting a defect group image and characteristic value corresponding to the defect group into the defect classification model. calculates the type and severity of the defect, the control unit crops the defect image in the form of a minimum rectangle including polygons of the defect group to generate the defect group image, and uses the characteristic values of the line segment calculated in advance Thus, a characteristic value for a defect in the defect group image is calculated.

According to another embodiment, in a computer-readable recording medium in which a computer program for performing a road pavement state determination method is recorded, the road pavement determination method includes a training image, a characteristic value associated with the training image, and the training image. Constructing a machine learning learning material including the type and severity information of the corresponding defect; training a defect classification model using the constructed learning data; and grouping the connected defects in the defect image from the defect image obtained from the road pavement image, expanding the shape of the defect outwardly to merge the overlapping defects to extract the defect group, and the image and characteristics of the extracted defect group and inputting the value into the trained defect classification model, and calculating the type and severity of the defect included in the defect image.

And according to another embodiment, in the computer program stored in the medium to perform the road pavement state determination method performed by the road pavement state determination apparatus, the road pavement determination method is a training image and a characteristic associated with the training image value, and constructing machine learning learning materials including type and severity information of a defect corresponding to the training image; training a defect classification model using the constructed learning data; and grouping the connected defects in the defect image from the defect image obtained from the road pavement image, expanding the shape of the defect in the outward direction to merge the overlapping defects to extract the defect group, and the image and characteristics of the extracted defect group and inputting the value into the trained defect classification model, and calculating the type and severity of the defect included in the defect image.

According to any one of the above-described problem solving means, the embodiments disclosed in the present specification detect a defect from a photographed image of a road pavement, and a road pavement state determination device capable of accurately determining the type and severity of the detected defect, and can suggest a way.

According to any one of the above-mentioned problem solving means, it is possible to present an apparatus and method for determining the state of a road pavement that can reliably determine the need for repair of the road pavement and the method to be used for the repair, without the human eye checking the image of the road pavement. can

Furthermore, according to any one of the above-mentioned problem solving means, in analyzing the photographed image of the road pavement and calculating the type and severity of the defect, the accuracy of the determination is increased by using an artificial intelligence model that learned the defect included in the photographed image. A possible road pavement condition determination apparatus and method may be proposed.

Effects obtainable in the disclosed embodiments are not limited to the above-mentioned effects, and other effects not mentioned are clear to those of ordinary skill in the art to which the embodiments disclosed from the description below belong. can be understood clearly.

1 is a block diagram illustrating a functional configuration of an apparatus for determining the state of a road pavement according to an embodiment.
2 is a flowchart illustrating a method for determining the state of a road pavement performed in an apparatus for determining a state of a road pavement according to an embodiment in a step-by-step manner.
3 is a flowchart illustrating in more detail a method for determining the state of a road pavement according to an embodiment.
4 is a flowchart illustrating step S302 in more detail in the method for determining the state of the road pavement according to the embodiment of FIG. 3 .
5 is a flowchart illustrating step S303 in more detail in the method for determining the state of the road pavement according to the embodiment of FIG. 3 .
6 is a flowchart illustrating step S304 in more detail in the method for determining the state of the road pavement according to the embodiment of FIG. 3 .
7 is a flowchart illustrating step S305 in more detail in the method for determining the state of the road pavement according to the embodiment of FIG. 3 .
8 is a flowchart illustrating step S306 in more detail in the method for determining the state of the road pavement according to the embodiment of FIG. 3 .
9 is an exemplary diagram illustrating an example of removing noise from a defect image in a method for determining the state of a road pavement according to an embodiment.
10 and 11 are exemplary views illustrating an example of forming a group in a defect image in a method for determining the state of a road pavement according to an embodiment.
12 is an exemplary diagram illustrating an example of calculating a minimum quadrangle as a group characteristic value in a method for determining the state of a road pavement according to an embodiment.

Hereinafter, various embodiments will be described in detail with reference to the accompanying drawings. The embodiments described below may be modified and implemented in various different forms. In order to more clearly describe the characteristics of the embodiments, detailed descriptions of matters widely known to those of ordinary skill in the art to which the following embodiments belong are omitted. In addition, in the drawings, parts irrelevant to the description of the embodiments are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a component is said to be "connected" with another component, it includes not only the case where it is 'directly connected', but also the case where it is 'connected with another component in between'. In addition, when a component "includes" a component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a functional configuration of an apparatus for determining the state of a road pavement according to an embodiment.

First, as shown in FIG. 1 , the apparatus 100 for determining the pavement state may include an input/output unit 110 , a communication unit 120 , a storage unit 130 , and a control unit 140 .

At this time, the road pavement state determination apparatus 100 is composed of a conventional information processing device, and uses machine learning learning data to learn a defect classification model, or detects a defect from a road pavement image using an already learned defect classification model. and the type and severity of the detected defect can be evaluated. To this end, the data set corresponding to the defect classification model may be stored in the apparatus 100 for determining the state of the road pavement, and this may be in a state before learning is completed or in a state in which learning is completed.

In addition, according to an embodiment, the road pavement state determination apparatus 100 may be implemented as a server-client system. For example, an electronic terminal used by the management subject of the road pavement, and a server that communicates with the electronic terminal and provides information on the type or severity of a defect determined from the road pavement image selected or uploaded by the electronic terminal to the electronic terminal It can be implemented as a server-client system comprising a.

In this case, the electronic terminal may be implemented as a computer, a portable terminal, a television, a wearable device, etc. that may include an interface capable of interacting with a user. Here, the computer includes, for example, a laptop, a desktop, and a laptop equipped with a web browser, and the portable terminal is, for example, a wireless communication device that ensures portability and mobility. , PCS (Personal Communication System), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), GSM (Global System for Mobile communications), IMT (International Mobile Telecommunication)-2000, CDMA (Code) All kinds of handhelds such as Division Multiple Access)-2000, W-Code Division Multiple Access (W-CDMA), Wireless Broadband Internet (Wibro), Smart Phone, Mobile Worldwide Interoperability for Microwave Access (WiMAX), etc. It may include a (Handheld)-based wireless communication device. In addition, the television may include IPTV (Internet Protocol Television), Internet TV (Internet Television), terrestrial TV, cable TV, and the like. Furthermore, a wearable device is, for example, a type of information processing device that can be worn directly on the human body, such as a watch, glasses, accessories, clothes, shoes, etc. can be connected with

In addition, the server may be implemented as a computer capable of communicating through a network with an electronic terminal in which an application for interaction with the user or a web browser is installed, or may be implemented as a cloud computing server. In addition, the server may include a storage device capable of storing data or may store data through a third server.

On the other hand, such a road pavement state determination apparatus 100 may acquire the road pavement data collected through the vehicle road pavement survey equipment. And the obtained road pavement data may include a road pavement image. Accordingly, the road pavement state determination apparatus 100 may detect defects from the pavement image, group the defects, and determine the type and severity of defects for each defect group.

Specifically, the road pavement state determination apparatus 100 includes an input/output unit 110 . Meanwhile, the input/output unit 110 may receive a user's input and output a result of data processing to the user.

Here, the input/output unit 110 is a configuration that allows the user and the road pavement state determination apparatus 100 to interact with each other, and an input unit for receiving a user's input, a user interface, a road pavement image, or a defect detected therefrom As an analysis result of an image or a defect, an output unit for displaying qualitative data of the defect may be included.

Specifically, the input unit may include devices capable of receiving various types of user input, such as a keyboard, a physical button, a touch screen, a camera, or a microphone. Also, the output unit may include a display panel, a speaker, or a headset. However, the present invention is not limited thereto, and the input/output unit 110 may include a configuration supporting various input/output.

The input/output unit 110 may output a screen configured using data stored in the storage unit 130, which will be described later, and receive a command input from the user in real time in response to the output screen. In particular, the input/output unit 110 provides the user with a user interface including a list of a plurality of routes in order to receive a selection of road pavement data for a plurality of road routes managed by the user, so that the user can select a specific route from the route list. of road pavement data can be selected.

Meanwhile, the input/output unit 110 may provide the user with a result of analyzing the road pavement image included in the road pavement data selected by the user. The analysis result can be data on the type or severity of a defect for each defect group included in the road pavement image.

Meanwhile, the road pavement state determination apparatus 100 may include a communication unit 120 . The communication unit 120 may include a communication module supporting at least one of various wired and wireless communication methods. For example, the communication module may be implemented in the form of a chipset.

The communication unit 120 may communicate with the irradiation equipment (not shown) through a network. The communication unit 120 may receive the road pavement data of the road to be investigated from the survey equipment.

In this case, the wireless communication supported by the communication unit 120 may be, for example, Wireless Fidelity (Wi-Fi), Wi-Fi Direct, Bluetooth, Ultra Wide Band (UWB), or Near Field Communication (NFC). Also, wired communication supported by the communication unit 130 may be, for example, USB or High Definition Multimedia Interface (HDMI). Also, the communication unit 130 may transmit data or a message to a destination through the Internet or a mobile communication network.

And the road pavement state determination apparatus 100 may include a storage unit 130 . Here, the storage unit 130 may store a program installed and executed in the road pavement state determination apparatus 100 and various data.

As various data stored in the storage unit 130, a data set constituting the defect classification model, learning materials for learning the defect classification model, road pavement data including a road pavement image, and defects are grouped from the pavement image. and data generated in the process of determining the type or severity of the defect for each defect group may be included.

In this case, the defect classification model may be configured including, for example, a 'Dense net' having good performance in image classification.

In addition, the machine learning learning material for learning the defect classification model is a road pavement image containing defects, a training image containing an image of each defect group, a characteristic value corresponding to each training image, and the corresponding training image. It may consist of a data set containing information on the type and severity of the corresponding defect group. In this case, the training image may be an image edited in the form of a minimum rectangle including defect images set as one group.

In addition, the characteristic value may be calculated in advance for each training image and related to the training image. The characteristic value will be described in more detail below.

Also, the type and severity of the defect group corresponding to each training image may be an average of the type and severity evaluated by one or more experts. For example, the type of defect can be identified as one of 'longitudinal crack', 'transverse crack', 'construction joint crack', 'fatigue crack', 'block crack', and 'sofa repair', Severity can be evaluated as 'high', 'moderate' or 'low'.

At this time, in learning the defect classification model using the learning data, the controller 140, which will be described later, inputs the training image and its associated characteristic values into the defect classification model, the type and severity of the defect group corresponding to the training image. can be trained to output.

Meanwhile, the road pavement state determination apparatus 100 may include a control unit 140 . The control unit 140 controls the overall operation of the road pavement state determination apparatus 100, and may include a processor such as a CPU.

The controller 140 may read data stored in the storage unit 130 or write new data into the storage unit 130 . In addition, the control unit 140 may provide a route data management service to the user by executing the program stored in the storage unit 130 .

Specifically, the controller 140 may determine the type and severity of the defect by analyzing the defects included in the road pavement image for each group by using the defect classification model stored in the storage unit 130 .

In this case, the controller 140 first detects a portion estimated to be a defect in the road pavement image in order to determine the type and severity of the defect, and classifies each defect group from the defect image of the detected portion.

In this case, the 'defect group' refers to each polygon formed by grouping the connected defective pixels in the defect image including the defect on the road pavement image to form a pixel group, and expanding the shape of the pixel group in the outward direction, at this time Overlapping polygons can be merged into one group.

Here, the 'pixel group' is a group formed by consecutive pixels estimated to be defects. For example, a plurality of pixels having a value of 0 or 1 when a defect image including a defect detected in a pavement image is binarized. It may be a cluster of pixels adjacent to each other having a value estimated to be a heavy defect, for example, a value of 1.

In addition, 'polygon' means a figure formed by expanding the shape of a pixel group by a preset number of pixels in the outer direction.

Meanwhile, the controller 140 may use various image processing techniques in the process of forming the defect group as described above. Also, the controller 140 may form each defect group and generate a defect group image corresponding to each defect group.

Furthermore, the control unit 140 may calculate a characteristic value for each defect group and output the type and severity of the defect for each defect group by using the image and the characteristic value of the defect group as input to the defect classification model.

Meanwhile, at this time, the characteristic value of each defect group is the area of the polygon forming the defect group, the area of the minimum rectangle including the polygon corresponding to the defect group, the ratio of the width to the length of the minimum rectangle, and the area of the polygon and the minimum rectangular area. Ratio, the number of nodes and junctions included in the polygon, the total number of line segments included in the polygon, the number of vertical line segments, the number of horizontal line segments, the total length of the line segments, the length of the vertical segments, the length of the horizontal segments , may include at least a portion of the weighted average width of all line segments. In this case, a node may include a starting point, an ending point, and an intersection point of a line segment included in a polygon, and a junction means an intersection point.

Meanwhile, in order to calculate the characteristic value of the above-described defect group, the controller 140 may extract a line segment component in a polygon forming the defect group. To this end, before forming a polygon, the controller 140 may thin each pixel group to extract each node, and may recognize and separate line segments based on the extracted node.

A detailed configuration in which the control unit 140 forms a defect group and determines the type and severity of a defect for each group will be described in more detail later with reference to FIGS. 2 to 8 .

Hereinafter, with reference to the flowchart of FIGS. 2 to 8 and the exemplary diagrams of FIGS. 9 to 12 , the above-described road pavement condition determination apparatus 100, in particular, the control unit 140 of the road pavement condition determination apparatus 100 performs A method of determining the state of the road pavement will be described in detail. 2 is a flowchart illustrating a method for determining the state of a road pavement according to an embodiment. 3 is a flowchart illustrating in more detail a method for determining the state of a road pavement according to an embodiment, and FIGS. 4 to 8 are flowcharts for explaining each step included in FIG. 3 in more detail. And Figures 9 to 12 are exemplary views for explaining the process of forming a defect group in the defect image by the road pavement state determination method.

The method for determining the state of a road pavement according to the embodiment shown in FIGS. 2 to 8 includes steps of time-series processing in the apparatus 100 for determining the state of a road pavement shown in FIG. 1 . Therefore, even if omitted below, the contents described above with respect to the apparatus 100 for determining the road pavement state shown in FIG. 1 may be applied to the method for determining the state of the road pavement according to the embodiment shown in FIG. 3 .

First, as shown in Fig. 2, the road pavement state determination apparatus 100 starts with a training image and a step (S201) of constructing machine learning learning materials including characteristic values of a defect group corresponding to the training image. In this case, the training image is a defect image including a defect corresponding to one defect group as described above.

And at this time, the characteristic value of the defect group included in the learning material is the defect group characteristic value calculated by the road pavement state determination apparatus 100 extracting the defect group and the minimum rectangular image corresponding thereto using the actual road pavement image. are values that substantially correspond to . That is, the characteristic values included in the learning material also include the area of polygons forming the defect group, the area of the minimum rectangle including the polygon corresponding to the defect group, the ratio of the width to the length of the minimum rectangle, the area of the polygon and the minimum rectangular area. Ratio, the number of nodes and junctions included in the polygon, the total number of line segments included in the polygon, the number of vertical line segments, the number of horizontal line segments, the total length of the line segments, the length of the vertical segments, the length of the horizontal segments , may include at least a portion of the weighted average width of all line segments.

Subsequently, the road pavement state determination apparatus 100 may train the defect classification model using the learning data (S202). The defect classification model can be trained by inputting the training image and the characteristic value and outputting the defect type and severity of the defect group corresponding to the training image.

Next, the road pavement state determination apparatus 100 may detect a defect using the actual road pavement image, and calculate the type and severity of the defect ( S203 ).

Here, step S203 may be performed by the method shown in FIG. 3 .

As shown in FIG. 3 , the apparatus 100 for determining the state of the road pavement may first obtain a defect image from the image of the road pavement ( S301 ). Here, the road pavement state determination apparatus 100 generates a defect image in a way that, in the road pavement image, objects such as facilities, for example, a manhole or a lane image, etc. are separated and removed, and only objects corresponding to the defect are included. can Alternatively, the road pavement state determination apparatus 100 may use a model for recognizing an object corresponding to a defect to obtain a defect image in which only the corresponding part is cropped.

Then, the road pavement state determination apparatus 100 may group the connected defects in the defect image (S302). Referring to FIG. 4 , the apparatus 100 for determining the state of a road pavement may binarize pixels included in a defect image ( S401 ). For example, the road pavement state determination apparatus 100 may divide and estimate all pixels in the defective image into defective pixels and non-defective pixels, which, for example, converts the road pavement image to grayscale, This may be performed in a manner in which pixel values having a value greater than or equal to the threshold value are converge to 1 and all pixel values having a value less than the threshold value are converged to 0 using the determined threshold value. In addition, one of the pixels having a value of 1 or 0 may be classified as a defective pixel, and the remaining pixels may be classified as a non-defective pixel.

In addition, the road pavement state determination apparatus 100 may group defective pixels adjacent to each other from the binarized defect image ( S402 ), and extract a pixel group formed by defective pixels connected to each other.

Subsequently, when the pixel group includes defective pixels less than or equal to a preset number, the road pavement state determination apparatus 100 may remove all of the pixel groups as noise ( S403 ).

Meanwhile, the road pavement state determination apparatus 100 may separate line segments from defects belonging to one pixel group, as shown in FIG. 3 ( S303 ). To this end, the road pavement state determination apparatus 100 may thin each pixel group to have a width of 1 pixel as shown in FIG. 5 ( S501 ). For this, the road pavement state determination apparatus 100 may use a conventional image thinning algorithm.

Subsequently, the road pavement state determination apparatus 100 may recognize a node at which each line is separated or intersected (S502). Here, the node may be the same as the node described above.

In addition, the road pavement state determination apparatus 100 may classify each line based on the nodal point to separate the line segments (S503).

Then again in FIG. 3 , the road pavement state determination apparatus 100 may calculate a characteristic value of each separated line segment ( S304 ). In this case, the characteristic value of the line segment may include a length of the segment, an average width or a maximum width, a moving direction of the segment, and the like. In this case, the length of the line segment may be calculated based on the thinned line segment, and the average width or the maximum width may be calculated from the original shape before the thinning of the line segment. In addition, the moving direction of the line segment may be calculated based on an angle formed by a starting point and an end point with respect to the thinned line segment. In this case, the direction of the line segment may be again divided into one of a longitudinal direction and a transverse direction. The longitudinal direction may correspond to a case in which the angle is 0 to 45 degrees, and the transverse direction may correspond to a case in which the angle is 45 to 90 degrees.

After calculating the characteristic value of the line segment as described above ( S601 ), the road pavement state determination apparatus 100 may remove some of the line segments as noise based on the calculated characteristic value ( S602 ). For example, when the length of the continuous line segment is less than or equal to the preset minimum length, the corresponding continuous line segment may be removed as noise ( S602 ).

For example, as shown in FIG. 9 , in a defect image, after recognizing each nodule, the length of each line segment is calculated, and a group of pixels having a length of a continuous line segment equal to or less than the minimum length may be removed. .

The road pavement state determination apparatus 100 may then generate a defect group by merging the pixel groups (S305). Specifically, as shown in FIG. 7 , the apparatus 100 for determining the pavement state may configure a polygon by expanding the shape of a pixel group by a certain number of pixels in the outer direction ( S701 ). For example, as shown in FIG. 10 or 11 , a polygon P may be configured by expanding a shape formed by defective pixels included in a pixel group by a predetermined pixel, for example, 15 pixels to the outside.

And the road pavement state determination apparatus 100 may form one polygon by merging two or more polygons overlapping some areas (S702). As shown in FIG. 11 , when an area where two polygons overlap each other exists, the two polygons may be merged into one.

Also, the road pavement state determination apparatus 100 may remove some polygons as noise. For example, if the number of line segments included in a polygon is less than or equal to a certain number, or the number of nodal points is less than or equal to a preset number, or if the area of a polygon is less than or equal to a preset area, the corresponding polygon and its corresponding defective pixels may be removed as noise. (S703).

Accordingly, each polygon remaining after being finally merged or removed may form a defect group.

And the road pavement state determination apparatus 100 may calculate the characteristic value of the defect group (S306).

In step S306 , the apparatus 100 for determining the state of the road pavement may generate a defect group image by first cropping the defect image to a minimum rectangular shape including polygons ( S801 ). Here, the defect group image is extracted in the form of a minimum rectangle including one defect group, respectively, and in this case, while cropping the defect group image from the defect image, defect pixels of other groups that may be included may be extracted while excluding the defect pixels. That is, in each defect group image, only defective pixels belonging to the corresponding defect group may be included.

Then, it is possible to calculate the characteristic values of the defect group image, that is, the defect group including the minimum rectangular area (S802). Here, at this time, the characteristic values of the defect groups are the same as those described above. 12, it is shown that the road pavement state determination apparatus 100 obtains a defect group image by cropping the defect image in the form of a minimum rectangle (V) including a polygon (P).

Meanwhile, in this case, the apparatus 100 for determining the state of the road pavement may use the characteristic values of the line segments calculated in advance in calculating the characteristic values of the defect group.

And when the calculation of the characteristic value of the defect group is completed in this way, the road pavement state determination apparatus 100 may determine the type and severity of the defect by using the image and the characteristic value of the defect group (S307).

At this time, the defect detection model of the road pavement state determination apparatus 100 may output a numerical value indicating the type of defect and a numerical value indicating the severity as a result value according to the learning. For example, the type of defect can be identified as one of 'longitudinal crack', 'transverse crack', 'construction joint crack', 'fatigue crack', 'block crack', and 'sofa repair', Severity can be evaluated as 'high', 'moderate' or 'low'.

In addition, the road pavement state determination apparatus 100 may display the calculated result value and the defect group image together to the user.

The term '~ unit' used in the above embodiments means software or hardware components such as field programmable gate array (FPGA) or ASIC, and '~ unit' performs certain roles. However, '-part' is not limited to software or hardware. '~' may be configured to reside on an addressable storage medium or may be configured to refresh one or more processors. Accordingly, as an example, '~' indicates components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, and procedures. , subroutines, segments of program patent code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables.

The functions provided in the components and '~ units' may be combined into a smaller number of elements and '~ units' or separated from additional components and '~ units'.

In addition, components and '~ units' may be implemented to play one or more CPUs in a device or secure multimedia card.

The road pavement state determination method according to the embodiment described with reference to FIGS. 2 to 8 may also be implemented in the form of a computer-readable medium for storing instructions and data executable by a computer. In this case, the instructions and data may be stored in the form of program codes, and when executed by the processor, a predetermined program module may be generated to perform a predetermined operation. In addition, computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may be a computer recording medium, which is a volatile and non-volatile and non-volatile storage medium implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. It may include both volatile, removable and non-removable media. For example, the computer recording medium may be a magnetic storage medium such as HDD and SSD, an optical recording medium such as CD, DVD, and Blu-ray disc, or a memory included in a server accessible through a network.

In addition, the road pavement state determination method according to the embodiment described with reference to FIGS. 2 to 8 may be implemented as a computer program (or computer program product) including instructions executable by a computer. The computer program includes programmable machine instructions processed by a processor, and may be implemented in a high-level programming language, an object-oriented programming language, an assembly language, or a machine language. . In addition, the computer program may be recorded in a tangible computer-readable recording medium (eg, a memory, a hard disk, a magnetic/optical medium, or a solid-state drive (SSD), etc.).

Accordingly, the method for determining the state of the road pavement according to the embodiment described with reference to FIGS. 2 to 8 may be implemented by executing the computer program as described above by the computing device. The computing device may include at least a portion of a processor, a memory, a storage device, a high-speed interface connected to the memory and the high-speed expansion port, and a low-speed interface connected to the low-speed bus and the storage device. Each of these components is connected to each other using various buses, and may be mounted on a common motherboard or in any other suitable manner.

Here, the processor may process a command within the computing device, such as, for example, to display graphic information for providing a graphic user interface (GUI) on an external input or output device, such as a display connected to a high-speed interface. Examples are instructions stored in memory or a storage device. In other embodiments, multiple processors and/or multiple buses may be used with multiple memories and types of memory as appropriate. In addition, the processor may be implemented as a chipset formed by chips including a plurality of independent analog and/or digital processors.

Memory also stores information within the computing device. As an example, the memory may be configured as a volatile memory unit or a set thereof. As another example, the memory may be configured as a non-volatile memory unit or a set thereof. The memory may also be another form of computer readable medium, such as, for example, a magnetic or optical disk.

In addition, the storage device may provide a large-capacity storage space to the computing device. A storage device may be a computer-readable medium or a component comprising such a medium, and may include, for example, devices or other components within a storage area network (SAN), a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory, or other semiconductor memory device or device array similar thereto.

The above-described embodiments are for illustration, and those of ordinary skill in the art to which the above-described embodiments pertain can easily transform into other specific forms without changing the technical idea or essential features of the above-described embodiments. you will understand Therefore, it should be understood that the above-described embodiments are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope to be protected through this specification is indicated by the claims described below rather than the above detailed description, and should be construed to include all changes or modifications derived from the meaning and scope of the claims and their equivalents. .

100: road pavement state determination device 110: storage unit
120: input/output unit 130: control unit
140: communication department

Claims (14)

carried out by the road pavement condition determination device,
Constructing a machine learning learning material comprising a training image and a characteristic value associated with the training image, and type and severity information of a defect corresponding to the training image;
training a defect classification model using the constructed learning data; and
From the defect image obtained from the road pavement image, the defects connected in the defect image are clustered, the shape of the defect is expanded outwardly to merge the overlapping defects to extract the defect group, and the image and the characteristic value of the extracted defect group inputting to the trained defect classification model, and calculating the type and severity of the defect included in the defect image,
The calculating step is
obtaining a defect image from the road pavement image;
forming a pixel group by grouping connected defective pixels in the defective image;
generating a defect group by expanding the shape of each pixel group to the outside;
calculating a characteristic value of the defect group; and
inputting a defect group image and characteristic value corresponding to the defect group into the trained defect classification model, and calculating a defect type and severity for each defect group included in the defect image;
Calculating the characteristic value of the defect group comprises:
generating the defect group image by cropping the defect image in the form of a minimum rectangle including polygons of the defect group; and
Comprising the step of calculating the characteristic value of the defect in the defect group image by using the characteristic values of the line segment calculated in advance, the road pavement state determination method. According to claim 1,
Forming the pixel group comprises:
binarizing pixels included in the defect image;
forming a pixel group by grouping adjacent defective pixels; and
A method for determining the state of a road pavement comprising the step of removing a group of pixels less than or equal to a set number of pixels as noise. According to claim 1,
The step of creating the defect group comprises:
separating line segments within each pixel group;
calculating characteristic values of each line segment; and
Constructing a polygon by expanding the shape of each pixel group to the outside, and merging the overlapping polygons to create a defect group,
Creating a defect group by merging the polygons includes:
forming polygons by expanding the shape of each pixel group to the outside;
merging the overlapping polygons into one;
removing polygons corresponding to preset criteria as noise; and
setting each polygon into one defect group;
The step of separating the line segment is
thinning lines included in each pixel group to have a predetermined width;
recognizing a node at which the thinned line intersects or separates; and
A method for determining the state of a road pavement, comprising the step of classifying a line based on the nodal point and dividing the line into each line segment. 4. The method of claim 3,
Calculating the characteristic value of the line segment comprises:
calculating a characteristic value for at least one of a length, a width, and a direction between a start point and an end point of each separated line segment; and
A method for determining the state of a road pavement comprising the step of removing a line segment having the characteristic value equal to or less than a preset value as noise. 5. The method of claim 4,
Calculating the characteristic value for the one characteristic includes:
calculating a length between a start point and an end point of the line segment based on the thinned line segment;
calculating a width from the original before thinning of the line segment; and
Calculating a direction based on an angle between the starting point and the ending point of the line segment based on the thinned line segment; including; a method for determining the state of a road pavement. 4. The method of claim 3,
The step of removing the polygon as noise is,
If any one of the number of line segments, the number of nodal points, or the area included in the polygon is less than or equal to a preset standard, removing the corresponding polygon and the corresponding defective pixels as noise, the road pavement state determination method. In the road pavement state determination device,
a communication unit for receiving an image of a road pavement subject to which the type and severity of the defect are determined;
A training image and a characteristic value related to the training image, and a defect classification model trained using machine learning learning data including information on the type and severity of a defect corresponding to the training image, a storage unit for storing the road pavement image ;
Obtaining a defect image from the road pavement image, grouping the connected defects in the defect image, expanding the shape of the defect outwardly to merge the overlapping defects to extract the defect group, and the defect group corresponding to the extracted defect group By inputting the defect group image and characteristic values into the defect classification model, the type and severity of defects included in the defect image are calculated, the defects connected in the defect image are grouped, and the shape of the defect is expanded outward. a control unit generating the defect group by merging overlapping defects; and
and an input/output unit for outputting data on the type and severity of the defect calculated by the control unit;
The control unit is
Defect pixels connected in the defect image are grouped to form a pixel group, the shape of each pixel group is expanded to the outside to create a defect group, the characteristic value of the defect group is calculated, and a corresponding defect group is formed. By inputting the defect group image and characteristic value into the defect classification model, the defect type and severity for each defect group included in the defect image are calculated,
The control unit is
The defect group image is generated by cropping the defect image in the form of a minimum rectangle including polygons of the defect group, and characteristic values for defects in the defect group image are calculated using the characteristic values of the line segments calculated in advance. , a road pavement condition determination device. 8. The method of claim 7,
The control unit is
A device for determining the state of a road pavement that binarizes pixels included in the defective image, groups adjacent defective pixels to form a pixel group, and removes a pixel group less than or equal to a set number of pixels as noise. 8. The method of claim 7,
The control unit is
Separate line segments within each pixel group, calculate the characteristic values of each line segment, expand the shape of each pixel group to the outside to compose polygons, merge overlapping polygons, and polygons corresponding to preset criteria After removing as noise, each polygon is set as one defect group,
A road pavement condition in which lines included in each pixel group are thinned to have a certain width, nodal points where the thinned lines intersect or are separated are recognized, and lines are divided based on the nodal points to separate each line segment. discrimination device. 10. The method of claim 9,
The control unit is
A road pavement state determination device for calculating a characteristic value for at least one characteristic of a length, width, and direction between a starting point and an ending point of each separated line segment, and removing a line segment having the characteristic value equal to or less than a preset value as noise. 11. The method of claim 10,
The control unit is
Based on the thinned line segment, the length between the starting point and the end point is calculated, the width is calculated from the original shape before the thinning of the line segment, and based on the angle between the starting point and the ending point of the line segment based on the thinned line segment. A road pavement condition determination device that calculates a direction. 10. The method of claim 9,
The control unit is
If any one of the number of line segments, the number of nodal points, or the area included in the polygon is less than or equal to a preset standard, the corresponding polygon and the corresponding defective pixels are removed as noise. A computer-readable recording medium in which a program for performing the method according to claim 1 is recorded. A computer program stored in a medium for performing the method according to claim 1, which is performed by the device for determining the state of the road pavement.

Images