JP2023034740A - Face evaluation supporting apparatus and face evaluation supporting method - Google Patents

Abstract

To provide a face evaluation supporting apparatus capable of improving evaluation quality in face evaluation.SOLUTION: A face evaluation supporting apparatus 4 is provided with an image data acquisition unit 11 for acquiring image data in which a target region including a tunnel face is photographed, a point cloud data acquisition unit 12 for acquiring point cloud data in which a three-dimensional shape is measured, a face evaluation unit 13 for evaluating the face based on the image data and the point cloud data, and a support information output unit 14 for outputting support information based on an evaluation result of the face evaluation unit 13. The face evaluation unit 13 is provided with a face extraction unit 130 for extracting a face region corresponding to the face from the target region, a roughness quantification unit 131 for quantifying the roughness of the face surface in the face region as a roughness feature amount based on the point cloud data, an area division unit 132 for dividing the face region into divided areas based on the distribution of representative values of the roughness feature amount, and a divided area evaluation processing unit 133 for evaluating an observation item related to the face as an evaluation result for each divided area.SELECTED DRAWING: Figure 4

Description

The present invention relates to a face evaluation support device and a face evaluation support method.

In order to ensure the safety and workability of tunnel construction, it is required to properly evaluate the condition of the tunnel face. In the past, professional engineers visually observed the face and recorded sketches of the observation results. A face evaluation system is being developed that uses image data (photographs) and point cloud data obtained by measuring the three-dimensional shape of the face.

For example, Patent Literature 1 discloses a rock mass weathering alteration evaluation apparatus that derives a weathering alteration evaluation category based on a captured image of a face rock. Further, Patent Literature 2 discloses a crack evaluation device that evaluates cracks developing in rock based on point cloud data obtained by measuring three-dimensional information of the face of a tunnel.

JP 2019-184541 A JP 2018-181271 A

In the devices disclosed in Patent Documents 1 and 2, the face is evaluated using the image data and the point cloud data, respectively. There are limits to improving quality. For example, since the image data includes images of supports, roadbeds, etc. in addition to the face, it is necessary to perform processing to separate the face region from the non-face region on the image data, and this processing is unnecessary. In sufficient cases, it can degrade the evaluation quality of the face. In addition, the face may include multiple rocks with different characteristics, and if the various observation items are evaluated without classifying these rocks, the condition of the face may not be evaluated appropriately. .

SUMMARY OF THE INVENTION It is an object of the present invention to provide a face evaluation support device and a face evaluation support method capable of improving evaluation quality when evaluating a face. .

In order to achieve the above object, a face evaluation support device according to an aspect of the present invention includes:
an image data acquisition unit that acquires image data in which a target region including a face of a tunnel is photographed;
a point cloud data acquisition unit that acquires point cloud data in which the three-dimensional shape of the target region is measured;
a face evaluation unit that evaluates the face based on the image data and the point cloud data;
a support information output unit that outputs support information based on the evaluation result of the face evaluation unit;
The face evaluation unit is
a face extraction unit that extracts a face region corresponding to the face from the target region based on at least one of the image data and the point cloud data;
a roughness quantifying unit that quantifies the roughness of the face surface in the face region as a roughness feature amount along a plurality of scanning lines set in a plurality of scanning directions with respect to the face region, based on the point cloud data; ,
an area division unit that obtains a representative value of the roughness feature amount for each intersection point where the plurality of scanning lines intersect, and divides the face area into division areas based on the distribution of the representative value for each intersection point;
and a segmented region evaluation processing unit that evaluates observation items related to the face as the evaluation result for each segmented region obtained by segmenting the face region by the region segmentation unit.

According to the face evaluation support device according to one aspect of the present invention, the face extraction unit extracts the face region corresponding to the face based on at least one of the image data and the point cloud data, and the roughness quantification unit converts the point Based on the group data, the roughness of the face surface in the face region is quantified as a roughness feature quantity, the region dividing unit divides the face region into segmented regions based on the distribution of the roughness feature quantity, and the segmented region evaluation processing unit , the observation items related to the face are evaluated for each segmented area. Therefore, the region corresponding to the face is divided into segmented regions, and the observation items related to the face are evaluated for each segmented region. Therefore, it is possible to improve the evaluation quality when evaluating the face.

Problems, configurations, and effects other than the above will be clarified in the mode for carrying out the invention, which will be described later.

1 is an overall view showing an example of a face evaluation support system 1; FIG. It is a figure which shows an example of the data processed by the face evaluation support system 1, and shows (a) image data and (b) point cloud data. 2 is a block diagram showing an example of a face evaluation support device 4; FIG. 4 is a functional explanatory diagram showing an example of a face evaluation support device 4; FIG. 4 is a flowchart showing an example of color correction processing by a color correction unit 110; FIG. 4 is an explanatory diagram showing an example of a first face extraction process for extracting a face region 101 from point cloud data; FIG. 9 is an explanatory diagram showing an example of a second face extraction process for extracting a face region 101 from image data; FIG. 7 is an explanatory diagram showing an example of roughness quantification processing by a roughness quantification unit 131; FIG. 11 is an explanatory diagram showing an example of area segmentation processing by an area segmentation unit 132; FIG. 11 is an explanatory diagram showing an example of processing for determining a predominant direction of a crack by a predominant direction of a crack determination unit 133A; FIG. 4 is an explanatory diagram showing an example of rock group determination processing by a rock group determination unit 133B; FIG. 13 is an explanatory diagram showing an example of crack interval determination processing by a crack interval determination unit 133C; FIG. 11 is an explanatory diagram showing an example of a crack form determination process by a crack form determination unit 133D; FIG. 10 is an explanatory diagram showing an example of inference processing of an evaluation classification regarding an observation item by an evaluation classification inference unit 133E; FIG. 11 is an explanatory diagram showing an example of continuity break determination processing by the first continuity break detection unit 134A; FIG. 11 is an explanatory diagram showing an example of continuity break determination processing by a second continuity break detection unit 134B; FIG. 13 is an explanatory diagram showing an example of continuity break determination processing by a third continuity break detection unit 134C; FIG. 13 is an explanatory diagram showing an example of an artifact detection process by an artifact detection unit 134D; FIG. 10 is an explanatory diagram showing an example of prediction processing of a risky skin-spotting area; FIG. 10 is a diagram showing a face observation sheet 150 that is a printed output example of face observation sheet information 15; FIG. 10 is a diagram showing a face sketch screen 160 as a screen display example of face sketch information 16. FIG.

Embodiments for carrying out the present invention will be described below with reference to the drawings. In the following, the range necessary for the description to achieve the object of the present invention is schematically shown, and the range necessary for the description of the relevant part of the present invention is mainly described. It shall be by technology.

FIG. 1 is an overall view showing an example of a face evaluation support system 1. As shown in FIG. FIG. 2 shows an example of data processed by the face evaluation support system 1, showing (a) image data and (b) point cloud data.

The face evaluation support system 1 is a system used to observe and evaluate the state of the face F at the tunnel site. The face evaluation support system 1 includes an image capturing device 2, a three-dimensional shape measuring device 3, a face evaluation support device 4, an operator terminal device 5, and an administrator terminal device 6 as its main components. . Each device 2 to 6 is connected to a wired or wireless network 7 and configured to be able to mutually transmit and receive various data. Note that the number of the devices 2 to 6 may be plural, and the configuration of the network 7 is not limited to the example of FIG.

The image capturing device 2 is a device that captures an image of a target area 100 including a tunnel face F, and is composed of, for example, a digital camera having a color image sensor. For example, the image capturing device 2 is operated by an on-site worker to capture an image of the target region 100 under predetermined image capturing conditions (image capturing position, installation position of the color sample 20, lighting, etc.). Color image data is generated in which pixel values for the three components of RGB are respectively recorded. The color image data is sent to the face evaluation support device 4 via the network 7 or a recording medium.

The image capturing device 2 is used by being installed on a roadbed with a tripod or the like. It may be remotely operated. Further, the image capturing device 2 may generate grayscale image data instead of color image data. In this embodiment, the image capturing device 2 generates color image data, and a color correction unit 110 described later performs color correction. Hereinafter, the color image data is abbreviated as “image data”. .

In the image data, not only the face F but also the support and the roadbed are photographed. Further, when the image data is captured, as shown in FIG. 2A, the color sample 20 is also captured in the image data because the color sample 20 is placed in front of the face F. .

The color sample 20 has a predetermined black-and-white pattern composed of two colors, black and white. For example, three rectangular areas (with an area of to the same extent). The color sample 20 adopts a point-symmetrical or line-symmetrical pattern so that the pattern remains the same even when the top and bottom are reversed. In addition, since the color sample 20 is a black-and-white pattern only, for example, the pattern of the black portion is printed on blank copy paper or photographic paper (A3, B4, A4 size, etc.) by a printer (even if it does not support color printing). It can be easily produced by pasting it on a board such as a blackboard, a white board, or cardboard.

The three-dimensional shape measuring device 3 is a device that measures the three-dimensional shape of the target region 100 including the tunnel face F, and is composed of, for example, a distance sensor using laser or ultrasonic waves, a stereo camera, or the like. By measuring the target region 100 under predetermined measurement conditions (measurement positions, etc.), the three-dimensional shape measuring device 3 measures the distance (depth ) are recorded to generate point cloud data. The point cloud data is sent to the face evaluation support device 4 via the network 7, a recording medium, or the like.

The three-dimensional shape measuring device 3 may use either an active method or a passive method as a method for measuring a three-dimensional shape. The image capturing device 2 and the three-dimensional shape measuring device 3 may be configured integrally, for example, configured by an RGB-D (Red Green Blue Depth) camera capable of generating both image data and point cloud data. may be

In the point cloud data, the three-dimensional shape of the target region 100 is recorded by the distance to the face F with the axial direction of the tunnel as the z direction, as shown in FIG. 2(b). At that time, it is not necessary to install the color sample 20 . It should be noted that the imaging range by the image data and the measurement range by the point cloud data are preferably similar ranges like the target region 100, but both ranges include at least the entire face F. The target region 100 A different range may be used as long as it corresponds to . In addition, although it is preferable that the image data capturing time and the point cloud data measuring time are basically in the same time zone, there may be a time difference within a range in which the state of the face F does not change. .

The face evaluation support device 4 is composed of a general-purpose or dedicated computer, and for example, a stationary computer or a portable computer is used. The face evaluation support device 4 acquires image data and point cloud data from the image capturing device 2 and the three-dimensional shape measurement device 3, respectively, and performs various processes on the image data and point cloud data to obtain the face F Evaluate various observations related to The face evaluation support device 4 generates support information based on the evaluation result, and provides the support information to, for example, the worker terminal device 5, the manager terminal device 6, and the like.

The face evaluation support device 4 also includes a face management database 400 that associates and stores image data, point cloud data, and support information. In the face management database 400, a plurality of data regarding the face F are registered, and attribute information such as an identifier (ID), site name, site position, observation date and time, etc. are added, and managed in an identifiable manner. be.

The operator terminal device 5 is used, for example, by an on-site operator. The worker terminal device 5 is composed of a general-purpose or dedicated computer, and for example, a portable computer such as a tablet terminal or a smart phone is used. The operator terminal device 5 receives various input operations, and outputs various information via display screens and voices by software such as applications and web browsers. In addition, the worker terminal device 5 transmits and receives various data to and from the face evaluation support device 4 and the like via the network 7 .

The administrator terminal device 6 is used, for example, by an on-site administrator (who may be the same person as the worker). The administrator terminal device 6 is composed of a general-purpose or dedicated computer, and for example, a stationary computer or a portable computer is used. The administrator terminal device 6 receives various input operations, and outputs various information via display screens and voices by software such as applications and web browsers. In addition, the administrator terminal device 6 transmits and receives various data to and from the face evaluation support device 4 and the like via the network 7 .

FIG. 3 is a block diagram showing an example of the face evaluation support device 4. As shown in FIG. The face evaluation support device 4 includes a storage unit 40 configured by HDD, SSD, memory, etc., a control unit 41 configured by processors such as CPU, GPU, MPU, etc., and an input unit 42 configured by keyboard, mouse, etc. , a display unit 43 configured by a display or the like, a communication unit 44 that is an interface with the network 7 based on a predetermined communication standard (either wired or wireless), and external devices such as printers, scanners, and USB memories. and an external device interface (I/F) unit 45, which is an interface with the device, and a media input/output unit 46, which is an interface with storage media such as CDs and DVDs. Note that the input unit 42 and the display unit 43 may be configured by a shared device such as a touch panel.

The storage unit 40 stores an operating system (OS) that is a basic program, a face evaluation support program 10 that controls the operation of the face evaluation support device 4, various data used in the face evaluation support program 10 (face management database 400, etc.). ), spreadsheet software for executing spreadsheet processing, document creation software for executing document creation processing, web browser software, and the like are stored.

Although the face evaluation support program 10 and various data used in the face evaluation support program 10 are basically stored in the storage unit 40, these programs and data are stored in the communication unit 44 or externally. It may be acquired from an external storage device via the device I/F unit 45 or may be acquired from a storage medium via the media input/output unit 46 . Various data used in the face evaluation support program 10 are preferably created in a format that can be displayed and edited by spreadsheet software, document creation software, and web browser software.

The control unit 41 functions as an image data acquisition unit 11 , a point cloud data acquisition unit 12 , a face evaluation unit 13 , and a support information output unit 14 by executing the face evaluation support program 10 .

FIG. 4 is a functional explanatory diagram showing an example of the face evaluation support device 4. As shown in FIG. The face evaluation support device 4 operates as a subject that implements the face evaluation support method, and the processing contents performed by each part of the face evaluation support device 4 correspond to each step of the face evaluation support method.

The image data acquisition unit 11 acquires image data in which the target region 100 including the face F of the tunnel is captured. The image data acquisition unit 11 acquires image data from the image capturing device 2 via, for example, the communication unit 44, the external device I/F unit 45, the media input/output unit 46, and the like. The image data acquisition unit 11 also includes a color correction unit 110 that performs color correction on the color image data. The image data may be captured in the past and stored in the face management database 400 . In this case, the image data acquisition unit 11 may refer to the face management database 400 .

The point cloud data acquisition unit 12 acquires point cloud data in which the three-dimensional shape of the target region 100 including the tunnel face F is measured. The point cloud data acquisition unit 12 acquires point cloud data from the three-dimensional shape measuring device 3 via, for example, the communication unit 44, the external device I/F unit 45, the media input/output unit 46, and the like. Note that the point cloud data may have been measured in the past and stored in the face management database 400 , in which case the point cloud data acquisition unit 12 may refer to the face management database 400 .

The face evaluation unit 13 evaluates the face F based on the image data acquired by the image data acquisition unit 11 and the point cloud data acquired by the point cloud data acquisition unit 12 . The image data used by the face evaluation unit 13 is image data after color correction has been performed by the color correction unit 110 .

The face evaluation unit 13 includes a face extraction unit 130 , a roughness quantification unit 131 , an area partitioning unit 132 , a partitioned area evaluation processing unit 133 , and a face area evaluation processing unit 134 . Details of each part will be described later.

The segmented area evaluation processing unit 133 includes a fracture dominant direction determination unit 133A, a rock group determination unit 133B, a fracture interval determination unit 133C, a fracture morphology determination unit 133D, and an evaluation division inference unit 133E. Note that any one of the units 133A to 133E may be omitted.

The face area evaluation processing unit 134 includes a first continuity break detection unit 134A, a second continuity break detection unit 134B, a third continuity break detection unit 134C, an artifact detection unit 134D, and a first continuity break detection unit 134D. 134E, a second skin-spotting risk point prediction unit 134F, and a skin-spotting risk point evaluation unit 134G. Note that any one of the units 134A to 134G may be omitted.

The support information output unit 14 outputs support information based on the evaluation result of the face F by the face evaluation unit 13 . The support information is output in a format that can be handled by arbitrary software such as applications and web browsers. Further, output destinations of the support information include, for example, the face management database 400, the worker terminal device 5, the manager terminal device 6, external devices (printer, USB memory), and the like. The support information output unit 14 may output support information in response to the image data and the point cloud data being acquired by the image data acquisition unit 11 and the point cloud data acquisition unit 12. The support information may be output in response to a request from the terminal device 5, administrator terminal device 6, or the like. Also, the support information stored in the face management database 400 may be output in response to a request from the operator terminal device 5, the manager terminal device 6, or the like.

The content of the support information includes, for example, face observation sheet information 15 in which evaluation results are recorded in accordance with a predetermined format, and face sketch information 16 in which evaluation results are superimposed and displayed on a display screen that displays image data. be done.

Details of each part of the face evaluation support device 4 (each process of the face evaluation support method) and evaluation results of the face F by the face evaluation support device 4 will be described below with reference to FIGS. 5 to 21 .

(1) Color correction processing by the color correction unit 110 The color correction unit 110 detects the target area 100 including the face F and the color sample 20 having a predetermined black and white pattern as image data by color. When the photographed color image data is acquired, color correction is performed on the color image data.

FIG. 5 is a flowchart showing an example of color correction processing by the color correction unit 110. As shown in FIG.

First, in step S101, the color tone of the entire color image is corrected using the white and black pixels of the color sample 20 included in the color image data.

Next, in step S102, the color image data is converted into a grayscale image, and the brightness of the pixel located in the middle when all pixels constituting the grayscale image are arranged in order of brightness is taken as a brightness intermediate value. demand.

Next, in step S103, all pixels are divided into a high-brightness pixel group consisting of pixels having brightness higher than the brightness intermediate value and a low-brightness pixel group consisting of an image having brightness equal to or lower than the brightness intermediate value. Classify.

Next, in step S104, the grayscale image is subjected to binarization processing using, for example, a minimum threshold value of 110 to a maximum value of 253, and a white frame around the outer circumference of the color sample 20 is detected. Then, in order to remove noise from the grayscale image, expansion processing is performed to combine interrupted line segments, and contour detection processing is performed. The position of the color sample 20 is detected based on the size, shape, area ratio, etc. of the contour obtained as a result.

Next, in step S105, the average values of the three components of RGB are obtained for the pixel group corresponding to the white portion of the color sample 20, and the maximum value among the average values is selected. Then, each difference value between the maximum value and each average value is determined as a correction value for the high brightness pixel group. For example, when the average values of RGB of the pixel group corresponding to the white portion are obtained as "230, 216, 231", "231" is selected as the maximum value, and the correction value for the high brightness pixel group is "1 , 15,0”.

Next, in step S106, the high-lightness pixel group is color-corrected by adding the correction values for the high-lightness pixel group determined in step S105 for each of the three components to each pixel of the high-lightness pixel group. For example, if one specific pixel belonging to the high-brightness pixel group is "123, 234, 135", the pixel after correction with the above correction values is "124, 249, 135".

Next, in step S107, the average values of the three components are obtained for the pixel group corresponding to the black portion of the color sample 20, and the minimum value among the average values is selected. Then, each difference value between the minimum value and each average value is determined as a correction value for the low-brightness pixel group. For example, if the RGB average values of the pixel group corresponding to the black portion are obtained as "24, 4, 6", "4" is selected as the minimum value, and the correction value for the low brightness pixel group is "20 , 0, 2”.

Next, in step S108, the low-lightness pixel group is color-corrected by subtracting the correction value for the low-lightness pixel group determined in step S107 for each of the three components from each pixel of the low-lightness pixel group. For example, if one specific pixel belonging to the low-lightness pixel group is "43, 30, 35", the pixel after correction with the above correction values is "23, 30, 33".

As described above, the color correction unit 110 performs a series of color correction processes shown in FIG. Even if various conditions such as settings are different, the face F can be appropriately evaluated based on the corrected image data.

(2) Face Extraction Processing by Face Extractor 130 The face extractor 130 extracts the face region 101 corresponding to the face F from the target region 100 based on at least one of image data and point cloud data. As a method for extracting the face region 101 from the target region 100, there are a first face extraction process of analyzing and extracting point cloud data, a second face extraction process of extracting from image data using a machine learning model, A third face extraction process using both the first face extraction process and the first face extraction process may be mentioned.

FIG. 6 is an explanatory diagram showing an example of the first face extraction process for extracting the face region 101 from the point cloud data. In the first face extraction process, as shown in FIG. 6A, when the point cloud data is measured in a coordinate system with the tunnel axis as the z-axis (coordinate values decrease toward the face side), The face extraction unit 130 extracts the face region 101 based on the z-axis coordinate value of each point. For example, the traveling distance L that the face surface advances in one blasting for tunnel construction is about 1.5 m with respect to the z-axis, and the unevenness of the face surface with respect to the z-axis is less than the traveling distance L. In addition, the support is constructed at a position that is more than the travel distance L with respect to the face surface.

Therefore, as shown in FIG. 6B, the face extraction unit 130 extracts the travel distance L The range up to is extracted as the face region 101 . In addition, as shown in FIG. 6(c), the face extracting unit 130 captures a range equal to or greater than the coordinate value obtained by adding the travel distance L to the minimum value of the z-axis of the point cloud data as an area of the support and roadbed portion. A face region 101 is extracted by performing mask processing on the image data acquired by the data acquisition unit 11 .

As another face extraction process using the point cloud data, the face extraction unit 130 extracts each normal vector of the triangular mesh formed from the point cloud data so that the component along the z-axis is dominant in the face region 101. However, the face region 101 may be extracted by utilizing the characteristic that the component perpendicular to the z-axis component is dominant in the support and roadbed.

FIG. 7 is an explanatory diagram showing an example of the second face extraction process for extracting the face region 101 from the image data. The machine learning model used in the second face extraction process is composed of, for example, a convolutional neural network (CNN), and is applied to the learning image data in which the face is captured and the face region 101 corresponding to the face F. Machine learning was performed using teacher data including correct labels and That is, as shown in FIG. 7, in the learning phase of machine learning, the extraction result of the face region 101 output by inputting the learning image data of the teacher data into the machine learning model and the correct label of the teacher data are combined. Machine learning is performed by comparing and adjusting the weighting parameters of the convolutional neural network based on the results of the comparison. As shown in FIG. 7, the face extraction unit 130 may extract not only the face region 101 but also the support region corresponding to the support and the roadbed region corresponding to the roadbed portion.

The face extraction unit 130 is a main body that executes the inference phase of machine learning, and inputs the image data acquired by the image data acquisition unit 11 to a trained machine learning model, thereby extracting the face region 101 from the target region 100. to extract

As described above, the face region 101 is extracted from the target region 100 by the face extraction processing by the face extraction unit 130 .

(3) Roughness quantifying process by roughness quantifying unit 131 The roughness quantifying unit 131 performs a plurality of scanning directions set in a plurality of scanning directions on the face region 101 extracted by the face extracting unit 130 based on the point cloud data. The roughness (unevenness) of the face surface in the face region 101 is quantified as a roughness feature amount along the scanning line of . The roughness quantification unit 131 quantifies amplitude values and wavelengths of a roughness approximation model when point cloud data corresponding to the face region 101 is approximated by a periodic function along a plurality of scanning lines as roughness feature quantities.

FIG. 8 is an explanatory diagram showing an example of roughness quantification processing by the roughness quantification unit 131. As shown in FIG. The roughness feature amount is a parameter that quantifies the roughness of the face in the z-axis direction based on the point cloud data. As shown in FIG. 8A, the roughness is set in a plurality of scanning directions, for example, in the vertical and horizontal directions (x-axis, y-axis) on the face surface and in the oblique direction, respectively, at predetermined intervals d1. , d2 along a plurality of scan lines i.

The roughness approximated along the scanning line i is set from the z-axis coordinate values in the point cloud data positioned within a certain distance a from the scanning line i, as shown in FIG. 8(b). The height difference h _i (see FIG. 8C) from the reference line (surface) on the scanning line i is the coordinate value of the point group _pj included in the point cloud data and the coordinate value from the scanning line i to the point group _pj (see FIG. 8(b)) and the distance r (see FIG. 8(b)). Here, pj _,z are the z-axis coordinate values of the point group _pj , and _z0 is the value of the reference line (plane).

The roughness of the face surface is approximated by a roughness approximation model by paying attention to the magnitude of the height difference and the periodicity of the height for each scanning line i. In the roughness approximation model, the height difference H(t) _i from the reference plane at the position t of the scanning line i is modeled (approximated) by the following equation (2) using a periodic function.

Here, the wavenumber _kj is a value corresponding to the reciprocal of the wavelength _λj , which is the repeat distance of the height difference. The coefficients A _j and B _j are feature amounts, and are obtained by using, for example, the least squares method and sparse modeling for the wave number k _j . The above-mentioned roughness approximation model is a modeling that includes characterization by an average value of height difference by superposing j periodic functions. On the other hand, the wave number _kj may be set using the crack interval (frequency), which is an observation item of the face F.

The roughness quantifying unit 131 quantifies, for example, the amplitude value and wavelength in the roughness approximation model as the roughness feature amount for each scanning line i. The amplitude value C _j of the roughness approximation model is obtained from the coefficients A _j and B _j by the following [Equation 3]. Also, the wavelength λ _j of the roughness approximation model is determined by the wavenumber k _j (=1/λ _j ) at which the amplitude value C _j reaches its maximum value.

As described above, the roughness quantification processing by the roughness quantification unit 131 quantifies the roughness of the face surface in the face region 101 as the roughness feature amount.

(4) Region segmentation processing by region segmentation unit 132 The region segmentation unit 132 obtains a representative value of the roughness feature amount digitized by the roughness quantification unit 131 for each intersection point where a plurality of scanning lines intersect, and The face region 101 extracted by the face extraction unit 130 is divided into division regions 102 based on the distribution of the representative values for each.

FIG. 9 is an explanatory diagram showing an example of area partitioning processing by the area partitioning unit 132. As shown in FIG. When dividing the face region 101 into the divided regions 102, the region division unit 132 obtains a representative value of the roughness feature amount at each intersection point P where a plurality of scanning lines intersect. For example, as shown in FIG. 9(a), the amplitude value C and the wavelength λ are quantified as roughness feature quantities at intersection points Pi _{, j, k, and l} where four vertical and oblique scanning lines intersect. In this case, the representative value of the roughness feature is obtained from the average (simple average or weighted average), vector sum, or the like of the roughness feature amounts of the four scanning lines, that is, the amplitude value C and the wavelength λ.

Next, as shown in FIG. 9B, the area partitioning unit 132 sets a representative value (amplitude value C and wavelength λ) for each intersection point P, a reference value C 0 of the amplitude value C, and a reference value C ₀ of the wavelength λ. By comparing the value λ ₀ , it is determined which of the four feature sections Ra, Rb, Rc, and Rd each intersection point P falls into. By setting a plurality of reference values for the amplitude value C and setting a plurality of reference values for the wavelength λ, the feature section is classified into feature sections other than the above. good too.

Then, the area division unit 132 divides the face area 101 into division areas 102 based on the distribution of the characteristic divisions Ra, Rb, Rc, and Rd into which the intersection points P are classified. FIG. 9(c) shows, for example, a segmented region 102 (upper left part) that is a set of intersections P classified into the feature segment Ra and a segmented region 102 (a set of intersections P that are classified into the feature segment Rd). A case is shown in which the area is divided into two divided areas 102 called the lower right portion).

As described above, the face region 101 is segmented into the segmented regions 102 based on the distribution of the roughness feature amount by the region segmentation processing by the region segmentation unit 132 .

(5) Divisional area evaluation processing by the divisional area evaluation processing unit 133 The divisional area evaluation processing unit 133 evaluates observation items related to the face F as an evaluation result for each divisional area 102 obtained by dividing the face area 101 by the divisional area division unit 132. do. The segmented region evaluation processing unit 133 evaluates various observation items for each segmented region 102 based on the image data and point cloud data corresponding to the segmented region 102. As shown in FIG. 133A, a rock group determination unit 133B, a fracture interval determination unit 133C, a fracture morphology determination unit 133D, and an evaluation classification inference unit 133E. Details of each section (each process) 133A to 133E will be described below.

(5-1) Crack predominant direction determination unit 133A
FIG. 10 is an explanatory diagram showing an example of processing for determining the predominant direction of cracks by the predominant direction of crack determination unit 133A. The crack predominant direction determination unit 133A polar-projects each normal vector of the triangular network composed of the point cloud data corresponding to the face region 101 onto the stereo net for each segmented region 102, and plots the polar-projected directions on the stereo net. Based on the distribution of , the predominant direction of cracks, which indicates the strike inclination of cracks predominant in the face, is determined for each segmented region 102 .

On the face, rock masses are often separated and excavated along the existing cracks, so there is a high possibility that the unevenness of the face will appear along the cracks. Therefore, in order to determine the predominant direction of cracks in each segmented region 102, the predominant direction of cracks determination unit 133A uses a triangular mesh composed of point cloud data corresponding to the segmented region 102, as shown in FIG. 10(a). Then, as shown in FIGS. 10(b) and 10(c), the normal vector of each triangle included in the triangular network is polar projected onto a stereo net (equal area projection net). In the polar projection onto the stereo net, each normal vector is projected in a direction coinciding with the tunnel axis (z-axis), with the base end of the arrow aligned with the center of the stereo net.

Then, as shown in FIG. 10(c), the crack predominant direction determination unit 133A creates a density contour diagram based on the plot distribution corresponding to the tip of the arrow of each normal vector polar-projected on the stereo net. Then, in the density contour map, the range where the density is higher than a predetermined reference value is determined as the predominant direction of cracks. In this case, the crack predominant direction determination unit 133A determines the crack predominant direction for each divided area, but it may be determined that one divided area includes a plurality of dominant crack directions.

If the point cloud data is irregularly distributed on the face, it is assumed that the number of plots polar-projected on the stereonet does not correspond to the area of the triangle on the face. . In that case, the crack predominant direction determining unit 133A weights the areas of the triangles that make up the triangular network and plots them on the stereo net to determine the predominant direction of the cracks while reflecting the area on the face surface. You may make it judge.

(5-2) Rock group determination unit 133B
FIG. 11 is an explanatory diagram showing an example of rock group determination processing by the rock group determination unit 133B. The rock group determination unit 133B determines a rock group indicating the type of rock forming the face surface for each divided region 102 based on the amplitude value and wavelength of the roughness approximation model quantified by the roughness quantification unit 131.

The rock group is classified into a plurality of rock types by combining, for example, the weathering category of either massive or layered, and the strength category of hard, medium-hard, or soft. For example, in the rock group Ga of hard rock/mass (for example, granite), the amplitude value indicating the unevenness of cracks tends to be large, and the wavelength indicating the interval between cracks tends to be medium to long. Soft rocks and bedding (e.g., Tertiary shale, phyllite, etc.) rock group Ge are finely bedding and schistosity, so the amplitude value is small due to unevenness along the bedding and schistosity. and the wavelength becomes smaller. In the rock group Gc, which is soft rock/massive rock (for example, tuff breccia with low degree of consolidation), since the face surface by excavation is smooth, there is almost no unevenness, the amplitude value is small, and the wavelength is long. . The rock group determination unit 133B determines the rock group of each segmented region 102 based on the amplitude value and wavelength at each intersection point P, for example, by focusing on the tendency and characteristics of each rock group as described above. Note that the rock group determining unit 133B determines the rock group of each segmented region 102 by referring not only to the roughness feature amount but also to the evaluation segments for other observation items (for example, weathering alteration, compressive strength, etc.). You may do so.

(5-3) Crack interval determination unit 133C
FIG. 12 is an explanatory diagram showing an example of crack interval determination processing by the crack interval determination unit 133C. The crack interval determination unit 133C performs edge detection processing for detecting edges on the image data corresponding to the segmented region 102, and determines the relationship between the edge density and the crack interval based on the edge density of the edges detected by the edge detection processing. The crack interval is determined for each segmented region 102 by referring to the correspondence information in which the relationship is defined.

The crack interval determination unit 133C detects edges from the image data by arbitrary edge detection processing such as the Canny method, and calculates the edge density. Then, the crack interval determination unit 133C refers to the correspondence information for the relationship between the edge density and the crack interval determined for each rock type, for example, and the rock group correspondence relationship determined by the rock group determination unit 133B. In the information, the crack spacing is determined by specifying the crack spacing associated with the edge density calculated as described above.

It should be noted that although a plurality of pieces of correspondence information have been defined for each type of rock, the present invention is not limited to this. Anything is fine.

(5-4) Crack morphology determination unit 133D
FIG. 13 is an explanatory diagram showing an example of the crack shape determination processing by the crack shape determination unit 133D. The crack morphology determining unit 133D determines the crack morphology for each divided region 102 based on the wavelength characteristics according to the scanning direction of the scanning line when the roughness quantification unit 131 approximates the roughness feature quantity to the roughness approximation model. .

The roughness feature quantity quantified along the scanning line by the roughness quantification unit 131 can be used for estimating the crack morphology. In the scanning lines set in the vertical, horizontal and oblique directions, if the wavelength λ, which is the repeated distance of the height difference, is approximately constant regardless of the scanning line direction (within a predetermined reference value range), a crack The morphology determination unit 133D determines that the crack morphology of the partitioned area is "random square" or "earth and sand". On the other hand, when the wavelength λ differs greatly depending on the direction of the scanning line (outside the range of the predetermined reference value), the crack morphology determination unit 133D determines that the crack morphology of the partitioned region is “columnar” or “layered”. do.

(5-5) Evaluation division reasoning unit 133E
FIG. 14 is an explanatory diagram showing an example of the evaluation classification inference process for the observation item by the evaluation classification inference unit 133E. The evaluation segment inference unit 133E infers an evaluation segment for each segmented region 102 by inputting the image data corresponding to the segmented region 102 to the machine learning model. The machine learning model used by the evaluation classification reasoning unit 133E is machine learning based on teacher data including learning image data in which a face is photographed and a correct label to which an evaluation classification related to an observation item is assigned to the face F. It has been done.

Observation items to be inferred include, for example, compressive strength, weathering alteration, fracture interval, fracture morphology, fracture state, rock group, and the like. The evaluation classification inference unit 133E may estimate the evaluation classification for at least one of these observation items. The fracture interval, fracture morphology, fracture state, and rock group observation items can be determined by the above-described fracture interval determination unit 133C, fracture morphology determination unit 133D, and rock group determination unit 133B. The classification inference unit 133E may infer each observation item instead of these, or may infer each observation item in combination with these.

The machine learning model is composed of, for example, a convolutional neural network, and is preferably created for each observation item as shown in FIG. For example, a machine learning model for compressive strength is machine-learned using teacher data including learning image data in which a face is photographed and a correct label to which an evaluation category for compressive strength is assigned to the face F. It is. The evaluation classification inference unit 133E infers an evaluation classification related to compression strength for the partitioned area by inputting the image data corresponding to the partitioned area into the trained machine learning model. At that time, the image data may be input to the machine learning model after being divided into images of small regions having a predetermined size. It is also possible to infer an evaluation category related to compressive strength for each small area. Machine learning models for other observation items are configured in the same manner as described above, so descriptions thereof are omitted.

(6) Face Region Evaluation Processing by Face Region Evaluation Processing Unit 134 The face region evaluation processing unit 134 evaluates observation items related to the face F for the face region 101 extracted by the face extraction unit 130 as evaluation results. In order to evaluate various observation items, the face area evaluation processing unit 134, as shown in FIG. A detection unit 134C, an artificial object detection unit 134D, a first skin-spotting risky point prediction unit 134E, a second skin-spotting risky point prediction unit 134F, and a skin-spotting risk point evaluation unit 134G are provided. Details of each part (each process) 134A to 133G will be described below.

(6-1) First continuity break detector 134A
FIG. 15 is an explanatory diagram showing an example of continuity break determination processing by the first continuity break detector 134A. The first continuity crack detection unit detects continuity cracks continuously formed on the face surface based on the point cloud data. Specifically, the first continuity break detection unit detects the angle formed by the normal vector of two triangles including the common side from each side of the triangular network composed of the point cloud data corresponding to the face region 101. The continuity break existing in the face region 101 is detected by extracting edge candidates forming a part of the continuity break based on and judging the continuity between the candidates.

In the point cloud data, the uneven shape of the face surface is represented by a triangular network, but each side of each triangle represents the uneven edge of the face surface and the dividing line for convenience. there is Therefore, by extracting the sides of the triangle representing the uneven edges of the face and considering the continuity of each of the extracted sides, the continuity cracks are identified.

Determination (edge determination processing) of whether or not the side ab common to two adjacent triangles A and B represents an uneven edge is performed, as shown in FIG. It is done based on the angle to make. For example, if the angle θ formed by the two normal vectors nA and nB is within a predetermined threshold range with respect to 0° or 180°, the triangles A and B are considered to be on the same plane. , the side ab is determined not to be a bumpy edge; otherwise, the side ab is determined to be a bumpy edge. At that time, the angle θ formed by the normal vectors nA and nB may be obtained for determination, or the inner product of the normal vectors nA and nB may be used for determination.

The first continuity break detection unit performs the above-described edge determination processing on all sides shared by two adjacent triangles, and extracts edge candidates forming a part of the continuity break. The continuity between the edge candidates is determined by determining whether or not there is continuity between the extracted edge candidates (continuity determination processing), and connecting the continuous edges as a polygonal line. By doing so, grouping is performed. In the continuity determination process, the two edges to be determined are defined as line segment vectors, and if the angle between the two line segment vectors is equal to or less than a predetermined threshold value, it is determined that they are continuous, and if not, it is determined that they are separated. do. The first continuity break detection unit detects a continuity break by deleting a polygonal line shorter than a predetermined threshold among the grouped polygonal lines and selecting a line with high straightness.

(6-2) Second continuity break detector 134B
FIG. 16 is an explanatory diagram showing an example of continuity break determination processing by the second continuity break detector 134B. The second continuity crack detection unit detects continuity cracks continuously formed on the face surface based on the image data. Specifically, the second continuity break detection unit detects a continuity break present in the face region 101 by inputting image data corresponding to the face region 101 into a machine learning model. The machine learning model used by the second continuity crack detection unit is, for example, configured by a convolutional neural network, and is given to learning image data in which the face is photographed and the continuity crack formed in the face F. Machine learning was performed using teacher data including correct labels and

(6-3) Third continuity break detector 134C
FIG. 17 is an explanatory diagram showing an example of continuity break determination processing by the third continuity break detector 134C. The third continuity crack detection unit detects continuity cracks continuously formed on the face surface based on the image data and the point cloud data. Specifically, the third continuity break detection unit performs edge detection processing for detecting edges on the image data corresponding to the face region 101, and converts the edges detected by the edge detection processing into point cloud data. On the other hand, a continuous crack present in the face region 101 is detected by fitting the predominant direction of the crack for each divided region 102 determined by the predominant direction of crack determination unit 133A.

The third continuity break detection unit detects edges from the image data by arbitrary edge detection processing such as the Canny method. Then, the third continuity break detection unit performs a fitting process of fitting the detected edge to the break dominant direction of each segmented region 102 by, for example, the least squares method or sparse modeling. to detect

(6-4) Artefact detection unit 134D
FIG. 18 is an explanatory diagram showing an example of an artifact detection process by the artifact detection unit 134D. The artifact detection unit 134</b>D detects an artifact present in the target area 100 based on the image data corresponding to the target area 100 . Specifically, the artifact detection unit 134D divides the image data corresponding to the target region 100 into image data corresponding to the face region 101 and images corresponding to regions other than the face region 101 (for example, support regions and roadbed regions). By inputting the image data into the machine learning model respectively, an artifact present in the face region 101 and an artifact present outside the face region 101 are detected. The machine learning model used by the artifact detection unit 134D is a machine learning model based on teacher data including learning image data in which the face is photographed and correct labels given to the face and the artifacts installed around it. Learning is done.

Artifacts to be detected include, for example, steel supports, rock bolts, mirror bolts, pre-construction works, blackboards, people, shadows of people and machinery, and the like. The artifact detection unit 134D may detect at least one of these artifacts. The artifacts to be detected in the face region 101 are, for example, mirror bolts, pre-construction work, blackboards, people, shadows of humans and machinery, etc. Examples of objects include rock bolts, steel supports, blackboards, people, shadows of people and machinery, etc., but are not limited to these examples. FIG. 18 illustrates a case where the artificial object detection unit 134D detects a plurality of pre-construction works in the face region 101 and a plurality of rock bolts in the support region.

The machine learning model is composed of, for example, a convolutional neural network, and includes a machine learning model for detecting artifacts from image data corresponding to the face region 101 and an artifact for detecting artifacts from image data corresponding to regions other than the face region 101. The machine learning model for may be created as a common machine learning model or may be created as separate machine learning models. Also, a machine learning model may be created for each type of artifact to be detected. Furthermore, the image data may be input to the machine learning model in a state of being divided into images of small regions having a predetermined size, and in that case, the artifact detection unit 134D is included in the target region 100. Artificial objects may be detected in units of small regions.

(6-5) First skin-spotting risk location prediction unit 134E
FIG. 19 is an explanatory diagram showing an example of prediction processing of a risky skin-spotting area. Based on the point cloud data, the first skin removal risk location prediction unit 134E predicts a skin removal risk location indicating a location at which there is a risk of skin removal in the face F. Specifically, as shown in FIG. 19, the first skin-spotting risk location prediction unit 134E projects the predominant direction of cracks for each of the divided regions 102 determined by the predominant direction of cracks determination unit 133A onto the stereo net as a great circle. When a closed polygon is formed by the dominant directions of cracks extending in at least three directions, the range of the face corresponding to the closed polygon is predicted as a risk of falling skin.

The first skin-spotting risk location predicting unit 134E geometrically utilizes the dominant crack direction for each segmented region 102 determined by the crack dominant direction determination unit 133A for the point cloud data, to estimate the skin-spotting risk location. Predict. For example, when the split dominant direction is projected as a great circle onto the stereonet and a spherical triangle is formed on the stereonet, the area corresponding to the spherical triangle is predicted to be a block that can escape from the face.

(6-6) Second skin removal risk location prediction unit 134F
The second skin-spotting hazard point prediction unit 134F predicts a skin-spotting hazard point indicating a point where there is a risk of skin-spotting on the face F based on the image data. Specifically, as shown in FIG. By inputting to a machine learning model in which machine learning is performed using teacher data including the correct labels given to the risky skin-slipping locations where there is a risk of skin-sagging, the skin-slipping-risk locations existing in the face region 101 are input to the machine learning model. to predict.

A machine learning model is composed of, for example, a convolutional neural network. The image data may be input to the machine learning model in a state of being divided into images of small regions having a predetermined size. It may be inferred whether or not there is a risk of skin fading for each small area included in .

(6-7) Skin-spotting danger location evaluation unit 134G
As shown in FIG. 19, the skin-spotting risk point evaluation unit 134G, based on the prediction result of the first skin-spotting risk point prediction unit 134E and the prediction result of the second skin-spotting risk point prediction unit 134F, More specifically, the risky skin-spotting evaluation unit 134G predicts the risky skin-spotting location prediction unit 134E and the second risky-skins-spotting prediction. The section 134F compares the areas at risk of skin removal predicted by the section 134F, and as a result of the comparison, it is determined that the risk of skin removal is high for the areas at risk of skin removal with a high degree of matching.

Note that the skin removal risk location evaluation unit 134G may predict the skin removal risk location in consideration of other observation items. For example, if the artificial object detected by the artificial object detection unit 134D, for example, a mirror bolt is detected, the skin removal risk location evaluation unit 134G determines that there is a high risk of skin removal. You may make it judge. Further, when either one of the first hazardous skin-spotting prediction unit 134E and the second skin-staining risky-site prediction unit 134F is omitted, the skin-staining risky-site evaluation unit 134G The skin removal risk location predicted by the evaluation unit may be output as is, or the skin removal risk location evaluation unit 134G itself may be omitted.

(7) Support Information Output Processing by Support Information Output Unit 14 As described above, the support information output unit 14 outputs a face observation sheet as support information based on the evaluation results of the face F for each observation item by the face evaluation unit 13. Information 15, face sketch information 16, and the like are generated and output in an output form such as data storage output, screen display output, or print output.

FIG. 20 is a diagram showing a face observation sheet 150 that is an example of a printed output of the face observation sheet information 15. As shown in FIG. The face observation sheet 150 records the evaluation results of the face F for each observation item in accordance with a predetermined format. In the example of FIG. 20, the face observation sheet 150 includes a face information column 151 in which information about the face F is recorded, and an evaluation classification column 152 in which the evaluation result of the face F is recorded.

The face information column 151 records, for example, the name of the site, the position of the site, the date and time of observation, and the like. In the evaluation category column 152, for example, evaluation categories for compressive strength, weathering deterioration, crack spacing, crack state, strike dip (fracture dominant direction), and the like are recorded.

In the evaluation classification column 152 shown in FIG. 20, the evaluation classification for each area when the face F is divided into three areas of the crest, the left shoulder and the right shoulder is recorded. The face evaluation unit 13 (particularly, the segmented region evaluation processing unit 133) evaluates each segmented region 102 with respect to each observation item and evaluates each segmented region 102, the crown, the left shoulder, and the right shoulder. By weighting and adding the evaluation results for each of the segmented regions 102 based on the area ratio of overlapping of the two regions, the evaluation results for the top, left shoulder, and right shoulder are converted.

Incidentally, the support information output unit 14 replaces or in addition to the face observation sheet information 15 that records the evaluation results for each of the three regions of the crown, left shoulder, and right shoulder, the face that records the evaluation results for each divided region 102 The observation sheet information 15 may be output. In addition, as the format of the face observation sheet information 15, the observation items included in the face observation sheet information 15, the layout of the evaluation results, the output format of the evaluation results (score format, table format, graph format, image superimposition format, etc.), etc. It is not limited to the example of FIG. 20, and may be changed as appropriate. For example, it may be changeable by an administrator or an operator.

FIG. 21 is a diagram showing a face sketch screen 160, which is an example of a screen display of the face sketch information 16. As shown in FIG. The face sketch screen 160 displays evaluation results of the face F for each observation item superimposed on a display screen that displays image data. In the example of FIG. 21, the face sketch screen 160 includes a face information column 161 that displays information about the face F, an image display area 162 that displays image data of the face F, a face area 101, a section area 102, and various observation areas. A display item selection field 163 for selecting an object to be displayed in the image display area 162 from among the items, and a display marker 164 for displaying the evaluation result of the face F in the image display area 162 according to the selection item of the display item selection field 163. Prepare.

The face information column 161 displays, for example, the name of the site, the position of the site, the date and time of observation, and the like. Image data obtained by the image data obtaining unit 11 is displayed in the image display area 162 . The image display area 162 is configured to be able to change the enlargement/reduction of the image, and is configured to be capable of accepting a sketch operation on the screen by the operator or administrator. Note that the image display area 162 may display an image before color correction is performed by the color correction unit 110, or may display an image after color correction is performed.

The display item selection column 163 includes observation item options such as strike/dip (fracture dominant direction), rock group, compressive strength, weathering alteration, fracture interval, fracture morphology, fracture state, continuity fracture, artificial object, and skin removal. It has dangerous parts.

The display marker 164 draws a frame around the peripheries of the face region 101 and the segmented region 102, and displays the evaluation result of each observation item. At that time, the display marker 164 may be displayed so that the evaluation result of each observation item is superimposed on the corresponding position on the image, or may be displayed so that the correspondence can be understood by a lead line or the like. good. Moreover, the display form (color, pattern, etc.) of the display marker 164 is not limited to the example of FIG. 21, and may be changed as appropriate.

As described above, according to the face evaluation support device 4 according to the present embodiment, the face extraction unit 130 extracts the face region 101 corresponding to the face F based on at least one of the image data and the point cloud data, A roughness quantifying unit 131 quantifies the roughness of the face surface in the face region 101 as a roughness feature amount based on the point cloud data, and an area dividing unit 132 divides the face area 101 based on the distribution of the roughness feature amount. It is divided into regions 102, and the divided region evaluation processing unit 133 evaluates the observation items regarding the face F for each divided region 102. FIG. Therefore, the face region 101 corresponding to the face F is divided into segmented regions 102, and the observation items regarding the face F are evaluated for each segmented region 102. FIG. Therefore, when evaluating the face F, the evaluation quality can be improved.

In addition, the sectioned area evaluation processing unit 133 can evaluate each sectioned area 102 for each observation item by including the sections 133A to 133E as described above. Furthermore, the face area evaluation processing unit 134 can evaluate the entire face area 101 for each observation item by including the respective units 134A to 134G as described above.

(Other embodiments)
The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention. All of them are included in the technical idea of the present invention.

In the above embodiment, the face evaluation support program 10 has been described as being stored in the storage unit 40 . On the other hand, the face evaluation support program 10 may be provided by being recorded in a computer-readable storage medium such as a CD-ROM or a DVD as a file in an installable format or an executable format. Further, the face evaluation support program 10 may be provided by being downloaded via the network 7 from an external device. Various machine learning models used by the face evaluation unit 13 may be stored in the storage unit 40 or may be stored in an external device and accessed via the network 7 .

In the above embodiment, a case where a convolutional neural network is adopted as a specific method of machine learning using a machine learning model was explained, but the machine learning model can be any other machine learning method (supervised learning only). (including unsupervised learning and reinforcement learning) may be adopted. Other machine learning methods include, for example, tree types such as decision trees and regression trees, ensemble learning such as bagging and boosting, other neural network types such as recurrent neural networks (including deep learning), and hierarchical types. Clustering, non-hierarchical clustering, k-nearest neighbor method, clustering type such as k-means method, principal component analysis, factor analysis, multivariate analysis such as logistic regression, support vector machine, and the like.

1... face evaluation support system, 2... image capturing device, 3... three-dimensional shape measuring device,
4... Face evaluation support device, 5... Worker terminal device, 6... Administrator terminal device, 7... Network,
10 ... face evaluation support program,
11... Image data acquisition unit, 12... Point cloud data acquisition unit, 13... Face evaluation unit,
14... support information output unit, 15... face observation sheet information, 16... face sketch information,
20 ... color sample,
40... Storage unit, 41... Control unit, 42... Input unit, 43... Display unit, 44... Communication unit,
45... External device I/F section, 46... Media input/output section,
100... Target area, 101... Face area, 102... Division area, 110... Color correction unit,
130... Face extraction unit, 131... Roughness quantification unit, 132... Region division unit,
133... Segmented area evaluation processing unit, 133A... Split dominant direction determination unit,
133B... rock group determination unit, 133C... fracture interval determination unit,
133D ... fracture morphology determination unit, 133E ... evaluation classification inference unit,
134 ... face area evaluation processing unit, 134A ... first continuity crack detection unit,
134B... second continuity break detector, 134C... third continuity break detector,
134D...artificial object detection unit, 134E...first skin removal risk location prediction unit,
134F...Second skin-spotting risk point prediction unit, 134G...Skin-spotting risk point evaluation unit,
150... face observation sheet, 151... face information column, 152... evaluation category column,
160... face sketch screen, 161... face information column, 162... image display area,
163... Display item selection column, 164... Display marker, 400... Face management database

Claims (14)

an image data acquisition unit that acquires image data in which a target region including a face of a tunnel is photographed;
a point cloud data acquisition unit that acquires point cloud data in which the three-dimensional shape of the target region is measured;
a face evaluation unit that evaluates the face based on the image data and the point cloud data;
a support information output unit that outputs support information based on the evaluation result of the face evaluation unit;
The face evaluation unit is
a face extraction unit that extracts a face region corresponding to the face from the target region based on at least one of the image data and the point cloud data;
a roughness quantifying unit that quantifies the roughness of the face surface in the face region as a roughness feature amount along a plurality of scanning lines set in a plurality of scanning directions with respect to the face region, based on the point cloud data; ,
an area division unit that obtains a representative value of the roughness feature amount for each intersection point where the plurality of scanning lines intersect, and divides the face area into division areas based on the distribution of the representative value for each intersection point;
a segmented region evaluation processing unit that evaluates an observation item related to the face as the evaluation result for each segmented region in which the face region is segmented by the region segmentation unit;
Face evaluation support device. The segmented area evaluation processing unit
each normal vector of a triangular network formed from the point cloud data corresponding to the face region is polar projected onto a stereo net for each of the segmented regions;
Based on the distribution of plots polar-projected on the stereo net, a crack predominant direction determining unit for determining, for each of the divided regions, a predominant crack direction indicating the strike and inclination of the crack that is predominant in the face surface,
The face evaluation support device according to claim 1. The roughness quantification unit
quantifying the amplitude value and wavelength of the roughness approximation model when the point cloud data corresponding to the face region is approximated by a periodic function along the plurality of scanning lines as the roughness feature amount,
The segmented area evaluation processing unit
a rock group determination unit that determines a rock group indicating a type of rock forming the face surface for each of the divided areas based on the amplitude value and the wavelength;
The face evaluation support device according to claim 1 or 2. The segmented area evaluation processing unit
performing edge detection processing for detecting edges on the image data corresponding to the segmented area;
Based on the edge density of the edges detected by the edge detection processing, the crack interval is determined for each of the divided regions by referring to correspondence information that defines the relationship between the edge density and the crack interval. A crack interval determination unit is provided,
The face evaluation support device according to any one of claims 1 to 3. The roughness quantification unit
quantifying the wavelength of the roughness approximation model when the point cloud data corresponding to the face region is approximated by a periodic function along the plurality of scanning lines as the roughness feature amount,
The segmented area evaluation processing unit
a crack morphology determination unit that determines a crack morphology for each of the divided regions based on the wavelength characteristics corresponding to the scanning direction of the scanning line when approximated to the roughness approximation model;
The face evaluation support device according to any one of claims 1 to 4. The segmented area evaluation processing unit
Machine learning is performed on the image data corresponding to the segmented region using teacher data including learning image data in which the face is photographed and a correct label to which the face is assigned an evaluation category related to the observation item. an evaluation classification inference unit that infers the evaluation classification for each classification area by inputting to the machine learning model obtained by
The face evaluation support device according to any one of claims 1 to 5. The face evaluation unit is
A first continuity crack detection unit that detects continuity cracks continuously formed on the face surface based on the point cloud data,
The first continuity break detection unit includes:
An edge forming a part of the continuity crack based on an angle formed by normal vectors of two triangles including the common side from each side of the triangular network composed of the point cloud data corresponding to the face region. Detecting the continuity break present in the face region by extracting candidates for and determining the continuity between the candidates,
The face evaluation support device according to any one of claims 1 to 6. The face evaluation unit is
a second continuity crack detection unit that detects a continuity crack continuously formed on the face surface based on the image data;
The second continuity break detection unit
Machine learning is performed on the image data corresponding to the face region using teacher data including learning image data in which the face is photographed and correct labels assigned to the continuity cracks formed in the face. detecting the continuity fracture present in the face region by inputting into a machine learning model performed;
The face evaluation support device according to any one of claims 1 to 7. The segmented area evaluation processing unit
each normal vector of a triangular network formed from the point cloud data corresponding to the face region is polar projected onto a stereo net for each of the segmented regions;
Based on the distribution of plots polar-projected on the stereo net, a crack predominant direction determining unit for determining, for each of the divided regions, a predominant crack direction indicating the strike inclination of the crack that is predominant in the face surface,
The face evaluation unit is
A third continuity crack detection unit that detects a continuity crack continuously formed on the face surface based on the image data and the point cloud data,
The third continuity break detection unit
performing edge detection processing for detecting edges on the image data corresponding to the face region;
The continuity crack existing in the face region is detected by fitting the edge detected by the edge detection process to the crack dominant direction for each of the divided regions determined by the crack dominant direction determining unit. ,
The face evaluation support device according to any one of claims 1 to 8. The face evaluation unit is
An artifact detection unit that detects an artifact present in the target area based on the image data,
The artifact detection unit is
The image data is divided into the image data corresponding to the face area and the image data corresponding to areas other than the face area. By respectively inputting to a machine learning model in which machine learning is performed with teacher data including the correct label given to the artificial object, the artificial object existing in the face region and the objects other than the face region detecting each of said artifacts present;
The face evaluation support device according to any one of claims 1 to 9. The segmented area evaluation processing unit
each normal vector of a triangular network formed from the point cloud data corresponding to the face region is polar projected onto a stereo net for each of the segmented regions;
Based on the distribution of plots polar-projected on the stereo net, a crack predominant direction determining unit for determining, for each of the divided regions, a predominant crack direction indicating the strike inclination of the crack that is predominant in the face surface,
The face evaluation unit is
a first skin-spotting risk location prediction unit that predicts the skin-spotting risk location based on the point cloud data;
The first skin-spotting risk location prediction unit includes:
projecting the predominant direction of cracks for each of the divided regions determined by the predominant direction of cracks determination unit onto a stereo net;
When a closed polygon is formed by the dominant directions of the cracks extending in at least three directions, the range of the face corresponding to the closed polygon is predicted as the risk of skin falling off.
The face evaluation support device according to any one of claims 1 to 10. The face evaluation unit is
a second skin-spotting risk location prediction unit that predicts the skin-spotting risk location based on the image data;
a skin-stripping risk location evaluation unit that predicts the skin-stripping risk location based on the prediction result of the first skin-stripping risk-location prediction unit and the prediction result of the second skin-stripping risk location prediction unit. ,
The second skin-spotting risk location prediction unit,
Machine learning is performed on the image data corresponding to the face region using teacher data including learning image data in which the face is photographed and a correct label given to the risky skin fallout portion in the face. By inputting into the machine learning model, predicting the risk of skin falling,
The skin removal risk area evaluation unit
The location at risk of skin-spotting predicted by the first risky-skin-spotting prediction unit and the location at risk of skin-spotting predicted by the second risky-skin-spotting prediction unit are compared, and the comparison result is coincidence. Determining that the risk is high for the skin removal risk location with a high degree of
The face evaluation support device according to claim 11. The image data acquisition unit
a color correction unit that performs color correction on color image data obtained by photographing the target area in color, including the face and a color sample having a predetermined black and white pattern, as the image data;
The color corrector is
converting the color image data into a grayscale image, and determining the brightness of a pixel located in the middle when all pixels constituting the grayscale image are arranged in order of brightness as a brightness intermediate value;
classifying all the pixels into a high-brightness pixel group consisting of pixels having brightness higher than the intermediate brightness value and a low-brightness pixel group consisting of an image having brightness equal to or lower than the intermediate brightness value;
For the pixel group corresponding to the white portion of the color sample, the average values of the three components of RGB are obtained, the maximum value among the average values is selected, and the maximum value and the average values are selected. color-correcting the high-brightness pixel group by adding a difference value to each of the three components of the high-brightness pixel group as a correction value for the high-brightness pixel group;
For the pixel group corresponding to the black portion of the color sample, each average value of the three components is obtained, the minimum value of the average values is selected, and each difference between the minimum value and the average value is selected. color correcting the low-lightness pixel group by subtracting each of the three components from each of the pixels of the low-lightness pixel group as a correction value for the low-lightness pixel group;
The face evaluation support device according to any one of claims 1 to 12. an image data acquisition step of acquiring image data in which a target region including the face of the tunnel is photographed;
A point cloud data acquisition step of acquiring point cloud data in which the three-dimensional shape of the target region is measured;
A face evaluation step of evaluating the face based on the image data and the point cloud data;
a support information output step of outputting support information based on the evaluation result of the face evaluation step;
The face evaluation step includes:
a face extraction step of extracting a face region corresponding to the face from the target region based on at least one of the image data and the point cloud data;
a roughness quantification step of quantifying the roughness of the face surface in the face region as a roughness feature amount along a plurality of scanning lines set in a plurality of scanning directions with respect to the face region based on the point cloud data; ,
a region segmentation step of obtaining a representative value of the roughness feature value for each intersection point where the plurality of scanning lines intersect, and segmenting the face region into segmented regions based on the distribution of the representative value for each intersection point;
an area evaluation processing step of evaluating an observation item related to the face as the evaluation result for each divided area in which the face area is divided by the area dividing step;
Face evaluation support method.

Images