JP2023108245A - Hollow measuring device, hollow measuring method, and hollow measuring program - Google Patents

Abstract

To provide a hollow measuring device, a hollow measuring method, and a hollow measuring program capable of easily specifying and measuring a hollow formed on the surface of a feature.SOLUTION: A hollow measuring device 200 includes: a measurement data acquisition unit 201 that acquires point group data of a measurement space obtained by measuring a measurement space including at least part of a hollow formed on the surface of a feature with a three-dimensional sensor; a target region designating unit 221 that designates a target region including the hollow in the point group data in the measurement space; a hollow identification unit 222 that identifies the hollow based on the point group data of the target region; a volume calculation unit 223 that calculates the volume of the hollow based on the point group data of the identified hollow. The hollow identification unit 222 selects either first identification process of identifying the hollow from the leveled ground or second identification process of identifying the hollow from the uneven ground to identify the hollow.SELECTED DRAWING: Figure 1

Description

The present invention relates to a dent measuring device, a dent measuring method, and a dent measuring program.

In the electric power industry, the gas industry, and the like, it is necessary to periodically inspect pipes laid underground, and the ground is excavated with an excavator or the like to inspect the pipes. When excavating, the amount of excavation is adjusted so that the pipes are exposed, but excessive excavation may damage the underground pipes. It is

On the other hand, in the construction industry and civil engineering industry, there is a demand for a technology that can measure the volume of excavated areas in order to grasp the remaining amount of earth and sand necessary for backfilling when backfilling excavated areas caused by construction work. there is Further, in the railway industry and the like, there is a risk of a subsidence accident if a track on the ground subsides due to hollowing out of the ground.

As a technique for measuring a dent caused by excavation or the like, a method of measuring the depth and volume of the dent based on point cloud data measured using a three-dimensional sensor is known. As an example of such technology, Patent Document 1 can be cited.

Patent Literature 1 discloses the following data analysis device, data analysis method, and program. A neighborhood space setting means sets each element point as a target point, and sets an element point neighborhood space including the target point and having a predetermined three-dimensional shape and size. The three-dimensional spatial filtering means extracts a feature point derived from a step or a crack to be detected when a predetermined plurality of element points of the point group or more are included in the space near each element point. . The crack detection means detects the position of the detection target from the arrangement of the feature points.

JP 2015-4588 A

It is described that the technique of Patent Document 1 enables automation of the work of detecting deformations, such as cracks and steps, that occur on the surface of the feature based on the three-dimensional coordinate data of the point cloud measured from the surface of the feature. It is However, the technique of Patent Literature 1 detects dents such as cracks and steps, and has a problem that it is impossible to specify the dents and measure the volume and dimensions of the dents.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a dent measuring device, a dent measuring method, and a dent measuring program capable of easily specifying and measuring a dent formed on the surface of a feature in view of the above-described problems. .

A depression measuring device according to an embodiment acquires point cloud data of a measurement space obtained by measuring a measurement space including at least part of a depression formed on a surface of a feature with a three-dimensional sensor. a target area designating unit that designates a target area containing a dent whose volume is to be obtained for the point cloud data of the measurement space; and a dent identifying unit that identifies the dent based on the point cloud data of the target area. a volume calculation unit that calculates the volume of the dent based on the point cloud data of the dent; A depression is identified by selecting either one of a second identification process of identifying a depression from uneven ground having a surface unevenness amount equal to or greater than a threshold value.

In addition, the depression measurement method according to one embodiment includes the step of acquiring point cloud data of a measurement space obtained by measuring a measurement space including at least part of a depression formed on the surface of a feature with a three-dimensional sensor. a step of specifying a target region containing a dent whose volume is to be obtained for the point cloud data of the measurement space; a step of identifying the dent based on the point cloud data of the target region; and calculating the volume of the dent based on the step of identifying the dent, wherein the step of identifying the dent includes a first identifying method for identifying the dent from the leveled land in which the amount of surface unevenness is smaller than a preset threshold, and the amount of unevenness of the surface is the threshold The depression is identified by selecting either one of the second identifying method for identifying the depression from the uneven ground.

A depression measurement program according to an embodiment is a depression measurement program that is executed by a computer, and is obtained by measuring a measurement space including at least a part of a depression formed on the surface of a feature with a three-dimensional sensor. A process of acquiring the point cloud data of the measurement space, a process of specifying the target area containing the dent for which the volume is calculated for the point cloud data of the measurement space, and a process of identifying the dent based on the point cloud data of the target area. and a process of calculating the volume of the dent based on the point cloud data of the identified dent, and in the process of identifying the dent, the dent is identified from the leveled land whose surface unevenness amount is smaller than a preset threshold. Either one of the first identification method and the second identification method of identifying a dent from uneven ground having a surface unevenness amount equal to or greater than a threshold value is selected to execute a process of identifying the dent.

Provided are a dent measuring device, a dent measuring method, and a dent measuring program capable of easily specifying and measuring a dent formed on the surface of a feature.

1 is a block diagram showing the configuration of a depression measuring device according to a first embodiment; FIG. It is a figure which shows a mode that a measurement space is measured with a three-dimensional sensor. FIG. 10 is a block diagram showing the configuration of a depression measuring device according to a second embodiment; FIG. 4 is a flow chart showing the operation of the depression measuring device shown in FIG. 3; 4 is a flow chart showing the flow of first identification processing in a depression identification unit; FIG. 2 is an image diagram of a feature having a hollow on a leveled surface; 10 is a flow chart showing the flow of second identification processing in a depression identification unit; FIG. 2 is an image diagram of a feature having a depression on uneven ground; FIG. 4 is an image diagram showing color data; FIG. 4 is an image diagram showing projection data; FIG. 10 is a diagram showing how a three-dimensional sensor measures a measurement space affected by occlusion. 4 is a flowchart showing the flow of first calculation processing in a volume calculator; FIG. 10 is an image diagram before interpolating a group of loss interpolation points given by the lowest point P1 to the missing portion; FIG. 10 is an image diagram after interpolating a group of loss interpolation points given by the lowest point P1 to the missing portion; FIG. 10 is an image diagram before interpolating a group of loss interpolation points given by an inclination angle θ of a dent in a missing portion; FIG. 10 is an image diagram after interpolating a group of loss interpolation points given by an inclination angle θ of a dent in a missing portion; FIG. 4 is an image diagram after measuring a dent with the dent measuring device shown in FIG. 3 ; 4 is a block diagram showing the hardware configuration of the depression measuring device shown in FIG. 3; FIG.

Embodiment 1
BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments. Also, for clarity of explanation, the following description and drawings are simplified as appropriate. Furthermore, in the following description, the same or equivalent elements are denoted by the same reference numerals, and overlapping descriptions are omitted.

FIG. 1 is a block diagram showing the configuration of the depression measuring device according to the first embodiment. As shown in FIG. 1 , the dent measuring device 200 has a measurement data acquisition unit 201 , a target area designation unit 221 , a dent identification unit 222 and a volume calculation unit 223 .

The measurement data acquisition unit 201 obtains measurement data, which is point cloud data of the measurement space obtained by measuring (photographing) the measurement space including at least a part of the depression 1 formed on the surface of the feature 10 with the three-dimensional sensor 20. Get 100. The target region designating unit 221 designates a target region 503 including the dent 1 whose volume is to be calculated for the measurement data 100 . The dent identification unit 222 identifies the dent 1 based on the point cloud data of the target region 503 . In addition, the depression identification unit 222 performs a first identification process to identify depression 1 (502) from leveled land 501 whose surface unevenness amount is smaller than a preset threshold, Either one of the second specifying process for specifying ( 702 ) is selected to specify the recess 1 . The volume calculator 223 calculates the volume of the depression 1 based on the point cloud data of the identified depression 1 .

According to the dent measuring device 200 according to the present embodiment, it is possible to easily specify and measure a dent formed on the surface of a feature.

Embodiment 2
Embodiments of the present invention will now be described with reference to FIGS. In this embodiment, the depression measuring device 200 described in the first embodiment will be described in more detail. First, FIG. 2 is a diagram showing how the measurement space is measured by a three-dimensional sensor.

As shown in FIG. 2, the dent measuring device 200 acquires measurement data 100, which is point cloud data of the measurement space obtained by measuring (photographing) the measurement space with the three-dimensional sensor 20, and in the measurement data 100, This device automatically identifies a depression 1 (502, 702) formed on the feature 10 from the above and calculates the volume of the identified depression 1 (502, 702).

In the present embodiment, an example of a feature 10 formed with a dent 502 on a level ground 501 with a flat surface and a dent 702 on an irregular ground 701 with an irregular surface is used to measure the dent. Processing performed by the device 200 will be described. In this embodiment, the leveled ground 501 is described as being a horizontal surface, but the leveled ground 501 may be a wall surface extending in the vertical direction, a slope, or a curved surface.

The three-dimensional sensor 20 is provided at a position capable of photographing the measurement space including at least part of the depressions 502 and 702 formed on the surface of the feature 10 . In the example shown in FIG. 2, the three-dimensional sensor 20 is installed near the obliquely upper part of the depressions 502 and 702 . The three-dimensional sensor 20 is, for example, a ToF (Time-of-Flight) camera, a stereo camera, or a 3D-LiDAR (Light Detection And Ranging) sensor that measures a distance in a three-dimensional space.

The three-dimensional sensor 20 generates measurement data 100 from captured images. The three-dimensional sensor 20 is communicably connected to the recess measuring device 200 via a network, which is, for example, a wired or wireless communication network.

The measurement data 100 is three-dimensional point cloud data including the distance of each measurement point from the sensor. The three-dimensional information may be latitude, longitude, and sea level (height) information, or may be a three-dimensional Euclidean coordinate system or a polar coordinate system with a specific position set by the user as the origin. In the following example, a three-dimensional Euclidean coordinate system (with X, Y, and Z coordinates in each direction) at the origin set by the user is assumed. Units of each coordinate are expressed in meters (m), centimeters (cm), and millimeters (mm), but other units may be used. Also, the value of the Z coordinate means height (depth) information.

A three-dimensional point is a point to which information such as the time at which the point cloud data was captured, laser reflection intensity, and color information such as red, blue, and green, etc. is assigned. There is no limit to the information given to the three-dimensional points, but at least three-dimensional coordinates (X, Y, Z coordinates), which are position information, are given. It is an assembled set. In this embodiment, the Z-axis direction of the Euclidean coordinate system means the vertical direction, and the two-dimensional plane spanned by the X-axis and the Y-axis means the horizontal plane.

The depression measurement device 200 is generated by combining other sensors such as a position and orientation sensor such as a GPS (Global Positioning System) or an IMU (Inertial Measurement Unit), an RGB (Red Green Blue) camera, an environment sensor, and the three-dimensional sensor 20. measured data 100 may be acquired. In addition, the depression measuring device 200 may have a function of cooperating with an external system (for example, an external device, inspection, construction, etc.).

Any method can be adopted as a method for obtaining the measurement data 100 . For example, the three-dimensional sensor 20 may be mounted on a vehicle such as an automobile or a moving body such as a mobile robot, and the measurement space may be measured while the three-dimensional sensor 20 is moved by the moving body. As one form of the acquisition method of the measurement data 100, if the three-dimensional sensor 20 is mounted on an aerial moving object such as a drone, it is possible to take an aerial photograph of the measurement space.

Next, the configuration of the depression measuring device 200 will be described with reference to FIG. FIG. 3 is a block diagram showing the configuration of the depression measuring device according to the second embodiment. As shown in FIG. 3 , the depression measuring device 200 is composed of a preprocessing section 210 , a measuring section 220 and a user interface section 230 .

The preprocessing unit 210 corresponds to the measurement data acquisition unit 201 described with reference to FIG. 1 and acquires the measurement data 100 from the three-dimensional sensor 20 . The preprocessing section 210 performs preprocessing for shaping data on the measurement data 100 before the processing in the measurement section 220 , and outputs the preprocessed measurement data 100 to the measurement section 220 . The preprocessing unit 210 has a format conversion unit 211 , a noise removal unit 212 and an angle conversion unit 213 .

The format conversion section 211 has a function of converting the data format of the measurement data 100 into a predetermined format that can be used by the depression measuring device 200 . The format conversion section 211 outputs the measurement data 100 converted into a predetermined format to the noise elimination section 212 . Note that when the measurement data 100 acquired from the three-dimensional sensor 20 is in a predetermined format, the process of the format conversion section 211 is omitted. The format conversion unit 211 may perform not only data format conversion, but also synthesis of a plurality of data, and fusion or integration with data acquired from other sensors or external systems.

The noise removal unit 212 has a function of removing unnecessary point cloud data such as outliers from the point cloud data included in the measurement data 100 . Noise removal methods include, for example, smoothing processing, filtering, outlier removal processing, correction processing, and the like. The noise removal section 212 outputs the noise-removed measurement data 100 to the angle conversion section 213 .

The angle conversion unit 213 has a function of performing angle conversion to rotate the measurement data 100 to a predetermined angle as preprocessing for the shortest measurement. The angle conversion unit 213 performs angle conversion of the measurement data 100 based on information from other sensors and external systems. For example, the angle conversion unit 213 acquires the vertical direction at a predetermined angle from the IMU, or refers to information such as topography/geography information from other sensors or external systems, CAD (Computer Aided Design), and construction drawings. By doing so, the angle conversion of the measurement data 100 can be performed. In this embodiment, angle conversion is performed so that the measurement data 100 has a predetermined angle by rotating the depressions 502 and 702 such that the depth direction is the vertical direction (Z-axis direction).

Further, the angle conversion unit 213 may perform angle conversion so that the measurement data 100 has a predetermined angle by rotating such that the recessed surfaces of the depressions 502 and 702 are horizontal. Further, the angle conversion of the measurement data 100 may be manually performed based on the information input by the user's operation accepted by the operation unit 231 without using the angle conversion unit 213 .

Subsequently, the measurement unit 220 identifies at least a portion of the depressions 502 and 702 based on the measurement data 100, and calculates (estimates) the volume of the depressions 502 and 702 from at least a portion of the identified depressions 502 and 702. have The measuring unit 220 is composed of a target area specifying unit 221 , a dent specifying unit 222 and a volume calculating unit 223 .

The target region designating unit 221 has a function of designating a target region 503 including dents 502 and 702 whose volumes are to be obtained for the measurement data 100 acquired from the preprocessing unit 210 based on designated conditions. The target area specifying unit 221 specifies the target area 503 based on conditions (height, width, depth, etc.) specified by user's operation received by the operation unit 231, for example. The user can specify the target area 503 by excluding areas where the depressions 502 and 702 do not exist within the measurement space. The target region designating unit 221 performs a process of storing information regarding the target region 503 based on the conditions acquired via the operation unit 231 in the memory 120 of the dent measuring device 200, and transmits information regarding the target region 503 to the dent specifying unit 222. Output. The information about the target region 503 includes the conditions acquired via the operation unit 231 and the target region point cloud, which is the point cloud data of the target region 503 .

Note that the target region designating unit 221 designates the entire measurement data 100 (full view of the measurement space) as the target region 503 when no conditions are designated. That is, the target region designating unit 221 outputs the target region point cloud corresponding to the measurement data 100 for which the preprocessing has been completed to the depression identifying unit 222 when no condition is designated. The target area designating unit 221 may save a plurality of pieces of information about the designated target area 503 and switch the target area 503 by reading the saved data.

The depression specifying unit 222 has a function of automatically specifying (clustering) the depressions 502 and 702 in the target region 503 based on the target region point cloud acquired from the target region specifying unit 221 . As a specific method for identifying the depressions 502 and 702, there is a method of extracting differences obtained by comparing point cloud data before and after the depressions 502 and 702 are generated on the surface of the feature 10. FIG. As another method of identifying the dents 502 and 702, a method of setting thresholds for the dimensions and volumes of the dents 502 and 702 and extracting portions exceeding the thresholds, and displaying images that are color-coded according to the dimensions and volumes of the dents 502 and 702. A method of displaying on the part 232 and the like can be mentioned. In addition, the depression identification unit 222 may identify the depressions 502 and 702 in cooperation with other sensors such as the above-described RGB camera.

The depression identification unit 222 selects either the first identification process or the second identification process according to the shape (the amount of unevenness) of the surface of the feature 10, An indentation 702 present on 701 can be identified. The hollow identification unit 222 generates a three-dimensional map, which is point cloud data of at least part of the identified hollows 502 and 702 , and outputs the map to the volume calculation unit 223 . The details of the first specifying process and the second specifying process in the depression specifying section 222 will be described later.

The volume calculator 223 has a function of calculating (estimating) the volumes of the dents 502 and 702 based on the three-dimensional map acquired from the dent specifying unit 222 . The volume calculation unit 223 selects either one of the first calculation process and the second calculation process according to the presence or absence of the influence of occlusion that cannot be measured by the three-dimensional sensor 20, and calculates the volumes of the depressions 502 and 702. It can be calculated (estimated).

The volume calculator 223 can also calculate (estimate) at least one of the height, width, and depth dimensions of the depressions 502 and 702 as well as the volume of the depressions 502 and 702 . When calculating the dimensions of the depressions 502 and 702, the maximum and minimum values of the X, Y, and Z coordinates are obtained from the point cloud data included in the three-dimensional map, and the difference between the maximum and minimum values of each coordinate is used to determine the dimensions. Calculate

As a method for confirming the presence or absence of the influence of occlusion, a target region 503 is specified by the target region specifying unit 221 so that the dents 502 and 702 are contained as just as possible. should be determined by the dimensions of

The volume calculation unit 223 outputs the calculation result of the volume to the display unit 232 . Note that the calculation result may include not only the volume but also the size calculation result. Details of the first calculation process and the second calculation process in the volume calculation unit 223 will be described later.

The volume calculator 223 may calculate the volumes of the depressions 502 and 702 using information acquired from other sensors. For example, the volume calculation unit 223 corrects the position information and size (dimension) information of the dents 502 and 702 using information from an RGB camera, an environment sensor, or the like, so that the volume calculation accuracy can be improved. . Also, the volume calculation unit 223 may calculate the volume using information acquired from an external system or information input by a user's operation received by the operation unit 231 . Here, as information for calculating the volume of the depressions 502 and 702, the type of the specific target (eg, material), the situation (eg, soil viscosity, soil moisture content, etc.), the environment (eg, precipitation, temperature and humidity etc.).

The user interface unit 230 has a function of visualizing and manipulating point cloud data handled by the depression measuring device 200 and displaying measurement results and the like measured by the measuring unit 220 . The user interface section 230 has an operation section 231 and a display section 232 . The user interface section 230 may include not only an interface circuit to the operation section 231 and the display section 232, but also an interface circuit to an external system (for example, an alarm system, etc.).

In addition, the user interface unit 230 performs a process of accumulating input or calculated information in the memory 120 of the dent measuring device 200 in chronological order, and stores data abstracted from the input or calculated information to the dent measuring device. A process of accumulating in the memory 120 of 200 may be performed. Furthermore, the information and data stored in the memory 120 may be read out and transmitted to a user or an external system.

The operation unit 231 receives operations from the user. The operation unit 231 has an input device with which the user can input instructions, information, and the like to the depression measuring device 200 . The input device may be a physical key such as a keyboard, mouse, numeric keypad, or button, or may be configured as a touch panel integrally with the display. By operating the operation unit 231 , the user can input, for example, conditions for designating the target region 503 to the depression measuring device 200 .

The display unit 232 has a display device for displaying the identification result of the depressions 502 and 702 identified by the depression identification unit 222, the calculation result of the volume calculated by the volume calculation unit 223, and the like. The display device is, for example, a flat panel display such as a liquid crystal display, plasma display, or organic EL (Electro-Luminescence) display.

Next, with reference to FIG. 4, the operation of the depression measuring device 200 according to this embodiment will be described. FIG. 4 is a flow chart showing the operation of the depression measuring device shown in FIG. As shown in FIG. 4, the dent measuring device 200 executes dent measuring processing in steps S301 to S307.

In step S<b>301 , the preprocessing unit 210 acquires measurement data 100 . In this step, measurement data 100 obtained by measuring the measurement space with the three-dimensional sensor 20 is obtained. The format conversion unit 211 that has acquired the measurement data 100 converts the data format of the measurement data 100 into a predetermined format as necessary.

Subsequently, in step S302, the noise removal unit 212 removes unnecessary point cloud data such as outliers from the point cloud data included in the measurement data 100 acquired in step S301.

Subsequently, in S303, the angle conversion unit 213 performs angle conversion to rotate the measurement data 100 subjected to the noise removal processing in step S302 to a predetermined angle. In this step, angle conversion is performed so that the depth direction from the top surface to the bottom surface of the depressions 502 and 702 in the measurement data 100 is vertically downward (Z-axis downward). In this way, the preprocessed measurement data 100 is generated.

Subsequently, in step S304, the target region designating unit 221 designates the target region 503 including the dents 502 and 702 whose volumes are to be obtained from the measurement data 100 for which the preprocessing of step S303 has been completed. In this step, a target region 503 is specified for the measurement data 100 for which preprocessing has been completed based on the conditions accepted by the operation unit 231, and a target region point cloud is generated.

Subsequently, in step S305, the depression identifying unit 222 identifies (clusters) the depressions 502 and 702 based on the target region point cloud generated in step S304. The depression measuring device 200 selects either a first identification process of identifying depressions 502 existing on level ground 501 or a second identification process of identifying depressions 702 existing on uneven ground 701 to determine depressions. identification. The depression measuring device 200 automatically switches between the first identification process and the second identification process according to the unevenness amount of the surface of the feature 10 to identify the depressions 502 and 702 .

Specifically, when the unevenness amount, which is the difference between the Z-coordinate values of the convex portion and the concave portion on the surface of the feature 10, is smaller than the threshold value, the depression identifying unit 222 determines that the land is leveled 501 and performs the first identification process ( Step S400) is executed, and if the amount of unevenness on the surface of the feature 10 is equal to or greater than the threshold, it is determined that the ground is uneven 701, and the second identification process (Step S600) is executed.

Therefore, the flow of the first specifying process will be described with reference to FIGS. 5 and 6. FIG. FIG. 5 is a flow chart showing the flow of first identification processing in the recess identification section. FIG. 6 is an image diagram of a feature having a hollow on a level surface. The dent measuring device 200 executes the first identifying process (step S400) shown in steps S401 to S403 shown in FIG. 5 to identify the dent 502 on the leveled ground 501 shown in FIG.

First, in step S401, the dent specifying unit 222 performs plane removal to remove a plane portion (for example, a flat ground surface, a road surface, etc.) from the target region point group acquired from the target region specifying unit 221, and removes the plane removal points. Generate groups. In this step, the depression identifying section 222 preferably obtains from the target area designating section 221 the target area point cloud in which the areas in the measurement space where the depression 502 does not exist are excluded.

The planar portion can be extracted according to an algorithm such as RANSAC (Random Sample Consensus). By removing the flat portion, flat objects of the same shape such as the ground and wall surfaces are removed, and clustering can be performed correctly, so misclassification can be suppressed. The noise removal unit 212 may be configured to perform noise removal processing on the plane-removed point cloud as necessary.

Subsequently, in step S402, the group of plane-removed points generated in step S401 is clustered to generate plane-removed clusters. Clustering is a method of dividing or classifying point cloud data (here, plane-removed point cloud) into a plurality of clusters, and for example, a k-means method or the like can be employed.

Further, in step S403, point cloud interpolation is performed to interpolate the top surface interpolation point cloud, which is the point cloud data of the top surface portion 502a (the vertically upper surface) of the hollow 502 that is missing for each plane-removed cluster generated in step S402. By doing so, a three-dimensional map in which the upper surface interpolation point group is interpolated is generated. The three-dimensional map generated here is a three-dimensional map representing depressions 502 existing on the leveled ground 501 and is generated for each identified depression 502 . The three-dimensional coordinates of the upper surface interpolation point group are given by the conditions of the target region 503 specified in step S304.

Here, “interpolation” means calculating values of unmeasured parts based on values of parts obtained by measurement. The indentation identifying section 222 outputs the generated three-dimensional map to the volume calculating section 223 . The above is the flow of the first identification process.

On the other hand, the flow of the second specifying process will be described with reference to FIGS. 7 and 8. FIG. FIG. 7 is a flow chart showing the flow of second identification processing in the recess identification section. FIG. 8 is an image diagram of a feature having a depression on uneven ground. The dent measuring device 200 executes the second identifying process shown in steps S601 to S610 shown in FIG. 7 to identify the dent 702 on the uneven ground 701 shown in FIG.

First, in S601, the depression specifying unit 222 uses the value of the Z coordinate among the three-dimensional coordinates (X, Y, Z coordinates) of each three-dimensional point forming the target region point group acquired from the target region specifying unit 221 as a reference. 3D points are sampled as , and the target region point cloud is layered. Let n be the number of samplings at this time. In this step, the depression identifying section 222 preferably acquires the target area point cloud for which the conditions are not specified from the target area specifying section 221 . Note that the Z-axis indicates the depth direction of the depression 702 along the vertical direction. Also, the X-axis indicates the width direction of the recess 702 and the Y-axis indicates the depth direction of the recess 702 .

The hollow identifying unit 222 generates color data by color-coding the sampled three-dimensional points according to the depth (Z-coordinate value). FIG. 9 is an image diagram showing color data. The color data shown in FIG. 9 represent the side surface of the feature 10 viewed from the X-axis direction. In the example shown in FIG. 9, the darker the color, the deeper the depth, and the lighter the color, the shallower the depth.

Subsequently, in S602, the three-dimensional coordinates (X, Y, Z coordinates) of the three-dimensional points sampled in step S601 are projected onto the XY plane and converted into two-dimensional coordinates (X, Y coordinates). Generate projection data for FIG. 10 is an image diagram showing projection data. The projection data shown in FIG. 10 consists of two-dimensional point cloud data in which two-dimensional points obtained by converting three-dimensional coordinates to two-dimensional coordinates represent a plan view of the feature 10 viewed from the Z-axis direction. there is In the example shown in FIG. 10, a darker-colored area indicates a deeper depth, and a lighter-colored area indicates a shallower depth. Note that the XY plane refers to the recessed surface of the depression 702 orthogonal to the Z-axis.

Subsequently, in S603, the layer counter m is set to "1". The layer counter m is a counter for performing hollow search for searching the layered target area point cloud from the upper layer (m=1) to the lower layer (m=n) in the second specifying process. Note that the lower layer is a layer located vertically below the upper layer.

Subsequently, in S604, the layer point cloud, which is the point cloud data of the m-th layer among the layered target area point clouds, is acquired and added to a search cluster for grouping the searched layers.

Subsequently, in S605, the search clusters are clustered to generate layer clusters for each depression 702. FIG.

Subsequently, in S606, it is searched for each layer cluster whether or not there is a lower layer on the XY plane within the area of each layer cluster. As a result of the search, if a lower layer exists on the XY plane (step S606; YES), it is determined that a recess 702 exists, and the process proceeds to step S607. Then, in step S607, the layer point cloud of the lower layer searched in step S606 is obtained, and the process proceeds to step S608. If the lower layer does not exist on the XY plane (step S606; NO), it is determined that the depression 702 does not exist, and the process proceeds to step S608.

Subsequently, in S608, "1" is added to the layer counter (m=m+1). After that, the process returns to S604, and a series of depression search processes shown in steps S604 to S607 are repeated until the lowest layer (m=n) of the layered target area point cloud is searched. When the hollow search is completed up to the lowest layer where the layer counter is m=n, the process advances to step S609.

Subsequently, in S609, projection restoration is performed for the layer point cloud of each layer of m=1 to n acquired by the depression search in steps S604 to S608. In this step, the two-dimensional coordinates (X, Y coordinates) of the two-dimensional points forming the layer point group are converted into three-dimensional coordinates (X, Y, Z coordinates).

Subsequently, in step S610, a three-dimensional map for each depression 702 is generated by clustering the layer point cloud restored in three dimensions in step S609. The above is the flow of the second identification process.

Then, the depression identifying section 222 outputs the three-dimensional map generated by the first identifying process or the second identifying process to the volume calculating section 223 .

Here, as another method of specifying the depression 702 in the second identification process, for example, the image of the color data shown in FIG. 9 or the image of the projection data shown in FIG. The method of identifying the indentation 702 using As another method, an algorithm such as edge detection is used to detect the cut-like portion (outer contour of the recessed surface) from the projection data, and the lower layer exists in the area surrounded by the cut-like portion. A hollow search method for searching whether or not to perform may be employed.

Returning to FIG. 3, in S306, the volume calculator 223 calculates (estimates) the volume of the recess 1 based on the three-dimensional map generated in steps S400 and S600. When there is occlusion, the volume calculation unit 223 interpolates the missing interpolation point group obtained from the three-dimensional map of the depression 1 identified by the depression identification unit 222 into the missing portion 903 created by the occlusion, and then calculates the volume of the depression 1. A first calculation process for calculating is performed. If there is no occlusion, the volume calculation unit 223 performs a second calculation process of calculating the volume of the depression 1 from the three-dimensional map of the depression 1 identified by the depression identification unit 222 . The volume calculation unit 223 selects either one of the first calculation process and the second calculation process according to the presence or absence of the influence of occlusion to calculate the volume.

Here, FIG. 11 is a diagram showing how the three-dimensional sensor measures the measurement space under the influence of occlusion. In the following description, a case will be described in which the dent 1 formed on the surface of the feature 10 shown in FIG. 11 is the identification target.

As shown in FIG. 11, when there is an obstacle between the three-dimensional sensor 20 and the depression 1 of the specific target, the obstacle in the foreground hides part of the depression 1 behind the occlusion. occurs. Therefore, the area behind the obstacle in the recess 1 cannot be measured by the three-dimensional sensor 20 . In the example shown in FIG. 11, the three-dimensional sensor 20 is arranged on the front side of the depression 1 (left side in FIG. 11), and the front side portion of the feature 10 existing between the three-dimensional sensor 20 and the depression 1 is is an obstacle. If there is an occlusion effect, the 3D map acquired from the depression identifying unit 222 will be in a state where part of the point cloud data is missing due to the occlusion.

As described above, when the measurement data 100 has the missing part 903 that does not include the point cloud data, even if the volume is calculated for the three-dimensional map acquired from the depression identifying part 222, the volume of the occlusion part is obtained. Therefore, an accurate volume of the entire recess 1 cannot be calculated. Therefore, the dent measuring apparatus 200 calculates the volume of the dent 1 by automatically switching between the first calculation process (step S800) and the second calculation process (step S900) depending on the presence or absence of the influence of occlusion. Note that the user may visually check the measurement data 100 to determine whether or not there is an influence of occlusion, and manually switch between the first calculation process and the second calculation process.

The flow of the first calculation process will be described with reference to FIG. FIG. 12 is a flow chart showing the flow of the first calculation process in the volume calculator. As shown in FIG. 12, the depression measuring device 200 executes the first calculation process shown in steps S801 to S804 when it is confirmed that there is an influence of occlusion.

First, in step S801, outliers are removed from the point cloud data included in the three-dimensional map 904 of the depression 1 identified by the depression identification unit 222 in step S305, if necessary.

Subsequently, in step S802, from among the point cloud data included in the three-dimensional map from which the outliers have been removed in step S801, the lowest point, which is a three-dimensional point including the Z coordinate existing at the deepest position Search for P1 (bottom point search). Then, the lowest point search is performed to obtain the three-dimensional coordinates of the searched lowest point P1.

Subsequently, in step S803, a complete three-dimensional map is generated by performing point group interpolation for interpolating the missing interpolation point group in the missing portion 903 . A completed three-dimensional map is generated for each identified hollow 1 .

Here, there are two methods for performing point group interpolation in step S803. First, the first method will be described with reference to FIGS. 13 and 14. FIG. FIG. 13 is an image diagram before interpolating the defect interpolation point group given by the lowest point P1 to the defect portion. FIG. 14 is an image diagram after interpolating the group of loss interpolation points given by the lowest point P1 to the missing portion.

Measured data 100 shown in FIG. 13 is a point on the Y-axis front side (left side in FIG. 13) of the center axis due to the influence of occlusion, when the center axis is the X-axis passing through the lowest point P1 included in the three-dimensional map 904. At least a part of the group data has a missing part 903 that is missing. That is, the missing portion 903 shown in FIG. 13 does not include the actual lowest point (lowest point P1). In this case, the lowest point P1 retrieved in step S802 is assumed to be the bottom surface portion (vertically lower surface) of the identified depression 1. FIG.

The first method is suitable when the lowest point P1 retrieved in step S802 matches the actual lowest point. That the retrieved lowest point P1 matches the actual lowest point means that the actual lowest point is not missing and that the actual lowest point is not included in the missing portion 903. do. When applying the first method, the three-dimensional coordinates of the missing interpolation points are given by the lowest point P1 retrieved in step S802.

FIG. 14 shows a three-dimensional map 905 in which the missing interpolation point group given by the lowest point P1 is interpolated into the missing portion 903. FIG. The missing interpolation point group given by the lowest point P1 is the point group data located in the range of depth L (FIG. 13) on the back side of the Y axis from the center axis of the three-dimensional map 904 (on the right side in FIGS. 13 and 14). It is point cloud data that is symmetrical about the central axis. That is, the missing interpolation point group is point group data located within a range of depth L on the front side of the Y axis (left side in FIGS. 13 and 14) from the central axis. A three-dimensional map 905 generated by interpolating the missing interpolation point group given by the lowest point P1 into the missing portion 903 is a completed three-dimensional map. In this way, it is possible to estimate the shape of the missing portion 903 shown in FIG. 13 from the three-dimensional map acquired from the indentation identifying section 222 and generate a complete three-dimensional map.

On the other hand, the second method is suitable when the lowest point P1 retrieved in step S802 does not match the actual lowest point. That the retrieved lowest point P1 does not match the actual lowest point means that the actual lowest point is missing, and that the actual lowest point is included in the missing part 903. means. When applying the second method, the three-dimensional coordinates of the defect interpolation point group are given by the inclination angle of the depression 1 .

Therefore, the second method will be described with reference to FIGS. 15 and 16. FIG. FIG. 15 is an image diagram before interpolating the defect interpolation point group given by the inclination angle θ of the dent in the defect portion. FIG. 16 is an image diagram after interpolating the defect interpolation point group given by the inclination angle θ of the depression in the defect portion.

The measurement data 100 shown in FIG. 15 has a missing portion 903 in which the actual lowest point corresponding to the bottom portion of the depression 1 is missing due to occlusion. That is, the missing portion 903 shown in FIG. 15 includes the actual lowest point. In this case, the lowest point P1 retrieved in step S802 is a three-dimensional point located vertically above the actual lowest point. Therefore, when a completed three-dimensional map is generated by the same method as the first method using the lowest point P1 included in the three-dimensional map 904 shown in FIG. The error between the volumes can increase.

Therefore, when the actual lowest point is missing, the inclination angle θ of the dent 1 is calculated from the point cloud data of the three-dimensional map 904 before performing the point cloud interpolation, as shown in FIG. 16 . The tilt angle θ can be calculated, for example, using the three-dimensional coordinates of each three-dimensional point located on the farthest side and the foremost side when viewed from the three-dimensional sensor 20 in the three-dimensional map 904 . 15 and 16, the back side as viewed from the three-dimensional sensor 20 is the right side, and the front side as viewed from the three-dimensional sensor is the left side in FIGS. The inclination angle θ of the recess 1 calculated here is the angle formed by the inclined surface of the recess 1 and the horizontal direction.

Then, a group of loss interpolation points representing a shape along the inclination angle θ is interpolated from both ends of the upper surface portion of the depression 1 on the Y axis. Thereby, a virtual lowest point P2 is obtained. A three-dimensional map 905 generated by interpolating the missing portion 903 with the missing interpolation point group given by the inclination angle θ is a completed three-dimensional map. In this way, the shape of the missing portion 903 shown in FIG. 15 can be estimated from the three-dimensional map acquired from the depression identifying section 222, and a completed three-dimensional map can be generated. Note that the completed three-dimensional map is generated for each identified depression 1 .

Subsequently, in step S804, the volume of the completed three-dimensional map is calculated. The volume can be determined, for example, by constructing a convex hull of the point cloud data of the completed 3D map. Here the convex hull is the smallest convex set that contains the given set. A convex hull can be created by an algorithm using, for example, the Quickhull method. A convex hull created by an algorithm using the Quickhull method is composed of a plurality of tetrahedrons. Assuming that the base area of the tetrahedron is S and the height is h, the volume Vi of the tetrahedron can be calculated from the following formula (1) for the volume of the triangular pyramid.
Vi=Sh/3 Expression (1)

By calculating the volume of the tetrahedrons forming the created convex hull in this way, the volume of the completed three-dimensional map can be calculated.

On the other hand, in the second calculation process, the volume is calculated with respect to the three-dimensional map acquired from the depression identification unit 222 without interpolating the group of loss interpolation points. The volume is obtained from the dent identification unit 222 by, for example, creating a convex hull of the point cloud data of the three-dimensional map obtained from the dent identification unit 222 and calculating the volume of the tetrahedron that constitutes the created convex hull. The volume of the 3D map can be calculated. The calculation method of the volume by the convex hull is the same as the calculation method explained in the first calculation process.

Instead of the above-described method of creating a convex hull, for example, the volume of the dent 1 may be calculated for voxel data obtained by processing a three-dimensional map used for volume calculation into a cube called a voxel.

The volume calculation unit 223 then outputs the calculation result of the volume generated by the first calculation process or the second calculation process to the display unit 232 . By acquiring the measurement data 100 that is less affected by occlusion, the missing portion 903 can be reduced and the volume calculation accuracy can be improved. As a method of acquiring measurement data 100 with little influence of occlusion, there is a method of measuring the measurement space from various directions while moving the three-dimensional sensor 20 . As another method of acquiring measurement data 100 with little influence of occlusion, a plurality of three-dimensional sensors 20 are installed in the measurement space, and among the plurality of data acquired from the plurality of three-dimensional sensors 20, the most occlusion is obtained. A method of extracting the measurement data 100 having a small influence can be used.

Returning to FIG. 4, the display unit 232 acquires the calculation result from the volume calculation unit 223 in S307. The display unit 232 that has acquired the calculation result displays the calculation result. FIG. 17 is an image diagram after the depression is measured by the depression measuring device shown in FIG. As shown in FIG. 17, the calculation results include, in addition to the information about the volume V of the depression 1, for example, the measurement data 100, the target region 503, the identified depression 1, the dimensions of the depression 1 (height H, width W, It may contain various information about the depth D) and the like.

The depression measurement process according to the present embodiment can be implemented in a terminal such as a PC (Personal Computer) or a tablet used by the user. FIG. 18 is a block diagram showing the hardware configuration of the depression measuring device shown in FIG. As shown in FIG. 18, the depression measuring device 200 has an input/output interface 110, a memory 120, and a processor .

Input/output interface 110 is used to communicate with any other device. For example, the input/output interface 110 may be used to acquire the measurement data 100 from the three-dimensional sensor 20, or may be used to output data regarding calculation results to another device.

The memory 120 is configured by, for example, a combination of volatile memory and non-volatile memory. The memory 120 is used to store software (computer program) including one or more instructions executed by the processor 130, data used for various processes of the depression measuring device 200, and the like.

The processor 130 reads out software (computer program) from the memory 120 and executes it to perform the processing of the depression measuring apparatus 200 described using the flowchart shown in FIG. Here, the depression measurement program includes a process of acquiring point cloud data of the measurement space obtained by measuring the measurement space including at least part of the depression 1 formed on the surface of the feature 10 with the three-dimensional sensor 20; A process of designating a target region 503 containing a dent 1 whose volume is to be obtained with respect to the point cloud data of the measurement space, a process of specifying the dent 1 based on the point cloud data of the target region 503, and a point of the identified dent 1. and a process of calculating the volume of the depression 1 based on the group data. In addition, in the process of identifying dent 1, the dent measurement program includes a first identifying method for identifying dent 1 (502) from leveled land 501 in which the amount of unevenness on the surface is smaller than a preset threshold, and the amount of unevenness on the surface is equal to or greater than the threshold. The computer is caused to execute a process of specifying the depression 1 by selecting either one of the second identification method of identifying the depression 1 (702) from the uneven ground 701 of .

The processor 130 may be, for example, a microprocessor, an MPU (Micro Processor Unit), or a CPU (Central Processing Unit). Processor 130 may include multiple processors.

Note that the above-described program executed by the depression measuring device 200 can be stored using various types of non-transitory computer readable media and supplied to computers. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (eg, flexible discs, magnetic tapes, hard disk drives), magneto-optical recording media (eg, magneto-optical discs), CD-ROM (Read Only Memory) CD-R, CD - R/W, including semiconductor memory (eg Mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), Flash ROM, RAM (Random Access Memory)). The program may also be supplied to the computer on various types of transitory computer readable medium. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable media can deliver the program to the computer via wired channels, such as wires and optical fibers, or wireless channels. The embodiment has been described above.

Here, as a technology related to the above, when measuring the depth of a plurality of dents, the dents to be detected are visually confirmed from the point cloud data, and measurement is performed for each of the detected dents through visual confirmation. However, there is a problem that such a technique increases the cost.

Further, for example, in the technique of measuring a specific depth in a depression, it is necessary to acquire point cloud data of the entire depression in order to measure the dimensions and volume of the entire depression. However, with such a technique, if a part of the dent is missing due to occlusion or the like, it is not possible to calculate the dimensions and volume of the entire dent only from the point cloud data of the part of the dent. The problem is that it may disappear.

On the other hand, in the present embodiment, since the target region 503 including the depression 1 for which the volume is to be obtained is arbitrarily designated, the identification and measurement of the depression 1 can be easily performed, the cost can be reduced, and the identification of the depression 1 is possible. And the accuracy of measurement can be improved. Furthermore, by arbitrarily specifying the target region 503 including the depression 1 whose volume is to be calculated, not only the overall volume of the depression 1 but also the partial volume of the depression 1 can be calculated. Therefore, according to this embodiment, simulation of volume measurement is possible.

Further, in the present embodiment, even if a part of the recess 1 is missing due to occlusion or the like, the defect interpolation point group can be interpolated into the missing part 903 based on the point cloud data of at least a part of the recess 1. Indentation 1 can be measured.

In this embodiment, the depression 1 can be specified and measured regardless of whether the surface on which the depression 1 exists is the leveled ground 501 or the uneven ground 701 . Therefore, it can be applied to the case where it is required to specify and measure the dent 1 in an environment with a complicated shape.

The dent measuring device, the dent measuring method, and the dent measuring program described in the above embodiments are used for progress management of excavation work in the construction industry, monitoring of the excavation amount of excavation work in the gas and electric power industries, and inspection of roads and facilities in the civil engineering industry. It can be used for monitoring, abnormality monitoring of farmland in agriculture and forestry, and monitoring of the amount of water stored in a pond.

1, 502, 702 depression 10 feature 20 three-dimensional sensor 100 measurement data 110 input/output interface 120 memory 130 processor 200 depression measurement device 201 measurement data acquisition unit 210 preprocessing unit 211 format conversion unit 212 noise removal unit 213 angle conversion unit 220 Measurement unit 221 Target region designation unit 222 Hollow identification unit 223 Volume calculation unit 230 User interface unit 231 Operation unit 232 Display unit 501 Leveled ground 502a Upper surface portion 503 Target region 701 Rough ground 903 Missing portions 904, 905 Three-dimensional maps P1, P2 Bottom point

Claims (7)

a measurement data acquisition unit that acquires point cloud data of the measurement space obtained by measuring a measurement space including at least a portion of the depression formed on the surface of the feature with a three-dimensional sensor;
a target region designating unit that designates a target region including the depression whose volume is to be obtained with respect to the point cloud data of the measurement space;
a depression identification unit that identifies the depression based on the point cloud data of the target region;
a volume calculation unit that calculates the volume of the depression based on the identified point cloud data of the depression,
The recess identification part is
Either a first specifying process of specifying the dent from leveled land in which the amount of unevenness of the surface is smaller than a preset threshold or a second specifying process of specifying the dent from uneven ground in which the amount of unevenness of the surface is equal to or greater than the threshold. A dent measuring device that selects either one to specify the dent. In the first specific process,
A process of removing a plane portion from the point cloud data of the target area;
a process of generating a plane-removed cluster by performing clustering on the point cloud data from which the plane portion has been removed;
a process of capturing point cloud data of the upper surface portion of the depression for each of the plane removal clusters;
The depression measuring device according to claim 1, wherein In the second specific process,
A process of layering the depression surface of the depression orthogonal to the depth direction of the depression with respect to the point cloud data of the target region;
Obtaining point cloud data for each layer obtained by layering the depression surface, performing clustering for each search cluster generated by sequentially adding the point cloud data for each layer from the upper layer a process of sequentially retrieving the lower layers existing in the depression plane with respect to the layer cluster;
The depression measuring device according to claim 1 or 2, wherein The volume calculation unit
If there is an occlusion that cannot be measured by the three-dimensional sensor in the recess,
After interpolating the defect interpolation point cloud obtained from the point cloud data of the dent into the defect created by the occlusion, performing a first calculation process for calculating the volume of the dent,
In the absence of said occlusion,
performing a second calculation process for calculating the volume of the depression based on the point cloud data of the depression identified by the depression identification unit;
A depression measuring device according to any one of claims 1 to 3. The volume calculation unit
The depression measuring device according to any one of claims 1 to 4, wherein at least one of height, width and depth dimensions of the identified depression is calculated. acquiring point cloud data of a measurement space obtained by measuring a measurement space including at least a portion of a depression formed on the surface of a feature with a three-dimensional sensor;
designating a target region including the dent whose volume is to be obtained with respect to the point cloud data of the measurement space;
identifying the depression based on the point cloud data of the region of interest;
calculating the volume of the depression based on the identified point cloud data of the depression;
In the step of identifying the depression,
Either a first identifying method of identifying the dent from leveled ground in which the amount of unevenness of the surface is smaller than a preset threshold value, or a second identifying method of identifying the dent from uneven ground in which the amount of unevenness of the surface is equal to or greater than the threshold value. A recess measurement method for specifying the recess by selecting either one. A recess measurement program to be executed by a computer,
A process of acquiring point cloud data of a measurement space obtained by measuring a measurement space including at least a portion of a depression formed on the surface of a feature with a three-dimensional sensor;
a process of designating a target region containing the depression whose volume is to be obtained with respect to the point cloud data of the measurement space;
a process of identifying the depression based on the point cloud data of the target area;
a process of calculating the volume of the depression based on the identified point cloud data of the depression,
In the process of identifying the recess,
Either a first identifying method of identifying the dent from leveled ground in which the amount of unevenness of the surface is smaller than a preset threshold value, or a second identifying method of identifying the dent from uneven ground in which the amount of unevenness of the surface is equal to or greater than the threshold value. A dent measurement program that selects one of them and executes processing for identifying the dent.

Images