JP7332040B2 - 3D model generation device, method and program - Google Patents

Description

TECHNICAL FIELD The present disclosure relates to a three-dimensional model generation device, generation method, and generation program for a linear structure.

Optical and telecommunication services are provided by connecting physical facilities from the customer's home to the office. In order to continue to provide stable services, there is a need to maintain and manage facilities. Until now, maintenance workers have to go to the site and visually inspect each and every one of them in order to grasp the state of communication facilities such as utility poles and cables. , have judged the quality of the equipment. In recent years, as means for efficiently diagnosing utility pole equipment, a utility pole deterioration determination technique using an inspection vehicle described in Non-Patent Document 1 is known. The technology is a three-dimensional laser scanner (three-dimensional laser survey instrument), a camera, a GPS (Global Positioning System) receiver, an IMU (Inertial Measurement unit) and an inspection equipped with an odometer (Odometer) Using a vehicle, we will drive around the city while taking three-dimensional laser scans and images, perform three-dimensional surveying of columnar structures including utility poles, and collect point cloud data including XYZ coordinates and reflection intensity. . Based on the collected point cloud data, as described in Non-Patent Document 1, a three-dimensional model of a cable, which is a linear structure, is generated using machine learning, and the relaxation of the linear structure is calculated from the three-dimensional model. It is possible to quantitatively calculate the state of the linear structure, such as the degree and height information from the road surface.

The conventional technology extracts the feature value of the local area of the acquired point cloud, calculates the similarity with the feature value of the local area of the point cloud of the cable that has been clarified in advance, and determines whether it is the point cloud to be modeled or not. is making a decision. In other words, even non-linear structures that are not to be modeled are regarded as cables in machine learning if they have a high degree of similarity, and are erroneously modeled. For example, if a part of a feature such as a road, a roof, or a rain gutter includes a linear structure, a cable model is erroneously generated for that feature. Suppression of the erroneously generated model is an issue.

M. Inoue, "3D modeling of communication cables from a laser point cloud using a rule-based approach and machine learning", IWCS 2019. Hitoshi Arakaki, Ken Tsutsuguchi, Tetsuya Kinebuchi, ``Detection of tree trunk from measured point cloud of Mobile Mapping System'', The 45th Annual Meeting of the Institute of Image Electronics Engineers of Japan, R1-5, 2017

In the present disclosure, it is intended to provide a technique for suppressing model generation of non-linear structures and generating a three-dimensional model of a cable, which is a linear structure, from a point cloud obtained using a three-dimensional laser survey instrument. aim.

In order to achieve the above object, the three-dimensional model generation device of the present disclosure includes:
A point cloud data input unit that reads point cloud data in which a structure existing in a three-dimensional space is represented by a point cloud that exists in the three-dimensional space;
The point cloud read by the point cloud data input unit is acquired, and based on the linear structure database in which the point cloud and the linear structure are linked, the acquired point cloud is matched to the linear structure. a 3D model generation unit by machine learning that identifies a point group to be measured and generates a 3D model of a linear structure using the identified point group;
The teacher data of the three-dimensional model of the linear structure and the three-dimensional model of the linear structure generated by the three-dimensional model generation unit are matched, and the non-linear structure erroneously generated by the three-dimensional model generation unit is corrected. an erroneous model detection unit that identifies a three-dimensional model;
Based on the non-linear structure database in which the point groups of the non-linear structure specified by the erroneously generated model detection unit are accumulated and the linear structure database, the point group of the non-linear structure is non-3 a dictionary update unit that generates and learns a dimensional model and updates an update dictionary database in which point groups and linear structures are linked;
a result output unit that generates a three-dimensional model of a linear structure based on the update dictionary database updated by the dictionary update unit;
It is a three-dimensional model generation device for a linear structure.

Specifically, the three-dimensional model generation method of the present disclosure includes:
A point cloud data input unit reads point cloud data representing a point cloud in which a structure existing in a three-dimensional space exists in a three-dimensional space,
A three-dimensional model generation unit based on machine learning acquires the point cloud read by the point cloud data input unit, and based on the linear structure database in which the point cloud and the linear structure are linked, the acquired points are Identifying a point group that fits the linear structure from among the groups, generating a three-dimensional model of the linear structure using the identified point group,
The erroneously generated model detection unit compares the teacher data of the 3D model of the linear structure with the 3D model of the linear structure generated by the 3D model generation unit, and detects the erroneously generated model of the 3D model generated by the 3D model generation unit. Identify a three-dimensional model of the non-linear structure with
The dictionary update unit updates the non-linear structure based on the linear structure database and the non-linear structure database in which point groups of the non-linear structure specified by the erroneously generated model detection unit are accumulated. The point group is learned by generating a non-three-dimensional model, updating the update dictionary database that links the point group and the linear structure,
A result output unit generates a three-dimensional model of a linear structure based on the updated dictionary database updated by the dictionary update unit;
This is a three-dimensional model generation method for a linear structure.

Specifically, the three-dimensional model program of the present disclosure is a program for realizing a computer as each functional unit included in the detection device of the present disclosure, and for causing the computer to execute each procedure included in the detection method of the present disclosure. program.

According to the present disclosure, a technique is provided for generating a three-dimensional model of a cable, which is a linear structure, by suppressing model generation of a non-linear structure from point clouds obtained using a three-dimensional laser survey instrument. be able to.

FIG. 4 is a diagram for explaining a method of generating a three-dimensional model of a linear structure by suppressing three-dimensional modeling of a non-linear structure from point groups acquired using a three-dimensional laser surveying instrument; 1 shows an example of a hardware configuration of a system according to the present disclosure; Example of incorrectly generated linear structure (cable) in non-linear structure It is a comparison of the point cloud tables of the dictionary DB and the update dictionary DB. FIG. 10 is a flow chart showing an example of a processing procedure and processing contents of a three-dimensional model generation unit by machine learning using an update dictionary DB; FIG. It is a flowchart which shows an example of the processing procedure and processing content of maximizing modeling precision by learning dictionary DB repeatedly.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the embodiments shown below. These implementation examples are merely illustrative, and the present disclosure can be implemented in various modified and improved forms based on the knowledge of those skilled in the art. In addition, in this specification and the drawings, constituent elements having the same reference numerals are the same as each other.

(Embodiment example)
Examples related to the present disclosure are described below. FIG. 1 explains a method executed by a device that suppresses 3D modeling of a non-linear structure from point clouds acquired using a 3D laser survey instrument and generates a 3D model of a linear structure. It is a diagram. Here, 11: point cloud database (DB), 12: point cloud, 13: point cloud data input unit, 14: 3D model automatic generation unit by machine learning, 15: dictionary DB (linear structure), 16: Point group stored in dictionary DB (linear structure), 17: 3D model stored in dictionary DB (linear structure), 18: erroneous model detection unit, 19: stored in correct DB 110: Correct answer DB 111: Point group stored in dictionary DB (non-linear structure) 112: Dictionary DB (non-linear structure) 113: Dictionary update unit 114: Update dictionary DB, 115: a result output unit.

The point cloud DB 11 contains the position coordinates (XYZ coordinates) in the three-dimensional space of all features such as columnar and linear structures acquired by a three-dimensional laser surveying instrument mounted on a vehicle, tripod, cart, unmanned aerial vehicle, etc. is a database storing a point cloud 12 having

The point cloud data input unit 13 extracts the point cloud to be analyzed from the point cloud DB, and transfers the data to the 3D model automatic generation unit 14 by subsequent machine learning. The three-dimensional model automatic generation unit 14 generates a three-dimensional model of a linear structure, that is, a cable by machine learning of the point group to be analyzed. In the machine learning dictionary DB (linear structure) 15, a point cloud 16 of a cable, which is a linear structure extracted manually in advance, and a manually created three-dimensional model 17 of the cable to fit the point cloud are stored. It is Using the point group and the 3D model, the feature amount of the point group input to the 3D model automatic generation unit 14 is extracted, point groups with a high degree of similarity are detected, and the 3D model of the cable is generated.

Next, the generated three-dimensional model of the cable is sent to the erroneously generated model detection unit 18, and the three-dimensional model 19 of the linear structure (cable) manually created in advance for the point group to be analyzed is generated. It is compared with the stored three-dimensional model of the correct answer DB 110 to identify the location where the linear structure (cable) was erroneously generated in the non-linear structure. A point group 111 around a location where a linear structure (cable) is erroneously generated in the non-linear structure and its surrounding points are stored in a dictionary DB (non-linear structure) 112 .

The point group stored in the dictionary DB (nonlinear structure) 112 and the point group and three-dimensional model stored in the dictionary DB (linear structure) 15 are sent to the dictionary updating unit 113 . The dictionary updating unit 113 learns the point group sent from the dictionary DB (non-linear structure) 112 as a non-linear structure, and the point group having high similarity with the feature amount of the local region of the point group is It is specified as not subject to 3D model generation. The point group sent from the dictionary DB (linear structure) 15 is used as a linear structure (cable) and re-learned as a target for 3D model generation as in the conventional method, and an updated dictionary DB 114 is created. A three-dimensional model of the linear structure (cable) is generated by the three-dimensional model automatic generation unit 14 based on machine learning using the update dictionary DB. The generated three-dimensional model is sent to the result output unit 115 and displayed.

FIG. 2 shows an example of the hardware configuration of the system according to the present disclosure. A system according to the present disclosure includes a computer 100 functioning as a 3D model generation device according to the present disclosure. Computer 100 may be connected to network 105 . Network 105 is a data communication network. Communication is by electronic and optical signals over network 105 .

Computer 100 includes a processor 101 and memory 102 coupled to processor 101 . Processor 101 is an electronic device made up of logic circuits that respond to and execute instructions. Memory 102 is a tangible computer 100 readable storage medium in which a computer program is encoded. In this regard, memory 102 stores data and instructions, or program code, readable and executable by processor 101 to control its operation. One of the components of memory 102 is a program module.

The program module causes the processor 101 to execute processing of arbitrary functional units provided in the detection device according to the present disclosure. For example, the program module performs the processing of the point cloud data input unit 13, the 3D model automatic generation unit 14 by machine learning, the erroneously generated model detection unit 18, the dictionary update unit 113, the result output unit 115, and the accuracy evaluation unit 56. to execute.

The program modules contain instructions for controlling processor 101 to perform the processes described herein. The program modules, although shown already loaded into memory 102 , may be configured to be located on storage device 106 for later loading into memory 102 . Storage device 106 is a tangible computer-readable storage medium that stores program modules. Alternatively, storage device 106 may be another type of electronic storage device that is connected to computer 100 via network 105 .

As explained above, the apparatus of the present disclosure can be implemented by a computer and a program, and the program can be recorded on a recording medium or provided through a network.

Hereinafter, details of each functional unit will be described. FIG. 3 is an example in which a linear structure (cable) is erroneously generated in a non-linear structure. A 3D model 17 is applied to and connected to local regions in the point cloud 12 that have a high degree of similarity with the feature values of the point cloud 16 stored in a dictionary DB (linear structure), and a 3D model automatic generation unit based on machine learning. The three-dimensional model of the cable generated in 14 is shown in solid lines. On the other hand, the three-dimensional model 19 of the manually created linear structure (cable) stored in the correct answer DB is indicated by broken lines. Comparing the two, the 3D model generated by the 3D model automatic generation unit 14 based on machine learning incorrectly generates a cable model on the ground, which is a non-linear structure. This reduces the cable modeling accuracy.

FIG. 4 shows a comparison of the point cloud tables of the dictionary DB and the update dictionary DB. The point cloud 16 stored in the dictionary DB 15 is labeled for each local region and managed by a table 31. FIG. For example, if the point cloud in a certain local area is a cable, label No. 21 given. The No. concerned A point cloud of the local region assigned 21 is recognized as a linear structure (cable) and becomes a target for generating a three-dimensional model. On the other hand, in the point group table 32 stored in the update dictionary DB 114, whether or not the model can be generated is registered for each label number. For example, label No. Since 21 is registered as being capable of model generation, it is set as a target for generation of a three-dimensional model. Label No. Since 99 is a point group of a non-linear structure, it is registered as not subject to model generation. By managing the point group, not only does the point group of the linear structure be learned to improve the accuracy like the conventional dictionary DB, but also the point group of the non-linear structure is negatively learned to obtain the original This contributes to improving the accuracy of three-dimensional modeling of the target object.

FIG. 5 is a flow chart showing an example of the processing procedure and processing contents of the three-dimensional model generation unit 14 by machine learning using the update dictionary DB. The three-dimensional model generation unit 14 based on machine learning executes steps of local shape analysis 41, three-dimensional model composing point detection 42, piecewise continuous line detection 43, and piecewise continuous line connection 44. FIG.

In the local shape analysis 41, from the 3D point group information input from the point cloud DB 11, the 3D model point groups forming the 3D model of the linear structure are grouped based on the distribution of the point groups, Extract the feature quantity of the point cloud of the local region.
In the 3D model configuration point detection 42, a local region having a high degree of similarity with the feature quantity of the linear structure or non-linear structure registered in the update dictionary DB 114 is detected from the feature quantity of the point group of the local region. .

When the detected feature quantity is a linear structure (cable), piecewise continuous line detection 43 and piecewise continuous line connection 44 are executed.
The piecewise continuous line detection 43 finds the continuous curve existing in the local area and detects a plurality of piecewise continuous lines.
A piecewise continuous line connection 44 connects a plurality of piecewise continuous lines detected by the piecewise continuous line detection 43 . The piecewise continuous line connection 44 outputs a piecewise continuous line model (three-dimensional model of a linear structure) connected from the three-dimensional model of the cable stored in the update dictionary DB 114 . The detailed operation of this process is referred to in Non-Patent Document 2.

On the other hand, if the detected feature quantity is a non-linear structure, the process ends before execution of piecewise continuous line detection 43 . When the detected feature quantity is a linear structure (cable) and a non-linear structure, the non-linear structure is preferentially processed. This prevents the non-linear structure from being mistaken for a linear structure (cable) and generating a three-dimensional model.

Next, FIG. 6 is a flow chart showing an example of processing procedures and processing contents for maximizing modeling accuracy by repeatedly learning the dictionary DB. Prepare a point cloud DB-A51, a dictionary DB (linear structure) 52, and a correct DB-A53, create a dictionary DB (non-linear structure)-A54 as shown in FIG. An updated dictionary DB-A55 is created from the (non-linear structure)-A54 and the dictionary DB (linear structure)-52. The updated dictionary DB-A55 is sent to the accuracy evaluation unit 56, and the modeling accuracy of the three-dimensional model is compared and evaluated using the dictionary DB (linear structure) 52, the point group DB-B57, and the correct DB-B58. Here, for the point cloud DB-B57 and the correct answer DB-B58, data of areas different from those of the point cloud DB-A51 and the correct answer DB-A53 are prepared. This is to exclude the point cloud used for learning from the point cloud DB-A51 and to evaluate the accuracy correctly. When the accuracy evaluation unit determines that the update dictionary DB-A55 has higher accuracy than the dictionary DB (linear structure) 52, the update dictionary DB-A55 is a dictionary DB for generating a three-dimensional model of the linear structure. Register as As shown in FIG. 1 again, the update dictionary DB-A55 uses the point cloud DB-B57 and the correct answer DB-B58 to create a dictionary DB (non-linear structure)-B59, and update the dictionary DB-B60. to update the dictionary DB. When the accuracy evaluation unit 56 determines that the accuracy of the updated dictionary DB-A 55 is low, the dictionary DB (linear structure) 52 is registered. By repeating the above flowchart, the modeling accuracy of the three-dimensional model of the linear structure is improved.

Next, the accuracy evaluation method of the accuracy evaluation unit 56 will be described. Let Ng1 be the number of three-dimensional models of linear structures (cables) generated by the point cloud DB-B 57 and the dictionary DB (linear structure) 52 . On the other hand, let Ng2 be the number of three-dimensional models of linear structures (cables) generated by the point cloud DB-B57 and the update dictionary DB-A55. Also, let Nmg be the number of three-dimensional models stored in the correct DB-B 58 . Ng1 and Nmg are compared, and the number Mg1 of which the three-dimensional model of the linear structure (cable) such as the position and length in the three-dimensional space matches is calculated. Similarly, Ng2 and Nmg are compared, and the number Mg2 of matching three-dimensional models of linear structures (cables) such as positions and lengths in the three-dimensional space is calculated.

For Ng1, Nmg, and Mg1, the precision (Precision_1) indicating the accuracy of the model and the recall (Recall_1) indicating the completeness of the model are calculated as follows to evaluate the accuracy of the model.
(Number 1)
Precision_1 [%]=Mg1/Ng1×100 (1)
(Number 2)
Recall_1 [%]=Mg1/Nmg×100 (2)

Similarly, precision (Precision_2) and recall (Recall_2) representing completeness are calculated for Ng2, Nmg, and Mg2. Relevance rate and recall rate will be explained. When the precision rate is 100%, it indicates that the number of 3D models of automatically generated linear structures (cables) is the same as the number of 3D models of manually created linear structures (cables), and the accuracy is expensive. In addition, when the recall rate is 100%, it indicates that the number of 3D models of the manually created linear structure (cable) is automatically generated without omission, and the accuracy is high. The precision is evaluated by comparing the values of Precision_1 and Precision_2, and Recall_1 and Recall_2. For example, when Ng1, Nmg, and Mg1 are 1010, 786, and 671, respectively, the precision (Precision_1) is 66.4%, and the recall (Recall_1) is 85.4%. On the other hand, when Ng2, Nmg, and Mg2 are 948, 786, and 683, respectively, the precision (Precision_2) is 72.0%, and the recall (Recall_2) is 86.9%. Therefore, since Precision_2>Precision_1 and Recall_2>Recall_1, it can be said that the update dictionary DB-A 55 has higher precision than the dictionary DB (linear structure) 52 .

If both the precision and the recall are not highly accurate, for example, if "Precision_2>Precision_1 and Recall_2<Recall_1" or "Precision_2<Precision_1 and Recall_2>Recall_1", the harmonic mean is calculated. In this case, the numerator is the same and the denominator is different as shown in formulas (1) and (2). Therefore, when calculating the average rate, the harmonic average that calculates the average of the reciprocal of the rate is used. use. The harmonic mean (F) is shown below using precision and recall.
(Number 3)
F [%] = 2/(1/relevance + 1/recall) (3)

Therefore, when calculated using the above precision and recall, the values of the harmonic mean of the dictionary DB (linear structure) 52 are precision (Precision_1) = 66.4% and recall (Recall_1) = 85.4. %, so F=74.7%. On the other hand, the harmonic mean value of the update dictionary DB-A55 is 78.8% because the precision (Precision_2)=72.0% and the recall (Recall_2)=86.9%. The F values are compared, and the dictionary DB with the larger harmonic mean is adopted.

(Effect of disclosure)
The three-dimensional model generation device, generation method, and generation program for a linear structure according to the present disclosure are considered to have the following advantages over conventional techniques.
First, in the prior art, even non-linear structures that are not subject to three-dimensional modeling are regarded as cables in machine learning if the point group has a high degree of similarity with the linear structure being learned. 3D modeling is done by mistake. In contrast, the present disclosure can suppress the three-dimensional modeling of the non-linear structure by learning that the information of the three-dimensionally modeled non-linear structure is not subject to three-dimensional modeling.
Secondly, in the conventional technology, creating a dictionary DB from scratch is costly, but the present disclosure can reduce the cost of creating a dictionary DB by performing additional learning in a point group with low accuracy in an existing dictionary DB.
Third, according to the present disclosure, after updating the dictionary database based on the information of the non-linear structure, the modeling accuracy is evaluated, and the re-learning process is repeated to maximize the modeling accuracy. can.

The present disclosure can be applied to the information and communications industry.

11: Point cloud database (DB)
12: Point cloud 13: Point cloud data input unit 14: 3D model automatic generation unit by machine learning 15: Dictionary DB (linear structure)
16: Point cloud stored in dictionary DB (linear structure) 17: 3D model stored in dictionary DB (linear structure) 18: Misgenerated model detection unit 19: Stored in correct DB 3D model 41: Local shape analysis unit 42: 3D model configuration point detection unit 43: Piecewise continuous line detection unit 44: Piecewise continuous line connection unit 56: Accuracy evaluation unit 91: 3D model generation device 100: Computer 101: Processor 102: Memory 103: Display unit 104: Storage device 105: Network 106: Storage device 110: Correct answer DB
111: Point cloud stored in dictionary DB (non-linear structure) 112: Dictionary DB (non-linear structure)
113: Dictionary update unit 114: Update dictionary DB
115: Result output unit

Claims (4)

A point cloud data input unit that reads point cloud data in which a structure existing in a three-dimensional space is represented by a point cloud that exists in the three-dimensional space;
The point cloud read by the point cloud data input unit is acquired, and based on the linear structure database in which the point cloud and the linear structure are linked, the acquired point cloud is matched to the linear structure. a 3D model generation unit by machine learning that identifies a point group to be measured and generates a 3D model of a linear structure using the identified point group;
The teacher data of the three-dimensional model of the linear structure and the three-dimensional model of the linear structure generated by the three-dimensional model generation unit are matched, and the non-linear structure erroneously generated by the three-dimensional model generation unit is corrected. an erroneous model detection unit that identifies a three-dimensional model;
Based on the non-linear structure database in which the point groups of the non-linear structure specified by the erroneously generated model detection unit are accumulated and the linear structure database, the point group of the non-linear structure is non-3 a dictionary update unit that generates and learns a dimensional model and updates an update dictionary database in which point groups and linear structures are linked;
a result output unit that generates a three-dimensional model of a linear structure based on the update dictionary database updated by the dictionary update unit;
A three-dimensional model generation device for a linear structure. An accuracy evaluation unit that compares teacher data of the three-dimensional model output by the result output unit and the three-dimensional model of the linear structure to evaluate the modeling accuracy of the three-dimensional model output by the result output unit. prepared,
Using the result of the accuracy evaluation unit, determine whether the update dictionary database is adopted or not, and update the linear structure database so that the modeling accuracy of the three-dimensional model of the linear structure is increased.
3. The linear structure three-dimensional model generating device according to claim 1. A point cloud data input unit reads point cloud data representing a point cloud in which a structure existing in a three-dimensional space exists in a three-dimensional space,
A three-dimensional model generation unit based on machine learning acquires the point cloud read by the point cloud data input unit, and based on the linear structure database in which the point cloud and the linear structure are linked, the acquired points are Identifying a point group that fits the linear structure from among the groups, generating a three-dimensional model of the linear structure using the identified point group,
The erroneously generated model detection unit compares the teacher data of the 3D model of the linear structure with the 3D model of the linear structure generated by the 3D model generation unit, and detects the erroneously generated model of the 3D model generated by the 3D model generation unit. Identify a three-dimensional model of the non-linear structure with
The dictionary update unit updates the non-linear structure based on the linear structure database and the non-linear structure database in which point groups of the non-linear structure specified by the erroneously generated model detection unit are accumulated. The point group is learned by generating a non-three-dimensional model, updating the update dictionary database that links the point group and the linear structure,
A result output unit generates a three-dimensional model of a linear structure based on the updated dictionary database updated by the dictionary update unit;
A method for generating a three-dimensional model of a linear structure. 3. A linear structure three-dimensional model generating program for realizing a computer as each functional unit provided in the linear structure three-dimensional model generating apparatus according to claim 1 or 2.

Images