KR20230138108A - Drone image analysis method using open source - Google Patents

Abstract

The drone image analysis method using open source to solve the above technical challenges is a method of using drones, which are attracting attention as a key field in the era of the 4th Industrial Revolution, using traditional photogrammetry technology and image analysis technology in the information and communication field ( This is about a drone image analysis method using open source, including the step of performing SfM (Structure-from-Motion)-based 3D image analysis through the development and convergence of CV (Computer Vision).

Description

Drone image analysis method using open source}

The present invention relates to a drone image analysis method using open source.

Recently, spatial information in the era of the 4th Industrial Revolution is a medium for connecting reality and virtuality that serves as cyber infrastructure, and is the core of convergence and integration, and is used for augmented reality (AR), autonomous driving, smart-construction, and smart cities. City), a technological revolution based on the implementation of Digital Twin Space (DTS), where technological boundaries between industries are disappearing and converging, moving from 'simple spatial information' based on surveys and maps centered on the location and properties of past objects to the entire space. It is expanding into ‘converged spatial information’ that recognizes comprehensive situations. The method of acquiring spatial information was collected through remote sensing technology of satellites and manned aircraft, but the method of constructing and utilizing spatial information using UAV (Unmanned Aerial Vehicle), which has recently been attracting attention as a core field of the 4th Industrial Revolution, was used. There is great interest. The penetration rate of UAVs is gradually increasing due to improvements in platform and mounted sensor-related technologies, diversification, miniaturization, and flexible market price formation compared to performance. In particular, the relaxation of UAV-related laws and regulations, the promotion of industry revitalization policies, and the publication of various research cases in various academic societies. With the introduction of , UAV applications are spreading in all directions, not only in the academic field but also in the industrial field. UAV-based remote sensing data is advantageous compared to existing methods in terms of economic feasibility by repeatedly providing high spatial and temporal high-resolution data according to the requester's conditions for natural environment characteristics that were impossible to identify due to the resolution limitations of images acquired through remote sensing technology in the past. In addition, applied research using UAV is effectively used in plant ecology analysis, geospatial information construction, environmental and disaster disaster monitoring, and archeology. Images acquired by UAV, a short-range aerial photogrammetry platform, are obtained using multi-image stereoscopic technology (MVS; Multi). With the development of computer imaging technology such as View Stereopsis, images acquired by shooting from various angles can be overlapped to model them in three dimensions and create a point cloud. However, when interpreting UAV images, image processing, interpretation, and data are acquired through expensive foreign commercial software, which imposes a significant economic burden on the user. Therefore, as an alternative system to expensive commercial software is required, the use of cost-free open-source software is expected to be effectively utilized in the development of the domestic UAV and spatial information industries in the future. Open source software (OSS; Open Source Software) that uses publicly available source code is software in which source codes that use various algorithms are shared publicly, allowing anyone to use the code without any particular restrictions. This is in contrast to proprietary software, where access to and modification of the source code is impossible, and one software is completed through the collaboration of multiple developers through Free Open Source Software (FOSS). Additionally, users and developers can build a community to ensure software stability and updates through feedback and improvements. Since it costs nothing, it is economical and can be freely modified and distributed, so various results can be derived depending on the user's purpose and usage method, and it can be applied to various industrial fields, providing excellent scalability and versatility. The purpose of this study is to analyze images acquired by UAV with various SfM-based open source image analysis tools and commercial image analysis tools, and to examine the effectiveness of UAV image interpretation using open source, not limited to commercial, through mutual comparison and analysis. This will relieve the economic burden and contribute to the development of the spatial information industry.

1.2 Research trends

1.2.1 Domestic research trends

The research trends on high-density point group generation and orthoimage production using UAV-based images in various domestic industrial fields are as follows. Geun-Hwan Lee (2011) analyzed the feature point extraction algorithms SURF (Speeded Up Robust Features) and SIFT (Scale-Invariant Feature Transform) and presented an image information processing system that enables real-time location estimation in small unmanned aerial vehicles. Kim Seok-gu (2014) used a fixed-wing unmanned aerial vehicle to acquire spatial information of various regions and verified the possibility of building a spatial information system using UAV through a post-processing process. In particular, applicability was studied through experiments in the field of cadastral spatial information, among other fields. Subong Lim (2016) conducted research on updating digital maps and building spatial information using UAVs and summarized the basic theories and technologies of UAV aerial photogrammetry. In particular, it was confirmed that 3D point clouds of various terrains were created using SIFT technology, which performs image registration, and SfM technology, which reconstructs camera posture and image geometry through bundle adjustment, and that spatial information was constructed, DEM, and orthoimages were produced. . Jinhwan Park (2017) performed VRS surveys on small sloped areas with relatively low accuracy compared to flat areas, and then used the rotary wing unmanned aerial vehicle Inspire 1 to automatically match images using the SfM-based SIFT technique. Orthoimages and DEMs were produced using Agisoft's Photoscan software, and the feasibility of rapid updating and construction of spatial information was evaluated. Park Chang-soo (2018) acquired vertical and horizontal image data of the research site using a rotary-wing UAV through the creation of a 3D realistic model of campus facilities using unmanned aerial images, and used ground control points to obtain 3-D realistic models of campus facilities through aerial triangulation and image registration. A study was conducted to produce a 3D realistic model and verify location accuracy.

1.2.2 National and international research trends

Research using overseas UAVs began with monitoring of vegetation and environment in order to efficiently manage flat and large areas of land due to the characteristics of the country where the researcher belongs. Johnson et al (2003) conducted a study to calculate the area ratio of vegetation and soil by acquiring 20cm high-resolution images using a fixed-wing small UAV in a vineyard in California, USA, and Lelong et al (2008) used a digital camera The normalized vegetation index (NDVI) was calculated by acquiring images in four bands, including modified RGB (visible) and BG-NIR (near infrared).

Recently, the scope of research has been expanded through the convergence of technologies in the UAV and computer vision fields. Furukawa et al (2010) used the SfM technique on images captured by UAV to generate an initial point cloud, recalculate it using the square patch method, and then A study was conducted to generate a high-density point cloud using the -View Stereo) technique. JOskar (2014) extensively reviewed research trends on the characteristics and application areas of software and presented future prospects through quality analysis of open source software developed through Github. SBrocks (2016) installed two digital cameras in a barley field at an altitude of 10 m to acquire image data, extracted high-density point clouds through commercial software Agisoft's Photoscan and open source software VisualSFM. A study to monitor plant growth fluctuations every two months. was carried out. Simone Bianco (2018) conducted a study to analyze pipelines and algorithms and reproduce 3D models using various SfM-based open sources using photos of various objects taken with a digital camera. Accordingly, based on the implications of existing research, this study presented the effectiveness of an open source image analysis tool using UAV by comparing images acquired by UAV with SfM-based open source and commercial image analysis tools.

1.3 Research scope and method

The spatial scope of this study is the Caisson 24 Cafe structure located in Songdo-dong, Yeonsu-gu, Incheon, and the temporal scope is 4:20 PM to 4:37 PM on April 18, 2018. The technical scope covers the process of unmanned aerial photogrammetry using UAVs and the process of constructing spatial information using various open source image analysis tools and commercial image analysis tools based on SfM. The research method first examines the pipeline algorithm for each step of image analysis, selects an open source image analysis tool that can efficiently build spatial information, and selects a research site capable of acquiring 3D spatial information. Afterwards, the coordinates of the ground control point (GCP) and inspection point (CP) were observed through Network-RTK surveying through a GNSS receiver around the target site, and by complying with the National Geographic Information Institute's “Public Surveying Work Guidelines using Unmanned Aerial Vehicles”, DJI's popular rotary wing UAV Acquire RGB image data of the target site through Phantom 4 Pro. Using the acquired data, an SfM-based commercial image analysis tool (Pix4Dmapper) and various open source image analysis tools, reproduce a 3D model in the form of a point cloud containing 3D spatial information of the target site and produce an orthoimage to determine mutual reproducibility and distortion. We examine the effectiveness of image interpretation using UAV-based open source by comparing accuracy.

[1] Geun-Hwan Lee, 2011, “Development of an image information processing system suitable for small unmanned aerial vehicle location estimation”, Master’s thesis, Konkuk University [2] Seok-gu Kim, 2014, “Study on construction and use of spatial information using unmanned flight system”, doctoral thesis, Mokpo National University [3] Soobong Lim, 2016, “Generating spatial information and assessing accuracy using unmanned aerial photogrammetry”, PhD thesis, Chungnam National University [4] Jinhwan Park, 2017, “Production of orthoimagery and digital elevation model of small slope area using low-end unmanned aerial vehicle”, Master’s thesis, Kyungpook National University [5] Changsoo Park, 2018, “Creating 3D realistic models of campus facilities using unmanned aerial images”, Master’s thesis, Korea National University of Transportation

The technical problem to be solved through embodiments of the present invention relates to drone image interpretation using open source, and more specifically, to analyze objects through automatic flight of drones, which are currently being used in the construction, civil engineering, and safety diagnosis industries. The purpose is to perform 3D modeling using open source data acquired from 3D location information and aerial images.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

The drone image analysis method using open source to solve the above technical challenges is about a drone image interpretation method using open source. More specifically, it is a method of analyzing drone images using open source. More specifically, it is a method of analyzing drone images using open source. It includes the stage of conducting SfM (Structure-from-Motion)-based 3D image analysis through the development and convergence of traditional photogrammetry technology and image analysis technology (CV; Computer Vision) in the information and communication field.

According to the embodiments of the present invention described above, it relates to a drone image analysis method using open source, and more specifically, three-dimensional location information of an object through automatic flight of a drone that has recently been used in the construction, civil engineering, and safety diagnosis industries. It has the effect of interpreting drone images by performing 3D modeling using open source data acquired from aerial images.

1 is a diagram showing a research flow according to an embodiment of the present invention.
Figure 2 is a diagram showing collinear conditions according to an embodiment of the present invention.
Figure 3 is a diagram showing coplanar conditions according to an embodiment of the present invention.
Figure 4 is a diagram showing conjugate geometric constraints according to an embodiment of the present invention.
Figure 5 is a diagram showing a Scale Invariant Feature Transform (SIFT) technique according to an embodiment of the present invention.
Figure 6 is a diagram showing the RANSAC algorithm according to an embodiment of the present invention.
Figure 7 is a diagram showing the RANSAC algorithm execution sequence according to an embodiment of the present invention.
Figure 8 is a diagram showing a Structure-from-Motion pipeline according to an embodiment of the present invention.
Figure 9 is a diagram showing the Structure-from-Motion principle according to an embodiment of the present invention.
Figure 10 is a diagram showing the Network-RTK VRS method according to an embodiment of the present invention.
Figure 11 is a diagram showing a GitHub page according to an embodiment of the present invention.

Chapter 2 Theoretical Background

2.1 Photogrammetry interpretation theory

2.1.1 Collaborative selection conditions

The collinearity condition is as shown in Figure 2, when light emitted from one point passes through the center of the camera lens and creates a subject image on film (CCD, CMOS), there are three 'subject - center of camera lens - image' This is the condition where is geometrically a straight line relationship.

Expressed in photogrammetry, an arbitrary point A existing in real-world space, the projection center L, and the photographed store A must lie on the same line.

Photogrammetry constructs the above collinear condition equation, and on the right side of the equation, a total of 9 unknowns are included, including the ground coordinates of point A (XA, YA, ZA) and the 6 external camera expression elements (Omega-Phi-Kappa, XL, YL, ZL). Obtain and determine 3D location information on the film. In order to interpret conditional expressions, ground control points (GCPs) are used to obtain external expression elements of the camera so that points on the film satisfy geometric conditions. Recently, aerial cameras have been combined with GNSS surveyors and inertial navigation systems (INS). Since six external expression elements are determined at the same time as the photo is taken, the facial expression process is simplified and the accuracy of photo interpretation is improved. Figure 3 is a linearized equation of the collinearity condition for 9 unknowns.

2.1.2 Co-existence conditions

As shown in Figure 2b, the coplanarity condition is based on a pair of overlapping two photos in a three-dimensional space in stereoscopic vision, two projection center exposure points and one target point, and the subject point and corresponding target point in the real world space. It is a traditional photogrammetry technique with the condition that a total of three left and right image points must exist on the same plane.

In Figure 3, the three vectors exist on the same plane, so the volume is “0”. thus

The coplanar condition can be expressed as follows.

In particular, the vector to cool down on top Assuming, it can be organized in the form of a product between matrices as shown in the following equation (2-4).

2.1.3 Conjugate geometric constraints

The conjugate geometric constraint condition is an image analysis theory constructed in the field of computer imaging technology that adds a constraint conjugate line to the coplanar condition, which is the basic principle of the traditional photogrammetry field. It is mainly used for real-time obstacle avoidance and detection within the field of view for robot games, autonomous vehicles, etc. It is used for cognitive 3D modeling such as object recognition. Figure 4 shows that when a pair of overlapping images are acquired at two points at different positions in three-dimensional space, the tie line resulting from the geometric relationship between the matching points (P, P') of image A and image B, respectively, is triangular. This is a condition that exists on the conjugate surface.

If a point P in three-dimensional space is projected to p in image A and to p' in image B, then the line connecting the two camera origins and the image plane meet the two points e, e' and the straight line l, l' connecting the projection point. is called the epipolar line. The conjugate line can be viewed as the intersection of the image plane and the epipolar plane connecting the three-dimensional point P and the origin of the two cameras. The conjugate lines can be determined after cross-representing the images.

2.2 Image interpretation technology

2.2.1 SIFT (Scale Invariant Feature Transform) technique

SIFT is an image processing algorithm designed by Professor Lowe at the University of British Columbia (UBC) in Canada in 2004 to extract characteristic points that are invariant to the scale and rotation of the image. The extracted feature points become basic data for the AAT (Automatic Aerial Triangulation) process for unmanned aerial vehicle image processing. In addition, the feature points converted through the algorithm are not affected by the brightness of the image or changes in the camera's shooting position, so it is an algorithm technology suitable for unmanned aerial vehicle image analysis and is widely used in AAT due to its high reliability. Feature point extraction using SIFT is performed in four steps. First, step 1 is the feature point extraction step, where preliminary feature points are found to select accurate feature points at various scales, and then stability is checked to select accurate feature points. The second stage is the location determination and filtering stage of the feature points, and among the feature points selected in the first step, unstable feature points with low contrast ratio or located at corners are removed. Afterwards, the feature points are located in continuous space using Taylor first and second order series expansion. Step 3 is the step of assigning the direction of the feature point and finding the vector quantity of the feature point to detect rotation changes in the image. The final fourth step is to describe the feature point and find the direction of the vector values around the feature point. In other words, in the 4 stages of SIFT, the first and second stages extract and select feature points, and the third and fourth stages are the process of creating a descriptor to extract the shape or characteristic information of the object. The surrounding area is centered around the selected feature points. The gradient information of the area is calculated to obtain the direction component, and the area of interest is reset to its center to create a descriptor. Figure 5 shows feature points extracted using the SIFT technique through steps 1 to 4.

2.2.2 RANSAC (Random Sample Consensus) technique

The RANSAC algorithm is an optimal feature point selection algorithm that divides multiple feature point data into inliers and outliers as shown in Figure 6 and estimates model parameters from original data with severe measurement noise.

During the sampling process, N sample corresponding points are obtained from all corresponding point candidates, viewed as true values, and the Homography Matrix is predicted to determine whether it is correct. If it is not a true value, the above process is repeatedly calculated to derive the optimal result. In this process, outliers with large errors are automatically removed. The execution sequence of the RANSAC algorithm is shown in Figure 7.

①: Sampling N feature points from all feature point candidates

②: Predict homography by assuming sampled feature points as true values

③: Predicted homography evaluation

④: If it is not a true value, repeat steps ①-③

2.2.2 SfM (Structure-from-Motion) technique

The pipeline flow chart showing the contents of each step-by-step image analysis process analyzed by combining traditional photogrammetry technology and image analysis technology in the computer information and communication field is shown in Figure 8.

Unlike traditional photogrammetry, the SfM technique reconstructs the camera's posture and image geometry without external expression elements or ground reference point coordinates. As shown in Figure 9, information such as camera type, image size, focal length, lens calibration, etc. is acquired from EXIF meta information stored in multiple duplicate photos obtained by dynamic shooting of the camera at various angles, and input an image corrected for lens distortion and main point. Use it as data. In addition, it is the latest image analysis technique that reproduces the subject in the form of a point cloud point cloud by extracting feature points for image registration and simultaneously restoring the positional relationship between the 3D shape and the camera through matching between feature points and motion estimation technology.

2.3 Network-RTK (Real Time Kinematic)

Network-RTK (Real Time Kinematic) is a real-time relative positioning technique that maintains high positioning accuracy over long-term lines regardless of the baseline distance between the reference station and the mobile station, enabling precise positioning. Currently, in Network-RTK's system, there are three methods for positioning virtual reference points: grid correction coefficient type, virtual observed value type (VRS), and FKP (plane correction coefficient). In Korea, the VRS (Virtual Reference Station) method is mainly used. In the Network-RTK positioning method, VRS is a method of setting a virtual reference station at a point adjacent to a random mobile station in the network and determining the exact location of the mobile station through RTK, as shown in Figure 10. In particular, VRS uses the existing Network-RTK GPS surveying method, which required a total of two receivers, one each for the mobile station and one for the base station, by using data from the National Geographic Information Institute's permanent observatory, allowing GPS surveying with one receiver and one device capable of Bluetooth communication. There is an advantage to this being possible.

2.4 Open source software and GitHub

Open source software refers to software in which the source code required to develop software is open to everyone without any particular restrictions. Source code is a text that describes computer program code in binary form in a programming language that can be read and interpreted by humans.

Developers develop new software based on source code released on open source software platforms, add new features to existing software, and discover and fix bugs in beta versions of software. During this process, everyone is given the right to read and modify the source code, and the final result is also freely distributed without belonging to any one person. As shown in Figure 11, GitHub is an open source software platform used by the largest number of developers in the world that implements Git, one of the version management systems, online. Developers on GitHub each have a personal page where their activities are recorded, which becomes a kind of portfolio and can be freely viewed by other developers. In particular, it is equipped with various communication functions such as SNS and MSN messenger to communicate with other developers necessary during the work process. Additionally, through the favorites function, you can receive notifications about the progress of projects of high interest, and it provides various quantification and visualization information that allows you to intuitively understand the overall information of the project.

Chapter 3. Data observation and image analysis

3.1 UAV filming flight plan establishment

As shown in Figure 3.1, the shooting location is Cason 24 Cafe located in Songdo-dong, Yeonsu-gu, Incheon (12-8 Songdo-dong). Prior to shooting, a plan for setting the shooting route is established using the Pix4Dmapper application. In the case of new cities such as Songdo-dong, Yeonsu-gu, there are many high-rise buildings and buildings, so a place with good radio reception was selected to secure visibility for UAV takeoff and landing points and flight control, and to observe ground reference points. In addition, considering the fact that there is a large floating population due to the nature of the target site, a safe landing point in case of emergency was searched and a UAV filming flight plan was established at a time when there was no influence of rain or wind due to the nature of UAVs being affected by the weather.

3.1 Research site (Caisson 24)

In addition, as shown in Figure 32 before UAV video shooting, the Ministry of Land, Infrastructure and Transport and the Korea Drone Association jointly conducted

Using the developed Ready to fly application, airspace such as nationwide no-fly zones and air traffic control areas

Check status and weather information, sunrise and sunset times, regional flight permit authorities and contact information, etc.

Check the information necessary for UAV flight. Study site: Songdo, Yeonsu-gu, Incheon Metropolitan City

In the case of consent, it was confirmed that it does not fall into a no-fly zone as shown in Figure 32.

32 Ground control point survey and equipment specifications

First, the terms necessary for detailed drawing work and 3D coordinate acquisition through UAV aerial triangulation

Anti-aircraft beacons must be installed in locations that can be visually identified from construction photos, but this study

As shown in Figure 33, a black acrylic plate is sized 30 cm wide

It was produced by attaching circular and cross-shaped paper in contrasting colors and white.

33 anti-aircraft sign

After production, open points with good radio wave reception around the target site and visibility during UAV video shooting

They were evenly distributed and installed in locations with excellent performance. In particular, precise coordinates and checkpoints

To obtain real-world coordinate results, be careful not to change or damage the ground control point location.

did. Figure 34 is the location of the ground control point observed by performing GNSS surveying.

34 Ground reference point location map

The real world of ground reference point coordinates and inspection points to be used when producing orthophotos and reproducing 3D models.

The equipment used to obtain coordinate results is Trimble's GNSS equipment R8 receiver as shown in Figure 35.

TSC3 controller was used, and the coordinate system was KGD2002's geoid model KNGeoid14.

Selected as map projection coordinates with suborigin as the origin, VRS survey room using Network-RTK method

After confirming the number of 12 satellites using the law, the receiver height was spaced and fixed at 2m, and then 10

The ground reference point coordinates and the real-world coordinates of the inspection point were obtained through point measurements in seconds.

The inspection point is the corner light of an artificial facility or natural feature that is clearly visible to the naked eye in the image.

Observe and determine coordinates. Figure 35 shows Trimble's R8 receiver, TSC3 controller, and

This is how the equipment is used.

Figure 35 R8 receiver installation and observation view

Table 31 below shows the specifications of the GNSS R8 receiver.

Table 32 below shows the specifications of the Trimble TSC3 controller.

33 UAV filming and equipment specifications

DJI's entry-level rotary wing UAV Phantom 4 Pro was used as filming equipment for the target area.

From 4:20 to 4:37 p.m. on April 18, 2018, the National Geographic Information Institute's unmanned aerial vehicle

In compliance with the public survey work guidelines for use, an overlap of 80% in the vertical and horizontal directions was applied at a flight altitude of 60 m.

A total of 110 pieces of RGB image data were acquired through automatic flight. Figure 36 is

This is the UAV equipment used to acquire the site.

36 UAV Phantom 4 Pro

Table 33 below shows the specifications of the Phantom 4 Pro.

34 Video shooting and image processing using UAV

341 Ground control and checkpoint observations

To obtain ground reference points and inspection points to be used when producing orthophotos and reproducing 3D models.

As a result of performing a precision survey, 3 ground reference points and 3 inspection points were obtained as shown in Table 34.

It was. The three acquired ground reference points are used in commercial image analysis tools and open source image analysis tools.

Input and analysis were performed to create orthoimages and 3D models.

342 Commercial image analysis tool (Pix4Dmapper)

Among the software that currently supports image analysis, it is a commonly used software.

Examples include Photoscan from Agisoft and Context Capture from Bentely. Among them, Switzerland

Pix4Dmapper, Pix4D's UAV image analysis post-processing program, is an SfM-based image analysis tool.

User convenience through a graphical user interface (GUI)

It provides a visualized interface environment that can be easily used. Also, within the program

Because lens distortion is corrected through camera calibration, a separate camera is required.

It is possible to skip the Mera test process. In addition, each camera model has a built-in correction model.

want. When entering image processing, the location of the shooting point is automatically visualized and tie-points are displayed.

And the GCP input process and photo-tagging process are convenient. When interpreting a video, first

Load 110 RGB images acquired by UAV and use them in the central coordinate system (Korea 2000/Central Belt)

2010: Enter three reference points based on EPSG 5186 and specify the location of the reference point in the image.

It was loud. After entering the real world coordinates, go to the ‘Point Cloud and Mesh’ item as shown in Figure 37.

This point cloud was created. Afterwards, run ‘DSM, Orthomosaic and Index’

Reproduce the 3D model including 3D spatial information as shown in Figure 38 and Figure 39-10.

Together, DSM and orthoimages were produced, respectively.

343 Open source (WebODM) video analysis tool

OpenDroneMap, developed by FOSS4G's project, is a drone video analysis tool.

It is an open source toolkit. Linux-based ODM, Node-ODM providing API for developers, web

Supports browser-based WebODM and points click through the same process as commercial analysis tools.

Loud and mesh work, DSM, DEM, and even orthophoto can be extracted. toolkit

Download was founded in 2008 and has 12 million users and 30.3 million posts as of 2015.

Download the source code from Github, a professional hosting company that operates the venue, and run it.

Install the required applications Docker, Python, and Git. Afterwards

Run the Docker Quickstart Terminal, an extension of Docker, and set it up as shown in Figure 311.

After entering the command, build the executable file.

After building, receive the hosting address 19216899100:8000 and access it through a web browser.

Processing the target site RGB data, a point cloud was created as shown in Figures 312-13.

After creating a point cloud, perform an absolute coordinate process that performs real-world coordinate conversion from model coordinates to ground coordinates using the ground control point, and extract pixel By inputting the variables of the PROJ4 library of the Korean coordinate system EPSG code written in C language, DSM and orthoimages were produced as shown in Figures 315 to 16.

344 Open source (COLMAP) image analysis tool

COLMAP is an open source software whose source code can be downloaded from Github.

is a general-purpose SfM and MVS pipeline with a graphical command line interface. video

Through my Sift technique, I extract feature points as shown in Figures 317-18 and image the extracted feature points.

Through geo-matching, a point cloud is constructed to reproduce the 3D model in Figure 319. this

Afterwards, the 3D model is reconstructed through densification (CMVS/PMVS) as shown in Figure 320.

Create a dense point cloud. In particular, COLMAP's user interface is

Provides access to all available features and assists in the reconfiguration process in real time.

To visualize and start the GUI, install the prebuilt package and run the Windows MS-DOS command

Enter ‘COLMAP’ in the window and run it to load the image.

An SfM-based MVS high-density point cloud was created, but the image had no spatial information.

The georeferencing function, which is a task for matching to the actual location, is not provided, so 3D

It was impossible to obtain spatial information.

345 open source (VisualSFM + CMPMVS) video analysis tool

Like COLMAP, VisualSFM also uses SfM-based images using the Sift technique to create 3D images.

It is a GUI-supported image analysis tool that reconstructs image objects into circles. Extract feature points from video

Feature point matching between images, mutual expression and bundle adjustment for 3D reproduction, high density

By integrating the image processing (PMVS/PMVS2 or CMVS) into one package, the target's 3

It is an open source that allows dimensional modeling to be reproduced. Figures 321-22 show points through the same process.

This is the process of acquiring the cloud.

VisualSFM also does not provide the same georeferencing function as COLMAP. under

However, through CMPMVS, a 3rd party program equipped with DEM and orthophoto production functions,

DEM and orthoimages were produced. In particular, CMPMVS does not provide a GUI.

Runs based on MS-DOS and works by inputting data and source code interpreted with VisualSFM

was carried out. Figure 323 is data generated by combining VisualSFM and CMPMVS.

346 Open source (Meshroom) video analysis tool

Meshroom is based on the AliceVision Photogrammetric Computer Vision framework.

It is a 3D reconstruction open source software. Point clouds and 3D preprocessing in SfM

Mesh and texture are provided, and DSM and DEM orthophoto production is not possible. Figure 324 shows

This is a point cloud obtained with Meshroom.

In particular, Meshroom provides several image analysis techniques, including the Sift method, making it easy to use.

Depending on the user's settings, various 3D reproduction results are possible.

Chapter 4 Comparison and analysis of commercial and open source image analysis tools

41 Image analysis 3D model quality comparison

411 open source video analysis tool point cloud

The more point clouds there are, the better the image quality is, using SfM-based open source.

A comparison of the number of point clouds of the high-density point cloud created is graphed as shown in Figure 41.

indicated. 117,171 points were obtained when interpreting WebODM video using the Sift technique.

87,438 points were obtained during COLMAP analysis. Also, when analyzed with VisualSFM, 88,093

Points, 91,095 points were obtained during Meshroom analysis. Therefore, among open source

WebODM's point cloud was obtained the most. Figure 42 is through open source.

This is a visualization of the acquired point cloud.

412 Open source 3D model reproducibility and distortion

Figure to compare the quality of 3D models reproduced using an open source image analysis tool.

The completeness was compared by enlarging it as shown in 43-45.

As a result of checking the quality by zooming in, the number of point clouds was larger than that of other analysis tools.

The reproducibility and distortion of the obtained WebODM were found to be superior to other open sources.

all. This used the same SIFT feature point extraction technique, but during the feature point matching process,

Difference in point cloud acquisition due to preliminary feature point filtering process of RANSAC algorithm

It is believed that the quality difference occurred due to this. Also, there is a lot of difference in side distortion.

You can see this flying with the naked eye, but due to the nature of UAV video, when shooting from the air to the ground,

It is believed that 3D reproduction is insufficient because there are blind spots on the sides.

3D model quality comparison with 413 commercial

WebODM and commercial video analysis tool with the highest quality among open sources

Visualize the 3D model reproduced with Pix4Dmapper to enlarge specific parts and compare completeness.

It was.

As a result of quality comparison, as shown in Figure 46, point noise and presence of

However, overall, the expression of roads and structures was good in both image analysis tools.

It came out. In particular, the image quality and distortion of the road section of WebODM are better than those of Pix4Dmapper.

Although it appeared well, the expression of the side parts and structures appeared to be somewhat lacking. this is

The feature point extraction and filtering process of commercial image analysis tools is superior to that of open source image analysis tools.

It is believed to have been done well.

42 Comparison and analysis of image interpretation accuracy

421 Commercial (Pix4Dmapper) image analysis tool accuracy

As shown in Table 41, 3 reference points (01, 02, 03) for the 3D reproduction model using a commercial image analysis tool.

and the values of three inspection points (c01, c02, c03) were compared and analyzed.

422 Open source (WebODM) video analysis tool accuracy

As shown in Table 41, 3 reference points (01, 02, 03) for the 3D reproduction model using a commercial image analysis tool.

and the values of three inspection points (c01, c02, c03) were compared and analyzed.

423 Comparison of accuracy of commercial and open source image analysis tools

Calculate the absolute average deviation of the X, Y, and Z axes using the deviation distribution for each inspection point in Tables 41 and 42.

Expressed as a graph in Figure 47, the analysis results show the X and Z axes of Pix4Dmapper and WebODM.

The car was the same and analyzed at 002 m on the Y axis. Through this, the convenience of commercial and open source

It was confirmed that the difference was slight, and the accuracy was more than 003m, which is the standard for cadastral boundary error.

I was able to obtain the Tao.

Overlap comparative analysis with 424 numerical maps

Figures 48-10 show orthoimages produced with UAV-based open source and commercial image analysis tools.

Realistic photorealistic video and 1:5000 scale produced from aerial photos provided by the National Geographic Information Institute (NGII)

It is compared by overlapping with the numerical map.

As a result of comparison and analysis, the information interpreted using UAV was better than the actual photorealistic image of the National Geographic Information Institute.

It was shown that the projection image consistency was good. In particular, by enlarging and analyzing the overlapping image in Figure 411,

As a result of checking the average deviation, Pix4mapper, a UAV-based commercial image analysis tool, showed an average of about

A deviation of 03m was confirmed, and WebODM, an open source video analysis tool, showed an average deviation of about 05m.

Shown the car. The average deviation of the National Geographic Information Institute is about 12m.

The reason why the matching performance was good is that the resolution (GSD) of the UAV is the resolution of the aerial photogrammetry image.

It is believed to be superior to the province. In addition, video analysis tools using commercial and open source

Confirming that the difference is small, a digital map was produced using an open source image analysis tool using UAV.

Proven ability.

Chapter 5 Conclusion

This study analyzed images acquired using UAV using SfM-based commercial and open source analysis tools.

Using , good quality 3D models and orthoimages were obtained. In particular, various open source

Results of comparing and analyzing quality and accuracy with commercial image analysis tools by utilizing and combining

The effectiveness of the open source video analysis tool was reviewed as follows.

First, among the open source video analysis tools WebODM, COLMAP, Meshroom, and VisualSFM.

WebODM's point cloud and video quality were found to be the best. SfM Young

In the Shanghai analysis process, the SIFT feature point extraction technique and the RANSAC feature point matching technique are used equally.

However, it was found that quality differences occurred due to differences in feature point filtering between each tool.

It came out. In addition, video analysis tools, excluding WebODM, generate high-density point clouds and

The definition was the same, but the absolute coordinate conversion and

It was confirmed that there are limitations in DEM, DSM, and orthophoto production.

Second, WebODM includes an SfM-based preprocessing process and geo-tagging of ground control points.

Through this process, it is possible to reproduce 3D models including 3D spatial information and produce orthoimages.

Proven. Although the quality of commercial image analysis tools was excellent, images were converted to WebODM.

3D model analysis (X:002, Y:003, Z:004) and commercial image analysis Pix4Dmapper (X:002, Y:001,

Z:004) It was confirmed that there was no significant difference in the accuracy of the 3D model, and public survey work regulations did not allow it.

It confirms the effectiveness of the open source image analysis tool by showing accuracy within 3cm, which is the secondary standard.

It was.

Third, use UAV-based open source image analysis to construct 3D spatial information and orthogonal projection

Create an image by overlapping the real-world photorealistic image and digital topographic map provided by the National Geographic Information Institute.

As a result of comparison, high consistency was confirmed. In particular, real-time appraisal images taken from aerial photographs

As a result of enlarging the quality and comparing it, the superiority of the image quality captured by the UAV was demonstrated. This

UAV image-based open source image analysis of aerial images with high costs and long update cycles

It is believed that replacing it with a tool will result in economic savings and shorten the video update cycle.

In the future, open source code will be developed by taking advantage of open source's ability to modify and redistribute.

Consideration of the image analysis pipeline and finding the optimal image analysis pipeline, compared to commercial image analysis tools.

We plan to conduct high-level quality and accuracy improvement research.

Claims (1)

Drone video analysis method using open source.