WO2024031197A1 - Système et procédé de géoréférencement de marqueurs à l'aide d'un véhicule aérien et utilisation de la position géoréférencée des marqueurs pour détecter un mouvement potentiel du sol et/ou une surcroissance de végétation - Google Patents

Système et procédé de géoréférencement de marqueurs à l'aide d'un véhicule aérien et utilisation de la position géoréférencée des marqueurs pour détecter un mouvement potentiel du sol et/ou une surcroissance de végétation Download PDF

Info

Publication number
WO2024031197A1
WO2024031197A1 PCT/CA2023/051073 CA2023051073W WO2024031197A1 WO 2024031197 A1 WO2024031197 A1 WO 2024031197A1 CA 2023051073 W CA2023051073 W CA 2023051073W WO 2024031197 A1 WO2024031197 A1 WO 2024031197A1
Authority
WO
WIPO (PCT)
Prior art keywords
marker
data
markers
georeferenced
movement
Prior art date
Application number
PCT/CA2023/051073
Other languages
English (en)
Inventor
Eric Bergeron
Original Assignee
Les Systemes Flyscan Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Les Systemes Flyscan Inc. filed Critical Les Systemes Flyscan Inc.
Publication of WO2024031197A1 publication Critical patent/WO2024031197A1/fr

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/02—Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures

Definitions

  • the present invention relates to the field of aerial inspection and monitoring. More particularly, it relates to a system and a method for performing automated georeferencing and inventory of markers using an aerial vehicle and using the georeferenced position of the markers for detecting potential movement of the ground in the vicinity of one or more of the georeferenced markers, as well as overgrowth of vegetation.
  • Satellite radar imagery is however limited in terms of precision in imagery and cannot be used to precisely identify and georeference small objects such as markers used in rights-of-ways (ROW) or in other regions to be monitored. Indeed, markers are embodied using a sign installed on a post, while radar imagery can commonly only be used to see objects of a larger size, such that radar imagery cannot be efficiently used to identify objects as small as the markers commonly used in rights-of-ways (ROW) or similar locations.
  • a system for georeferencing of markers using aerial vehicle and using the georeferenced position of the markers for detecting potential movement of the ground comprises: a data capture subsystem, a marker identification module a marker georeferencing module, a marker data source and a movement detection module.
  • the data capture subsystem is mounted to an aerial vehicle and comprises mapping capture equipment acquiring mapping data representative of a region of interest as the aerial vehicle is flown over the region of interest and positioning equipment simultaneously acquiring position data relative to the position and orientation of the aerial vehicle and the components performing the acquisition of the mapping data.
  • the marker identification module is configured to receive the mapping data generated using the data capture subsystem and to identify at least portions of markers located in the region of interest, based on the received mapping data, the marker identification module generating marker identification data therefrom.
  • the marker georeferencing module is configured to receive and process the marker identification data and the position data and to georeference at least a portion of the identified at least portions of markers therefrom, the marker georeferencing module generating marker georeferenced data.
  • the marker data source receives and stores the marker georeferenced data.
  • the movement detection module is configured to receive the marker georeferenced data associated to at least one marker identified based on the mapping data and position data acquired at different points in time and compare the marker georeferenced data of the at least one marker at the different points in time to detect if one of a movement or a movement trend of the at least one marker has occurred.
  • the mapping capture equipment includes a high-definition camera and the mapping data of the region of interest includes aerial images thereof.
  • the marker georeferencing module is configured to associate pixels corresponding to the at least portion of the identified at least portions of markers to a coordinate, as part of the marker georeferenced data.
  • mapping capture equipment further includes 3D remote scanning equipment and the mapping data of the region of interest further includes a point cloud of measurements of individual measurement points along the region of interest.
  • the marker georeferencing module is configured to associate points of the point cloud corresponding to the at least portion of the identified at least portions of markers to a coordinate, as part of the marker georeferenced data.
  • the marker identification module is further configured to identify specific reference features located in the region of interest based on the received mapping data, and include the data relative to the identification of the specific reference features in the generated marker identification data
  • the marker georeferencing module is configured to acquire reference georeferenced data including GPS control points associated to the specific reference features having known GPS position and to either correct or adjust the position data prior to performing georeferencing of the at least portion of the identified at least portions of the markers or to correct or adjust the georeferencing of the a least portion of the identified at least portions of the markers as a post processing step before generating the marker georeferenced data.
  • the positioning equipment includes a high precision global navigation satellite system receiver, an altimeter and an inertial measurement unit.
  • the region of interest is one of a pipeline right-of-way, a power line right-of-way, a railway right-of-way, and a coastal road.
  • the marker identification module is configured to identify at least portions of at least one of signs mounted on support posts planted into the ground, posts to which the signs are mounted, and a section of the posts positioned at a junction of a ground surface as the at least portions of markers located in the region of interest.
  • the movement detection module is further configured to analyze the movement of at least two markers located in the vicinity of one another and to determine if a movement trend can be detected from the movement of the markers located proximate to one another.
  • the movement detection module is further configured to characterize the movement trend to associate the movement trend with a type of ground movement and determine if the type of ground movement is representative of a potential geohazard.
  • the movement detection module is further configured to detect if the movement of the at least one marker results in the at least one marker being inoperative.
  • the marker georeferencing module is configured to determine and georeference a summit for each one of the identified markers.
  • the marker identification module is further configured to detect vegetation in the vicinity of the at least one marker identified based on the mapping data
  • the marker georeferencing module is configured to determine a height of the vegetation in the vicinity of the at least one marker from a ground surface and the movement detection module is further configured to determine if the height of the vegetation is above a vegetation height threshold indicative of overgrowth of the vegetation.
  • the marker identification module is further configured to detect vegetation in the vicinity of the at least one marker identified based on the mapping data
  • the marker georeferencing module is configured to determine a relative height of the vegetation in the vicinity of the at least one marker relative to the summit of the at least one marker
  • the movement detection module is further configured to determine if the relative height of the vegetation is above a vegetation height threshold indicative of overgrowth of the vegetation.
  • a method for georeferencing of markers using aerial vehicle and using the georeferenced position of the markers for detecting potential movement of the ground comprises the steps of: simultaneously acquiring mapping data of a region of interest and position data relative to the position and orientation of an aerial vehicle and components mounted to the aerial vehicle for performing the acquisition of the mapping data, as the aerial vehicle is flown over the region of interest; receiving the mapping data and identifying therefrom markers located in the region of interest to generate marker identification data; receiving and processing the marker identification data and the position data and georeferencing therefrom at least a portion of the identified markers to generate marker georeferenced data; storing the marker georeferenced data in a marker data source; and receiving and comparing marker georeferenced data associated with at least one marker identified based on the mapping data and position data acquired at different points in time and determining therefrom if a movement or a movement trend of the marker has occurred between the different points in time.
  • the step of receiving the mapping data and identifying therefrom markers located in the region of interest comprises performing image processing using predetermined image processing algorithms, in order to identify at least a portion of the markers located in the region of interest.
  • the step of receiving and processing the marker identification data and the position data and georeferencing therefrom at least a portion of the identified marker comprises associating pixels corresponding to at least a portion of a marker to a coordinate as part of the marker georeferenced data.
  • the step of receiving the mapping data and identifying therefrom markers located in the region of interest comprises performing object detection from a point cloud using predetermined algorithms or processes, in order to identify at least a portion of the markers located in the region of interest.
  • the step of receiving the mapping data and identifying therefrom markers located in the region of interest further comprises detecting specific reference features from the mapping data of the region of interest and including the data representative of the identified reference features from the mapping data of the region of interest in the generated marker identification data.
  • the method further comprises acquiring reference georeferenced data including known position of GPS control points associated to the specific reference features found in the mapping data, from a marker data source, and the step of receiving and processing the marker identification data and the position data and georeferencing therefrom at least a portion of the identified markers to generate marker georeferenced data further includes correcting or adjusting the position data prior to performing georeferencing of the a least portion of the identified markers and generating the marker georeferenced data or to correct or adjust the georeferencing of the a least portion of the identified markers as a post processing step before generating the marker georeferenced data.
  • the step of receiving and processing the marker identification data and the position data and georeferencing therefrom at least a portion of the identified marker includes associating points of the point cloud corresponding to at least a portion of a marker to a coordinate as part of the marker georeferenced data.
  • the step of receiving the marker georeferenced data associated to at least one marker identified based on the mapping data and position data acquired at a first time period and at second time period subsequent to the first time period and comparing the marker georeferenced data of the at least one marker at the first time period with the marker georeferenced data of the at least one marker at the second time period comprises analyzing the movement of at least two markers located in the vicinity of one another and determining therefrom if a movement trend can be detected from the movement of the markers located proximate to one another.
  • the method further comprises characterizing the movement trend to associate the movement trend with a type of ground movement and determining if the type of ground movement is representative of a potential geohazard.
  • the step of receiving the marker georeferenced data associated to at least one marker identified based on the mapping data and position data acquired at a first time period and at second time period subsequent to the first time period and comparing the marker georeferenced data of the at least one marker at the first time period with the marker georeferenced data of the at least one marker at the second time period comprises detecting if the movement of the at least one marker results in the at least one marker being inoperative.
  • the step of receiving and processing the marker identification data and the position data and georeferencing therefrom at least a portion of the identified markers to generate marker georeferenced data comprises the substep of determining and georeferencing a summit for each one of the identified markers.
  • the method further comprises detecting vegetation in the vicinity of the at least one marker identified based on the mapping data, determining a height of the vegetation from a ground surface in the vicinity of the at least one marker, and determining if the height of the vegetation is above a vegetation height threshold indicative of overgrowth of the vegetation.
  • the method further comprises detecting vegetation in the vicinity of the at least one marker identified based on the mapping data, determining a relative height of the vegetation in the vicinity of the at least one marker relative to the summit of the at least one marker, and determining if the relative height of the vegetation is above a vegetation height threshold indicative of overgrowth of the vegetation.
  • Figure 1 is a schematic representation illustrating elements used by the system for georeferencing of markers using aerial vehicle and using the georeferenced position of the markers for detecting potential movement of the ground, according to an embodiment.
  • Figure 2a is a schematic representation illustrating the system for georeferencing of markers using aerial vehicle and using the georeferenced position of the markers for detecting potential movement of the ground, according to an embodiment.
  • Figure 2b is a schematic representation illustrating the system for georeferencing of markers using aerial vehicle and using the georeferenced position of the markers for detecting potential movement of the ground, according to an alternative embodiment.
  • Figure 3 is a flowchart illustrating the method for georeferencing of markers using aerial vehicle and using the georeferenced position of the markers for detecting potential movement of the ground, according to an embodiment.
  • the embodiments of the system for performing georeferencing of markers using an aerial vehicle and using the georeferenced position of the markers for detecting potential movement of the ground and/or overgrowth of vegetation using the markers as a known reference to assess the height of ground cover and any obstruction caused by tree canopies, and corresponding parts thereof consist of certain components and/or architecture as explained and illustrated herein, not all of these components and/or configuration of the architecture are essential and thus should not be taken in their restrictive sense.
  • steps of the method for performing georeferencing of markers using an aerial vehicle and using the georeferenced position of the markers for detecting potential movement of the ground and/or overgrowth of vegetation described herein may be performed in the described order, or in any suitable order.
  • steps of the proposed method are implemented as software instructions and algorithms, stored in computer memory and executed by processors. It should be understood that servers and computers are therefore required to implement to proposed system, and to execute the proposed method. In other words, the skilled reader will readily recognize that steps of the method can be performed by programmed computers.
  • some embodiments are also intended to cover program storage devices, e.g., digital data storage media, which are machine or computer readable and encode machine-executable or computer-executable programs of instructions, wherein said instructions perform some or all of the steps of said above-described methods.
  • the embodiments are also intended to cover computers programmed to perform said steps of the above-described methods.
  • Computer devices are generally part of “systems” and include processing means, such as microcontrollers and/or microprocessors, CPUs or are implemented on FPGAs, as examples only.
  • the processing means are used in combination with storage medium, also referred to as “memory” or “storage means”.
  • Storage medium can store instructions, algorithms, rules and/or data to be processed.
  • Storage medium encompasses volatile or non-volatile/persistent memory, such as registers, cache, RAM, flash memory, ROM, as examples only.
  • the type of memory is of course chosen according to the desired use, whether it should retain instructions, or temporarily store, retain or update data.
  • each such computing device typically includes a processor (or multiple processors) that executes program instructions stored in the memory or other non-transitory computer-readable storage medium or device (e.g., solid-state storage devices, disk drives, etc.).
  • the various functions, modules, services, units or the like disclosed hereinbelow can be embodied in such program instructions, and/or can be implemented in applicationspecific circuitry (e.g., ASICs or FPGAs) of the computing devices.
  • ASICs applicationspecific integrated circuitry
  • FPGAs field-programmable gate arrays
  • a computer system can be a cloud-based computing system whose processing resources are shared by multiple distinct business entities or other users.
  • any block diagrams herein represents conceptual views of illustrative circuitry embodying the principles disclosed herein.
  • any flow charts and transmission diagrams, and the like represent various processes which may be substantially represented in computer readable medium and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
  • connection or coupling refer herein to any structural or functional connection or coupling, either direct or indirect, between two or more elements.
  • connection or coupling between the elements can be acoustical, mechanical, optical, electrical, thermal, logical, or any combinations thereof.
  • the expression “based on” is intended to mean “based at least partly on”, that is, this expression can mean “based solely on” or “based partially on”, and so should not be interpreted in a limited manner. More particularly, the expression “based on” could also be understood as meaning “depending on”, “representative of”, “indicative of”, “associated with” or similar expressions.
  • modules, data sources and other components of the system described herein can be in data communication through direct communication such as a wired connection or via a network allowing data communication between computing devices or components of a network capable of receiving or sending data, which includes publicly accessible networks of linked networks, possibly operated by various distinct parties, such as the Internet, private networks (PN), personal area networks (PAN), local area networks (LAN), wide area networks (WAN), cable networks, satellite networks, cellular telephone networks, etc. or combination thereof.
  • PN private networks
  • PAN personal area networks
  • LAN local area networks
  • WAN wide area networks
  • cable networks satellite networks
  • satellite networks satellite networks
  • cellular telephone networks etc. or combination thereof.
  • systems and method described herein can be used to detect movement of the ground in the vicinity of one or more of georeferenced markers found in a region of interest, by initially acquiring mapping data and position data relative to the region of interest using an aerial vehicle flying over the region of interest, identifying and georeferencing markers found in the region of interest from the acquired mapping data and position data, and comparing the data relative to the georeferenced markers with previously acquired data of the georeferenced markers, for detecting changes in position in the X,Y,Z plane (i.e. ground surface position, tilt and elevation) of the georeferenced markers as indicative of potential geohazards related to movement of the ground in the vicinity of one or more of the markers.
  • X,Y,Z plane i.e. ground surface position, tilt and elevation
  • FIG. 1 , 2a and 2b there are shown schematic diagrams of the operational modules and/or subsystems/units of the system 10 for performing georeferencing of markers 16 found in a region of interest, using an aerial vehicle 12, and using the georeferenced position of the markers 16 for detecting movement of the ground 18 in the vicinity of one or more of the georeferenced markers 16.
  • the architecture includes modules and components for performing the required actions and tasks for the system 10 to operate.
  • Figures 2a and 2b show the architecture of the system 10 in accordance with two alternative embodiments shown, where different equipment is used for capturing mapping data of the region of interest as it is flown over by the aerial vehicle 12.
  • the mapping capture equipment 21 includes a high-definition camera 22 capturing images of the region of interest as it is flown over by the aerial vehicle 12, while in Figure 2b, the mapping capture equipment 21 includes 3D remote scanning equipment capturing measurements of individual measurement points along the region of interest as it is flown over by the aerial vehicle 12.
  • the system 10 is an airborne image-based inspection system 10 including a data capture subsystem 20 mounted to an aerial vehicle 12, one or more system computing devices 14, such as servers, and a marker data source 30.
  • the system computing device 14 further includes a marker identification module 40, a marker georeferencing module 50, and a movement detection module 60.
  • the marker data source 30 is any data source medium which can receive and store data thereon such as a database, a data repository, a data store, a data file, etc.
  • each module described in the present application can be implemented via programmable computer components, such as one or more physical or virtual computers comprising a processor and memory having instructions stored thereon. It is appreciated, however, that other configurations are possible.
  • the region of interest being monitored by the aerial vehicle, and for which markers are georeferenced and used to detect potential geohazard is one of a pipeline right-of-way (ROW), a power line ROW, a railway ROW, a coastal road or the likes.
  • ROW pipeline right-of-way
  • ROW power line ROW
  • railway ROW a coastal road or the likes.
  • other types of regions of interest could be monitored.
  • the aerial vehicle 12 can be any aircraft capable of flying over the region of interest, at a convenient speed and flight altitude to allow acquisition of mapping data 24 with sufficient precision to perform the subsequent post processing as described below.
  • the aerial vehicle 12 can be an helicopter, a small airplane, an unmanned aerial vehicle UAV, or the like.
  • the aerial vehicle 12 is a UAV (i.e. a drone).
  • the UAV can be any type or size of UAV flown by a local or remote pilot.
  • the data capture subsystem 20 is mounted onto the aerial vehicle 12 and includes mapping capture equipment 21 and positioning equipment 25 to allow the acquisition of mapping data 24 of the region of interest as it is flown over by the aerial vehicle 12 and the georeferencing of the acquired mapping data 24 using position data 29 measured by the positioning equipment 25, while the mapping data 24 is collected by the mapping capture equipment 21.
  • mapping capture equipment 21 includes mapping capture equipment 21 and positioning equipment 25 to allow the acquisition of mapping data 24 of the region of interest as it is flown over by the aerial vehicle 12 and the georeferencing of the acquired mapping data 24 using position data 29 measured by the positioning equipment 25, while the mapping data 24 is collected by the mapping capture equipment 21.
  • the georeferencing of the collected mapping data 24 allows the subsequent georeferencing of the markers 16 identified in the captured images.
  • the mapping capture equipment 21 includes a high-definition camera 22a capturing images of the region of interest as it is flown over by the aerial vehicle 12. Therefore, it will be understood that, in such an embodiment, the mapping data 24 of the region of interest includes aerial images thereof.
  • the high-definition camera 22 is a specialized camera optimized for aerial image capture. The high-definition camera 22 is positioned on the aerial vehicle 12 with the field of view of the camera being directed towards the ground surface18a, such that the high-definition camera 22 can capture images of the region of interest, from above, as the aerial vehicle 12 is flown over the region of interest.
  • the mapping capture equipment 21 includes 3D remote scanning equipment, such as, for example and without being limitative, a Lidar sensor or the like.
  • 3D remote scanning equipment such as, for example and without being limitative, a Lidar sensor or the like.
  • the 3D remote scanning equipment (or additional sensors) allow acquisition of measurements of individual measurement points along the region of interest as it is flown over by the aerial vehicle 12, to form a point cloud representing a 3D version of the region of interest from which the subsequent identification and georeferencing of the markers 16 can be performed by the system 10, as described in more details below.
  • the positioning equipment 25 includes the required material for collecting position data 29 required to perform accurate direct georeferencing of the collected mapping data 24 acquired by the data capture subsystem 20 (e.g., images or laser scan point clouds), and therefore connecting the collected mapping data 24 to the corresponding geographic positioning on the ground.
  • the data capture subsystem 20 e.g., images or laser scan point clouds
  • the positioning equipment 25 includes a high precision global navigation satellite system (GNSS) receiver 26, an altimeter 27 and an inertial measurement unit (IMU) 28.
  • GNSS global navigation satellite system
  • IMU inertial measurement unit
  • the GNSS receiver 26, altimeter 27 and IMU 28 respectively measure the position, the altitude and the angular movement of the aerial vehicle 12 (or components of the mapping capture equipment 21 mounted thereon), during acquisition of the mapping data 24.
  • the GNSS receiver 26, altimeter 27 and IMU 28 cooperate to repeatedly measure the true 3D coordinates (e.g. X; Y; Z) and orientation angles of the components of the mapping capture equipment 21 carried by the aerial vehicle 12 and therefore allow direct georeferencing of the collected mapping data 24.
  • true 3D coordinates e.g. X; Y; Z
  • the data capture subsystem 20 is in data communication with the system computing device 14, such that the data acquired by the data capture subsystem 20, including the mapping data 24 and position data 29, can be communicated to the system computing device 14 for subsequent processing.
  • the data acquired by the data capture subsystem 20 can be captured inflight and temporarily stored on a local storage medium of the data capture subsystem 20, to be subsequently uploaded to the system computing device 14.
  • the marker identification module 40 is configured to receive the mapping data 24 generated using the data capture subsystem 20 and to identify markers located in the region of interest based on the received mapping data 24. In other words, the marker identification module 40 is configured to detect the presence of markers in the region of interest, from the mapping data 24 acquired by the system 10. The marker identification module 40 generates marker identification data 42 representative of the markers identified and located in the region of interest shown in the mapping data 24.
  • the marker identification module 40 can be configured to perform image processing using predetermined image processing algorithms, in order to identify at least a portion of the markers 16 located in the region of interest from the received mapping data 24.
  • predetermined image processing algorithms One skilled in the art will understand that numerous image processing algorithms could be used.
  • the marker identification module 40 can use object detection algorithms for performing the identification and localization of the markers (or at least portions thereof) located in the region of interest, from the received mapping data 24, such as, for example and without being limitative, the convolutional neural networks algorithm (R-CNN, Region-Based Convolutional Neural Networks), Fast R-CNN algorithm, YOLO algorithm (You Only Look Once) algorithm, or the like.
  • object detection algorithms for performing the identification and localization of the markers (or at least portions thereof) located in the region of interest, from the received mapping data 24, such as, for example and without being limitative, the convolutional neural networks algorithm (R-CNN, Region-Based Convolutional Neural Networks), Fast R-CNN algorithm, YOLO algorithm (You Only Look Once) algorithm, or the like.
  • R-CNN convolutional neural networks algorithm
  • YOLO You Only Look Once
  • the marker identification module 40 can be configured to perform object detection from the 3D point cloud using predetermined algorithms or processes, in order to identify at least a portion of the markers 16 located in the region of interest, from the received mapping data 24.
  • predetermined algorithms or processes such as a 3D point cloud processing algorithms or processes.
  • the marker identification module 40 can use 3D object detection algorithms based on deep learning (e.g. R-CNN based 3D object detection algorithms) to find the 2D crop of the object and estimate the 3D bounding box of the object.
  • mapping capture equipment 21 could include both a high-definition camera 22 capturing images of the region of interest as it is flown over by the aerial vehicle 12 and 3D remote scanning equipment capturing measurements of individual measurement points along the region of interest as it is flown over by the aerial vehicle 12.
  • the marker identification module 40 could be configured to perform the identification the markers 16 located in the region of interest from either one of the aerial images of the region of interest and the 3D point cloud of the region of interest of the mapping data 24, or a combination of both, for example to minimize the processing power and/or processing time, to maximize the accuracy of the detection, etc.
  • the marker identification module 40 can be implemented by the marker identification module 40, in order to identify and/or localize the markers 16 located in the region of interest, from the received mapping data 24.
  • the system 10 is further configured to store reference georeference data 84 including Global Positioning System (GPS) control points associated to specific reference features 85 located on the ground surface 18a over the course of a pipeline 80 located along a corresponding ROW defining the region of interest for which the one or markers 16 are to be georeferenced.
  • GPS Global Positioning System
  • Such reference features located at the GPS control points can for example be used by a system (not shown) for performing various maintenance operations of a pipeline such as, for example and without being limitative, inspection of the pipeline 80, using a Pipeline Inspection Gauge (PIG) (not shown).
  • POG Pipeline Inspection Gauge
  • the PIG when performing maintenance of pipeline using a PIG, the PIG cannot use outside positioning data, such as GPS coordinates from a GPS signal provider, given the working environment of the PIG which includes thick walls of a metallic pipeline 80 buried underground that cannot be crossed by the GPS signal.
  • outside positioning data such as GPS coordinates from a GPS signal provider
  • it is required to generate position data defining the position of the PIG along the pipeline while the data relative to the inner surface of the pipeline is collected. Therefore, it is common to define GPS control points along the length of the pipeline which are used to correct or validate the position of the PIG determined by the positioning system of the PIG during the inspection of the length of the pipeline.
  • the known position of the PIG when it reaches the corresponding GPS control point can be used to correct the position data generated by the positioning system of the PIG during the inspection of the length of the pipeline during post processing of the data to generate accurate inspection data.
  • the GPS control points of the reference georeference data 84 are associated to the reference features 85 located on the ground at the specific known GPS coordinates and including equipment allowing determination that the PIG has reached the corresponding GPS control point during the inspection.
  • the reference features 85 can include a sensor capable of detecting the magnetic field generated by the passage of the PIG within its course along the pipeline to determine that the PIG has reached the corresponding GPS control point.
  • the reference georeferenced data 84 including the GPS control points associated to the specific reference features 85 located on the ground surface 18a over the course of the pipeline 80 can be received by the system 10 and stored in the marker data source 30.
  • the reference georeferenced data can be acquired and used by the marker georeferencing module for georeferencing of the identified markers 16.
  • the marker identification module 40 is further configured to perform the identification and localization of the specific reference features 85 located in the region of interest from either one of the aerial images of the region of interest and the 3D point cloud of the region of interest of the mapping data 24 captured by the mapping capture equipment 21 (or a combination of both).
  • the marker identification data 42 generated by the marker identification module 40 also includes the data representative of the specific reference features 85 located in the region of interest.
  • the markers 16 can include a sign portion 16b (e.g., round shaped signs or polygonal shaped signs, having specific written content), mounted on support posts 16a planted into the ground 18.
  • the marker identification module 40 can be configured to perform the identification and localization of the sign portion 16a as being a marker for the region of interest in the marker identification data 42.
  • the marker identification module 40 can be configured to perform the identification and localization of the combination of the sign portion 16b and the post 16a as being a marker 16 for the region of interest in the marker identification data 42.
  • the marker identification module 40 can be configured to perform the identification and localization of the combination of the sign portion 16b and the post 16a as being a marker 16 for the region of interest and to specifically identify and locate a section of the post 16b positioned at the junction of the ground surface 18a in the marker identification data 42.
  • the markers 16 can include an identification code on the sign portion thereof, such as, for example and without being limitative, a QR code or the like, of a size sufficient to be detected by the marker identification module 40 from the mapping data 24 including aerial images of the region of interest.
  • the identification code can include hard coded information such as, for example and without being limitative a marker ID or serial number, an operator name or ID, an initial georeferenced position of the marker, a marker installation date, etc.
  • the identification code can provide a link, a marker ID or the like, to acquire marker information associated to this specific marker 16 from a data source associated to the marker, such as, for example and without being limitative, the marker data source 30 which will be described in more details below.
  • the marker information can also include the marker ID or serial number, the operator name or ID, the installation date, a last known georeferenced position of the marker, etc.
  • the marker identification module 40 can be further configured to perform vegetation detection in order to detect the presence of vegetation in the vicinity of the markers 16. More precisely, in an embodiment, the marker identification module 40 is configured to detect the vegetation in the vicinity of each one of the markers 16, from the received mapping data 24, using the abovedescribed processes, tools, or the like.
  • the marker identification data 42 can be stored in the marker data source 30.
  • the marker georeferencing module 50 is configured to receive the marker identification data 42 from the marker data source 30.
  • the marker georeferencing module 50 can be connected to the marker identification module 40 to receive the marker identification data 42 directly from the marker identification module 40.
  • the marker georeferencing module 50 is configured to receive and process the marker identification data 42 and the position data 29, to georeference the identified markers 16. In other words, the marker georeferencing module 50 uses the combination of the data relative to the georeferenced region of interest provided by the combination of the mapping data 24 and position data 29 and the marker identification data 42 identifying the markers 16 in this region of interest, to precisely georeference at least a portion of the markers 16 and generate marker georeferenced data 52.
  • the marker georeferencing module 50 can be configured to associate pixels corresponding to at least a portion of a marker 16 (i.e. at least a portion of the sign portion 16b pr the post 16a) to a coordinate (e.g. X; Y; Z) as part of the marker georeferenced data 52.
  • the marker georeferencing module 50 is therefore configured to associate points of the point cloud corresponding to at least a portion of a marker 16 (i.e. at least a portion of the sign portion 16b pr the post 16a) to a coordinate (e.g. X; Y; Z) as part of the marker georeferenced data 52.
  • the marker georeferencing module 50 can be configured to determine and georeference a summit 16c for each one of the identified markers 16. For example and without being limitative, in the embodiment shown in Figure 1 , the marker georeferencing module 50 can be configured to determine what is the georeferenced pixel or point of the point cloud that is the highest for the associated marker 16, and use the coordinate (e.g. X; Y; Z) of that georeferenced pixel or point of the point cloud as the summit 16c for the associated marker 16.
  • the coordinate e.g. X; Y; Z
  • the marker georeferencing module 50 can be further configured to determine a height of the vegetation 19 in the vicinity of each one of the detected markers 16.
  • the height of the vegetation 19 in the vicinity of each one of the detected markers 16 can be determined as a relative height with regard to the georeferenced summit 16c of the associated marker 16, as determined by the marker georeferencing module 50.
  • the height of the vegetation in the vicinity of each one of the detected markers 16 can be a height measured from a georeferenced position of the ground surface18a.
  • the marker georeferencing module 50 can be configured to acquire the reference georeferenced data 84 from the marker data source 30 and use the known position of the GPS control points associated to the specific reference features 85 found in the mapping data 24 and previously identified therefrom by the marker identification module 40 and included in the generated marker identification data 42, to either correct or adjust the position data 29 prior to performing georeferencing of the a least a portion of the markers 16 and generating of the associated marker georeferenced data 52 or to correct or adjust the georeferencing of the a least a portion of the markers 16 as a post processing task before generating of the associated marker georeferenced data 52, to ensure an optimal and precise georeferencing of the a least a portion of the markers 16.
  • the marker georeferenced data 52 including at least a subset of the markers 16 identified in the region of interest and their corresponding georeferenced position is stored in the marker data source 30.
  • the data from the marker data source 30 can be used to provide a list of the markers 16 of the region of interest and their precise georeferenced position.
  • the list of the markers 16 of the region of interest can be used by a marker inventory system to provide an inventory of the markers 16 of the region of interest, along with the relevant data/information for the corresponding markers (marker ID, location, state of the marker, etc.). For instance, this can be used by the operator of a pipeline, a railway, a electric network, or the like, to maintain a marker inventory.
  • the marker inventory system and/or the list of the markers of the region of interest and their precise georeferenced position can be used by he operator of the pipeline, the railway, the electric network, or the like in order to know the precise positioning of the markers 16 along a corresponding ROW and to guide manual operations where inspectors are required to visit the site of each marker 16 and perform visual inspection of the markers 16 to determine if the marker conforms with regulatory standards.
  • regulatory standards can include identification of the type of pipeline being visible (gas, liquid, etc.), emergency phone numbers being visible, etc.
  • the movement detection module 60 is configured to receive the marker georeferenced data 52 associated to at least one marker 16 identified based on the mapping data 24 and position data 29 acquired using the data capture subsystem 20 at different points in time and to compare the marker georeferenced data 52 of the at least one marker 16 to determine if a movement or a movement trend of the marker 16 has occurred during the different points in time (i.e. to determine if a marker 16 is moving over time).
  • the movement detection module 60 can be used to perform comparison of the georeferenced positioning of at least portions of markers 16 with previously acquired data, in order to determine if a movement or movement trend of at least a subset of the markers 16 is detected, which could be indicative of a movement of the ground 18.
  • the movement detection module 60 can be configured to receive the marker georeferenced data 52 associated to at least one marker 16 identified based on the mapping data 24 and position data 29 acquired using the data capture subsystem 20 at a first time period and at second time period subsequent to the first time period.
  • the movement detection module 60 can therefore compare the marker georeferenced data 52 of the at least one marker 16 at the first time period with the marker georeferenced data 52 of the at least one marker at the second time period, to determine if a movement of the marker 16 has occurred between the first time period and the second time period.
  • marker georeferenced data 52 of the at least one marker 16 for additional time periods can also be provided (e.g.
  • the different time periods can be spaced of 1 day, 1 week, 1 month, 1 year, etc.
  • the time difference between each one of the time periods can be different (i.e. the time periods need not be equally distributed over time).
  • the marker movement detection module 60 in order to compare the marker georeferenced data 52 of the at least one marker 16 at the first time period with the marker georeferenced data 52 of the at least one marker at the second time period, to determine if a movement of the marker 16 has occurred between the first time period and the second time period.
  • the movement detection module 60 can further be configured to analyze the movement of at least two markers 16 located in the vicinity of one another, to determine if a movement trend can be detected from the movement of the markers 16 located proximate to one another. In the case where a movement trend is detected, the movement detection module 60 can be configured to characterize the movement trend and associate the movement trend with a type of ground movement. In an embodiment, the movement detection module 60 can further be configured to determine if the type of ground movement is representative of a potential geohazard.
  • the movement detection module 60 can include a movement trend determination model 62 performing detection and/or characterization of movement trends, from the detected movements of the at least two markers located in the vicinity of one another.
  • the movement trend determination model 62 can be a machine learning model stored on a computer-readable memory and trained using a labelled dataset comprising movement trend data and returning trend detection/characterization data 64 being used by the movement detection module 60 to identify/characterize the movement trends.
  • One skilled in the art will understand that, in alternative embodiments, other algorithms, processes or the like could be used by the movement detection module 60 to identify and/or characterize the movement trends of the at least two markers 16 located in the vicinity of one another.
  • the movement detection module 60 can be configured to compare the georeferenced section of the markers 16 in the marker georeferenced data 52 corresponding to the section of the post 16a of the marker 16 positioned at the junction of the ground surface 18a to detect movements or movement trends of the posts 16a of the marker 16.
  • the movement detection module 60 can further be configured to perform different potential anomaly detection based on detected movements or movement trends of the at least one marker 16 between the at least two periods of time.
  • the movement detection module 60 can further be configured to detect if the movement of the at least one marker 16 results in the at least one marker being inoperative, e.g., the at least one marker 16 has fallen and is now laying on the ground 18 (i.e. a marker 16 was in an upright orientation at a first time period and is laying on the ground surface 18a at a subsequent time period), or if a maker 16 is now missing (i.e. a marker 16 was present at a first time period and is absent at a subsequent time period).
  • the movement detection module 60 can also be configured to determine if the vegetation height of the vegetation 19 associated to the at least one marker 16 (as determined by the marker identification module 40) is above a predetermined vegetation height threshold indicative or vegetation overgrowth, i.e. the vegetation 19 should be cut or trimmed, for example and without being limitative, to avoid visual obstruction of the marker 16 by the adjacent vegetation, which can negatively impact aerial inspection thereof.
  • FIG. 3 a flowchart illustrating an embodiment of the steps of a computer-implemented method 100 for performing georeferencing of markers using an aerial vehicle and using the georeferenced position of the markers for detecting potential movement of the ground, using the above-described system 10, is shown. It will be understood that the steps of the method described below can include the alternatives and variations described in connection with the corresponding elements described above in the performance of these steps, even though these alternatives and variations are not repeated herein.
  • the method 100 includes an initial step 102 of simultaneously acquiring mapping data of the region of interest and position data relative to the position and orientation of the aerial vehicle and components performing the acquisition of the mapping data.
  • the mapping data of the region of interest can include aerial images thereof, measurements of individual measurement points along the region of interest or a combination thereof.
  • the position data can include the position, the altitude and the angular movement of the aerial vehicle (or components of the mapping capture equipment mounted thereon), during acquisition of the mapping data.
  • the method includes the further step 104 of receiving the mapping data generated using the data capture subsystem and to identify therefrom markers located in the region of interest and generate marker identification data including data representative of the identified markers located in the region of interest.
  • step 104 includes the substep of performing image processing using predetermined image processing algorithms, in order to identify at least a portion of the markers located in the region of interest.
  • step 104 includes the substep of performing object detection from the 3D point cloud using predetermined algorithms or processes, in order to identify at least a portion of the markers located in the region, from the received mapping data.
  • step 104 includes the substep of detecting vegetation in the vicinity of each one of the markers, from the received mapping data.
  • step 104 includes the substep of detecting specific reference features from the mapping data of the region of interest and include the data representative of the identified reference features from the mapping data of the region of interest in the generated marker identification data.
  • the method 100 further includes the step 106 of receiving and processing the marker identification data and the position data, to georeference at least a portion of the identified markers and generate marker georeferenced data.
  • step 106 includes the substep of associating pixels corresponding to at least a portion of a marker to a coordinate, as part of the marker georeferenced data.
  • step 106 includes the substep of associating points of the point cloud corresponding to at least a portion of a marker to a coordinate, as part of the marker georeferenced data.
  • step 106 includes the substep of determining and georeferencing a summit for each one of the identified markers.
  • the method 100 includes the further step of acquiring reference georeferenced data including known position of GPS control points associated to the specific reference features found in the mapping data, from a marker data source, and step 106 further includes correcting or adjusting the position data prior to performing georeferencing of the a least portion of the identified markers and generating the marker georeferenced data or to correct or adjust the georeferencing of the a least portion of the identified markers as a post processing step before generating the marker georeferenced data.
  • step 106 further includes the substep of determining a height of the vegetation in the vicinity of the marker.
  • the height of the vegetation in the vicinity of each one of the detected markers is determined as a relative height with regard to the georeferenced summit of the associated marker.
  • the height of the vegetation in the vicinity of each one of the detected markers is a height from the ground surface.
  • the method 100 further includes the step 108 of storing in a marker data source the marker georeferenced data including at least a subset of the markers identified in the region of interest and their corresponding georeferenced position.
  • the method 100 further includes the step 110 of determining if a movement or a movement trend of the marker has occurred, using the marker georeferenced data associated to at least one marker identified based on the mapping data and position data acquired at different points in time.
  • step 110 includes the substeps of receiving the marker georeferenced data associated to at least one marker identified based on the mapping data and position data acquired at a first time period and at second time period subsequent to the first time period and comparing the marker georeferenced data of the at least one marker at the first time period with the marker georeferenced data of the at least one marker at the second time period.
  • step 110 includes the further substep of analyzing the movement of at least two markers located in the vicinity of one another and determining if a movement trend can be detected from the movement of the markers located proximate to one another.
  • the method can include the further substep of characterizing the movement trend to associate the movement trend with a type of ground movement and determine if the type of ground movement is representative of a potential geohazard.
  • the substeps of analyzing the movement of at least two markers located in the vicinity of one another and/or characterizing the movement trend to associate the movement trend with a type of ground movement and determine if the type of ground movement is representative of a potential geohazard can include using a machine learning model stored on a computer-readable memory and trained using a labelled dataset comprising movement trend data and returning trend detection/characterization data to identify/characterize the movement trends.
  • the method further includes a step determining if a vegetation height associated to at least one marker is above a vegetation height threshold indicative of overgrowth of the vegetation.
  • the method further includes a step of detecting if the movement of the at least one marker results in the at least one marker being inoperative, e.g., if the at least one marker has fallen and is now laying on the ground or if a previously identified maker is now missing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un système de géoréférencement de marqueurs et utilisant la position géoréférencée des marqueurs pour détecter un mouvement potentiel du sol, comprenant un sous-système de capture de données monté sur un véhicule aérien et comprenant un équipement de capture de mappage et un équipement de positionnement pour acquérir respectivement des données de mappage et des données de position lorsqu'ils se déplacent sur une région d'intérêt, un module d'identification de marqueurs configuré pour identifier des marqueurs situés dans la région d'intérêt à partir des données de mappage, un module de géoréférencement de marqueurs pour géoréférencer au moins une partie du marqueur identifié, une source de données de marqueurs pour recevoir et stocker les données géoréférencées de marqueurs, et un module de détection de mouvement pour recevoir les données géoréférencées de marqueurs associées aux marqueurs identifiés sur la base des données de mappage et des données de position acquises à différents instants et comparer les données géoréférencées de marqueurs pour détecter un mouvement ou une tendance de mouvement des marqueurs.
PCT/CA2023/051073 2022-08-11 2023-08-11 Système et procédé de géoréférencement de marqueurs à l'aide d'un véhicule aérien et utilisation de la position géoréférencée des marqueurs pour détecter un mouvement potentiel du sol et/ou une surcroissance de végétation WO2024031197A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202263371162P 2022-08-11 2022-08-11
US63/371,162 2022-08-11

Publications (1)

Publication Number Publication Date
WO2024031197A1 true WO2024031197A1 (fr) 2024-02-15

Family

ID=89850121

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CA2023/051073 WO2024031197A1 (fr) 2022-08-11 2023-08-11 Système et procédé de géoréférencement de marqueurs à l'aide d'un véhicule aérien et utilisation de la position géoréférencée des marqueurs pour détecter un mouvement potentiel du sol et/ou une surcroissance de végétation

Country Status (1)

Country Link
WO (1) WO2024031197A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170084037A1 (en) * 2015-09-17 2017-03-23 Skycatch, Inc. Generating georeference information for aerial images
US20180239031A1 (en) * 2015-08-13 2018-08-23 Propeller Aerobotics Pty Ltd Integrated visual geo-referencing target unit and method of operation
US20190054937A1 (en) * 2017-08-15 2019-02-21 Bnsf Railway Company Unmanned aerial vehicle system for inspecting railroad assets
US20210055417A1 (en) * 2019-08-23 2021-02-25 Cnh Industrial America Llc Methods for generating treatment prescriptions based on uav-derived plant height data and related crop management systems

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180239031A1 (en) * 2015-08-13 2018-08-23 Propeller Aerobotics Pty Ltd Integrated visual geo-referencing target unit and method of operation
US20170084037A1 (en) * 2015-09-17 2017-03-23 Skycatch, Inc. Generating georeference information for aerial images
US20190054937A1 (en) * 2017-08-15 2019-02-21 Bnsf Railway Company Unmanned aerial vehicle system for inspecting railroad assets
US20210055417A1 (en) * 2019-08-23 2021-02-25 Cnh Industrial America Llc Methods for generating treatment prescriptions based on uav-derived plant height data and related crop management systems

Similar Documents

Publication Publication Date Title
US11378718B2 (en) Unmanned aerial vehicle system and methods
Suh et al. Mapping hazardous mining-induced sinkhole subsidence using unmanned aerial vehicle (drone) photogrammetry
Kong et al. Development and application of UAV-SfM photogrammetry for quantitative characterization of rock mass discontinuities
KR101933216B1 (ko) 드론과 공간정보를 이용한 하천지형정보 생성 방법
RU2669656C2 (ru) Обнаружение состояния с использованием обработки изображений
US20180160080A1 (en) Wide area intermittent video using non-orthorectified feature matching in long period aerial image capture with pixel-based georeferencing
US20180218214A1 (en) Condition detection using image processing
US20190011920A1 (en) Method and system for generating flight plan of unmanned aerial vehicle for aerial inspection
US20210221506A1 (en) Unmanned aerial vehicle system and methods
KR100795396B1 (ko) 항공레이저 데이터와 수치정사영상을 이용한 도시 변화모니터링 방법
US10740608B2 (en) Wide area intermittent video using non-orthorectified feature matching in long period aerial image capture with pixel-based georeferencing
Calantropio et al. UAV strategies validation and remote sensing data for damage assessment in post-disaster scenarios
Forsmoo et al. Structure from motion photogrammetry in ecology: Does the choice of software matter?
Stone et al. Alternatives to LiDAR-derived canopy height models for softwood plantations: a review and example using photogrammetry
Flores-de-Santiago et al. Assessing the effect of flight altitude and overlap on orthoimage generation for UAV estimates of coastal wetlands
Jeon et al. A real-time drone mapping platform for marine surveillance
Mirko et al. Assessing the Impact of the Number of GCPS on the Accuracy of Photogrammetric Mapping from UAV Imagery
Zhang et al. Implementing remote-sensing methodologies for construction research: An unoccupied airborne system perspective
WO2023150888A1 (fr) Système et procédé de lutte contre les incendies et de localisation de points chauds d'un feu de végétation
Li et al. Evaluating the potentiality of using control-free images from a mini Unmanned Aerial Vehicle (UAV) and Structure-from-Motion (SfM) photogrammetry to measure paleoseismic offsets
Tarigan et al. Mapping a volcano hazard area of Mount Sinabung using drone: preliminary results
Tampubolon et al. UAV data processing for rapid mapping activities
Blanks UAS applications
Ross et al. Urban breeding gull surveys: a review of methods and options for survey design
WO2024031197A1 (fr) Système et procédé de géoréférencement de marqueurs à l'aide d'un véhicule aérien et utilisation de la position géoréférencée des marqueurs pour détecter un mouvement potentiel du sol et/ou une surcroissance de végétation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23851159

Country of ref document: EP

Kind code of ref document: A1