CN106204443A - A kind of panorama UAS based on the multiplexing of many mesh - Google Patents
A kind of panorama UAS based on the multiplexing of many mesh Download PDFInfo
- Publication number
- CN106204443A CN106204443A CN201610502799.3A CN201610502799A CN106204443A CN 106204443 A CN106204443 A CN 106204443A CN 201610502799 A CN201610502799 A CN 201610502799A CN 106204443 A CN106204443 A CN 106204443A
- Authority
- CN
- China
- Prior art keywords
- unit
- unmanned aerial
- aerial vehicle
- panoramic
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012937 correction Methods 0.000 claims abstract description 14
- 238000000034 method Methods 0.000 claims abstract description 14
- 230000000007 visual effect Effects 0.000 claims abstract description 14
- 230000008569 process Effects 0.000 claims abstract description 8
- 238000001514 detection method Methods 0.000 claims description 14
- 238000005259 measurement Methods 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 7
- 238000013507 mapping Methods 0.000 claims description 5
- 239000000284 extract Substances 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000000691 measurement method Methods 0.000 claims description 3
- 230000004888 barrier function Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 241000422846 Sequoiadendron giganteum Species 0.000 description 1
- 241000607479 Yersinia pestis Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 229910052500 inorganic mineral Inorganic materials 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 239000011707 mineral Substances 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Geometry (AREA)
- Remote Sensing (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Automation & Control Theory (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a kind of panorama UAS based on the multiplexing of many mesh, panorama UAS can make unmanned plane acquisition panoramic mosaic image without dead angle when flight by visual unit, to allow the control flight condition intuitively of remote manipulation user；The distance of target object distance unmanned plane can be measured by range cells；Reconstruction unit is utilized can earth's surface object or landform to be rebuild；Use hovering unit can control unmanned plane to hover in aerial so that photographing panorama picture range finding etc. operate；Use avoidance unit can avoid obstacle in unmanned plane during flying process, utilize control unit correction flight path to ensure the safety of unmanned plane during flying.
Description
Technical Field
The invention relates to the field of unmanned aerial vehicles, in particular to a panoramic unmanned aerial vehicle system based on multi-view multiplexing.
Background
Along with the improvement of science and technology and bigger market demand, unmanned aerial vehicle receives user's preference more and more, and its field of application mainly has: national ecological environment protection, mineral resource exploration, marine environment detection, land utilization investigation, water resource development, crop growth detection and estimation, agricultural operation, natural disaster detection and evaluation, urban planning and municipal management, forest pest protection and detection, public safety, national defense industry, digital earth, advertising photography and the like. However, the existing unmanned aerial vehicle has fewer functions, and due to the size, gravity and the limitation of the prior art, the unmanned aerial vehicle cannot have multiple functions, such as limited vision, automatic flight in a small space, and manual remote control of a user, so that the unmanned aerial vehicle is prevented from colliding with a wall and being damaged.
Disclosure of Invention
To the above-mentioned problem that current unmanned aerial vehicle exists, provide one kind now and aim at realizing collecting the range finding, rebuilding, hover and keep away the barrier panorama unmanned aerial vehicle system based on multi-purpose multiplexing as an organic whole.
The specific technical scheme is as follows:
the utility model provides a panorama unmanned aerial vehicle system based on multiplex many meshes, is applied to among the unmanned aerial vehicle, includes:
the vision unit comprises a plurality of cameras, the cameras are arranged on the unmanned aerial vehicle and are used for shooting a panoramic mosaic image with a visual center of 360 degrees, and partial visual angles of two adjacent cameras are overlapped;
the distance measurement unit is connected with the vision unit and acquires the distance between the unmanned aerial vehicle and a target object by adopting a binocular stereo vision measurement method;
the reconstruction unit is connected with the visual unit and used for preprocessing the panoramic stitching image to acquire three-dimensional point cloud data so as to reconstruct the terrain according to the three-dimensional point cloud data;
the hovering unit is used for controlling the unmanned aerial vehicle to hover at a space position matched with the three-dimensional space coordinate;
the obstacle avoidance unit is used for providing a flight path and a safety threshold value, detecting whether an obstacle which is away from the unmanned aerial vehicle on the flight path reaches the safety threshold value or not at regular time, and generating obstacle information and outputting the obstacle information when the obstacle reaches the safety threshold value;
and the control unit is respectively connected with the vision unit, the distance measurement unit, the reconstruction unit, the hovering unit and the obstacle avoidance unit, corrects the flight path according to the obstacle avoidance information, generates the three-dimensional space coordinate, and controls the hovering unit to execute hovering operation according to the three-dimensional space coordinate.
Preferably, the visual unit comprises:
the calibration module is used for respectively calibrating the plurality of cameras to acquire internal parameters and external parameters of each camera so as to acquire a mapping relation between coordinates of an image acquired by each camera in a world coordinate system and coordinates in an image coordinate system;
and the processing module is connected with the calibration module, respectively carries out denoising processing on each image, projects all the images to a uniform coordinate system, processes the images by adopting image interpolation values, and carries out matching and splicing on the processed images so as to obtain the panoramic spliced image.
Preferably, the ranging unit includes:
the acquisition module is used for acquiring the position coordinates of the target object;
the positioning module is used for acquiring the current position coordinate of the unmanned aerial vehicle;
and the calculation module is respectively connected with the acquisition module and the positioning module and is used for calculating to acquire the distance between the unmanned aerial vehicle and the target object according to the position coordinate of the target object and the current position coordinate of the unmanned aerial vehicle.
Preferably, the acquisition module performs binocular stereo vision measurement by controlling any two cameras adjacent to each other in the vision unit to acquire the position coordinates of the target object.
Preferably, after the panoramic stitching image is preprocessed, the reconstruction unit extracts feature point pairs in two continuous panoramic stitching images to match the images pairwise to obtain corresponding point pairs, and obtains the three-dimensional point cloud data by a triangulation method according to the corresponding points to reconstruct the terrain according to the three-dimensional point cloud data.
Preferably, the preprocessing includes denoising and/or radiation correcting the panoramic stitched image.
Preferably, the hovering unit includes:
the detection module is used for detecting characteristic points according to the panoramic mosaic image at preset time intervals so as to obtain effective characteristic points, and obtaining the three-dimensional coordinates of the unmanned aerial vehicle according to the positions of the characteristic points in the panoramic mosaic image and the positions of the characteristic points in a world coordinate system;
the judging module is connected with the detecting module and used for judging whether the difference between the three-dimensional coordinates of the two continuously acquired unmanned aerial vehicles is larger than a preset threshold value or not, and if the difference exceeds the preset threshold value, a correction instruction is output;
and the correction module is connected with the judgment module and used for correcting the current space position of the unmanned aerial vehicle according to the correction instruction.
Preferably, the method further comprises the following steps:
and the tracking unit is connected with the control unit and used for detecting the moving target, generating a tracking path according to the detection result and tracking the moving target according to the tracking path.
The beneficial effects of the above technical scheme are that:
according to the technical scheme, the visual unit can enable the unmanned aerial vehicle to obtain the panoramic mosaic image without dead angles during flying, so that a remote control user can intuitively control the flying condition; the distance between the target object and the unmanned aerial vehicle can be measured through the distance measuring unit; the reconstruction unit can be used for reconstructing the surface object or the terrain; the hovering unit is adopted to control the unmanned aerial vehicle to hover in the air so as to shoot panoramic images, measure distance and the like; the obstacle avoidance unit is adopted to avoid obstacles in the flight process of the unmanned aerial vehicle, and the control unit is used for correcting the flight route so as to ensure the flight safety of the unmanned aerial vehicle.
Drawings
Fig. 1 is a block diagram of an embodiment of a panoramic unmanned aerial vehicle system based on multi-view multiplexing according to the present invention;
fig. 2 is a schematic diagram of binocular stereo vision measurement.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
The invention is further described with reference to the following drawings and specific examples, which are not intended to be limiting.
As shown in fig. 1, a panoramic unmanned aerial vehicle system based on multi-view multiplexing is applied to an unmanned aerial vehicle, and includes:
the vision unit comprises a plurality of cameras, the cameras are arranged on the unmanned aerial vehicle and are used for shooting a panoramic mosaic image with a 360-degree visual angle by taking the unmanned aerial vehicle as a visual center, and partial visual angles of two adjacent cameras are overlapped;
the distance measurement unit is connected with the vision unit and acquires the distance between the unmanned aerial vehicle and a target object by adopting a binocular stereo vision measurement method;
the reconstruction unit is connected with the vision unit and used for preprocessing the panoramic mosaic image to acquire three-dimensional point cloud data so as to reconstruct the terrain according to the three-dimensional point cloud data;
the hovering unit is used for controlling the unmanned aerial vehicle to hover at a space position matched with the three-dimensional space coordinate;
the obstacle avoidance unit is used for providing a flight path and a safety threshold value, detecting whether an obstacle which is away from the unmanned aerial vehicle on the flight path reaches the safety threshold value or not at regular time, generating obstacle information when the obstacle reaches the safety threshold value, and outputting the obstacle information;
and the control unit is respectively connected with the vision unit, the distance measurement unit, the reconstruction unit, the hovering unit and the obstacle avoidance unit, corrects the flight route according to the obstacle avoidance information, generates a three-dimensional space coordinate, and controls the hovering unit to execute hovering operation according to the three-dimensional space coordinate.
This embodiment collects range finding, rebuilds, hovers and keeps away the barrier and has multiple functions as an organic whole, provides multiple functions for unmanned aerial vehicle's flight. The visual unit can enable the unmanned aerial vehicle to obtain the panoramic spliced image without dead angles during flying so as to enable a remote control user to intuitively control the flying condition; the distance between the target object and the unmanned aerial vehicle can be measured through the distance measuring unit; the reconstruction unit can be used for reconstructing the surface object or the terrain; the hovering unit is adopted to control the unmanned aerial vehicle to hover in the air so as to shoot panoramic images, measure distance and the like; the obstacle avoidance unit is adopted to avoid obstacles in the flight process of the unmanned aerial vehicle, and the control unit is used for correcting the flight route so as to ensure the flight safety of the unmanned aerial vehicle.
In a preferred embodiment, the visual unit comprises:
the calibration module is used for respectively calibrating the plurality of cameras to acquire internal parameters and external parameters of each camera so as to acquire a mapping relation between coordinates of an image acquired by each camera in a world coordinate system and coordinates in an image coordinate system;
and the processing module is connected with the calibration module, respectively carries out denoising processing on each image, projects all the images to a unified coordinate system, processes the images by adopting image interpolation, and carries out matching splicing on the processed images so as to obtain a panoramic spliced image.
In this embodiment, a plurality of cameras set up respectively in inorganic people's top, lateral part and bottom to obtain the image that has certain overlap degree, splice into panorama concatenation image with a plurality of images after the preliminary treatment, in order to realize obtaining bigger shooting area under the unchangeable condition of image scale, definition, reduce unmanned aerial vehicle flight number of times. The camera is calibrated through the calibration module, and the internal parameters and the external parameters of the camera are solved, so that the one-to-one mapping relation between the coordinates in the world coordinate system and the coordinates in the image coordinate system is obtained. And (3) utilizing a processing module to carry out preprocessing such as denoising and the like on the image, then projecting the image to a uniform coordinate system, carrying out operations such as image matching and the like after image interpolation operation, and finally obtaining a seamless panoramic stitching image.
In a preferred embodiment, the ranging unit includes:
the acquisition module is used for acquiring the position coordinates of the target object;
the positioning module is used for acquiring the current position coordinate of the unmanned aerial vehicle;
and the calculation module is respectively connected with the acquisition module and the positioning module and used for calculating the distance between the unmanned aerial vehicle and the target object according to the position coordinate of the target object and the current position coordinate of the unmanned aerial vehicle.
Further, the acquisition module acquires the position coordinates of the target object by controlling any two adjacent cameras in the vision units to perform binocular stereo vision measurement.
In this embodiment, the acquisition module controls any two adjacent cameras, so as to establish binocular stereo vision measurement and acquire the position coordinates of the target object.
The principle of binocular stereo vision measurement is as follows:
taking image a and image B in fig. 2 as an example: dotPIs a point on the observation target in space, and has three-dimensional coordinates in the left camera coordinate system of (X C ,Y C ,Z C ) The imaging positions on the two images are respectivelyP a(x 1,y 1)、P b(x 2,y 2) The geometric relationship of similar triangles can be used as follows:
wherein,x 1、y 1、x 2、y 2are the physical coordinates in the planar image,B C is an external parameter of the camera (base line distance),fthe internal parameters (focal length) of the camera, | | is the parallax, i.e. the pointPThe position shift in the two images. Since the cameras are in the same plane in the vertical directiony=y 1=y 2Parallax is-x 1-x 2|。
Acquiring attitude parameters (namely external parameters) of the camera at the current angle involved in binocular stereo vision measurement according to the internal parameters of the camera and distortion parameters of a lens, (b) measuring the position coordinates of a target object) And acquiring the current position coordinate of the unmanned aerial vehicle according to the positioning module, and then obtaining the distance between the target object and the unmanned aerial vehicle.
In a preferred embodiment, after the panoramic stitched image is preprocessed, the reconstruction unit extracts feature point pairs in two continuous panoramic stitched images to match the images pairwise to obtain corresponding point pairs, and obtains three-dimensional point cloud data according to the corresponding points by a triangulation method to reconstruct the terrain according to the three-dimensional point cloud data.
Further, the preprocessing includes denoising and/or radiometric correction of the panorama stitched image.
In this embodiment, the reconstruction unit performs processing such as denoising and radiation correction on the panoramic stitched image, extracts image feature points, performs pairwise matching on the image based on the feature points, and deletes a wrong matching pair to obtain a more accurate corresponding point. With the calibrated image and the corresponding image characteristic points, the three-dimensional point cloud coordinate can be obtained through a triangulation method for the registration result, the image is triangulated to obtain three-dimensional point cloud data, and the three-dimensional point cloud data is subjected to meshing processing and texture mapping, so that the terrain reconstruction is completed.
In a preferred embodiment, the hovering unit comprises:
the detection module is used for detecting the characteristic points according to the panoramic mosaic image at preset time intervals so as to obtain effective characteristic points, and obtaining the three-dimensional coordinates of the unmanned aerial vehicle according to the positions of the characteristic points in the panoramic mosaic image and the positions of the characteristic points in a world coordinate system;
the judging module is connected with the detecting module and used for judging whether the difference between the three-dimensional coordinates of the two continuously acquired unmanned aerial vehicles is larger than a preset threshold value or not, and if the difference exceeds the preset threshold value, a correction instruction is output;
and the correction module is connected with the judgment module and used for correcting the current space position of the unmanned aerial vehicle according to the correction instruction.
In this embodiment, the hovering unit is used to fix the drone at a preset height position and a horizontal position, that is, fix the drone at a set of three-dimensional coordinates. During hovering operation, a plurality of cameras are needed to photograph surrounding scenes, and then feature point detection is carried out on images to find effective feature points. And matching the characteristic points in each image to find out corresponding characteristic point pairs. And according to the binocular vision principle, reversely calculating the three-dimensional coordinates of the camera according to the positions of the characteristic point pairs in the images and the world coordinate system. And shooting a group of images at a certain interval, and if the difference between the obtained three-dimensional coordinate and the three-dimensional coordinate obtained last time is larger than a preset threshold value, correcting the current space position of the unmanned aerial vehicle through a correction module. And repeating the operations until the hovering operation is finished.
In a preferred embodiment, further comprising:
and the tracking unit is connected with the control unit and used for detecting the moving target, generating a tracking path according to the detection result and tracking the moving target according to the tracking path.
In this embodiment, the specific process of tracking the moving target is as follows:
1) motion estimation (global motion compensation): considering the displacement estimation of the background, namely global motion compensation, the influence of the background needs to be removed, the error is eliminated, and a foundation is laid for the subsequent moving target detection and tracking; estimating image background displacement generated by camera motion by adopting a three-parameter model of rotational translation;
2) target detection: and detecting the moving target by using a background difference method. The extraction of the moving object may be obtained by a difference between the current image and the background image. When the camera is still, the current image isI k+1(r,t) The background image isu k+1(r,t) The variation information of each pixel value in the image is D k+1(r,t)；
D k+1(r,t)=|I k+1(r,t)-u k+1(r,t)|
Wherein,I k+1(r,t) Andu k+1(r,t) It may be a certain pixel value of the image, such as the brightness, or it may be a certain feature of the image, such as the color. When D is present k+1(r,t) When a preset threshold is reached, the target can be considered as a moving target.
3) Target tracking: after target detection is finished, information such as the position, size and shape of a moving target is obtained from an image, the moving target is used as a template, an area is obtained from the image, and a correlation coefficient of corresponding pixels between the area and the template is calculated, so that the target is tracked.
On the basis of the technical scheme, further, when the unmanned aerial vehicle flies in the air, the unmanned aerial vehicle generally flies according to a planned route in advance. When the unmanned aerial vehicle encounters an obstacle (such as a building, a big tree and the like) in the flight process, the unmanned aerial vehicle can bypass the obstacle through the obstacle avoidance unit and can return to the originally planned route to continue the flight. When the unmanned aerial vehicle flies at low altitude, the plurality of flight direction cameras are controlled to be opened by the vision unit. And planning the flight path of the unmanned aerial vehicle by using the reconstructed topographic map. A safety threshold (i.e., a safety distance) is set according to the performance of the drone. The distance measurement unit is used for measuring the distance of the front target and judging whether the distance between the target and the unmanned aerial vehicle is smaller than a safe distance or not; if the current time is less than the preset time, the control unit controls the hovering unit to execute hovering operation to enable the unmanned aerial vehicle to hover and bypass the obstacle, and finally the unmanned aerial vehicle returns to the planned flight route to continue flying.
According to the invention, the unmanned aerial vehicle can be utilized to obtain large-scale clear panoramic mosaic images at multiple angles; through functions of ranging, hovering, obstacle avoidance, reconstruction and the like of the unmanned aerial vehicle, the unmanned aerial vehicle has higher flexibility and higher safety during flying. Still can realize unmanned aerial vehicle to the tracking of moving target for unmanned aerial vehicle's usage is abundanter. The unmanned aerial vehicle system integrates multiple functions of splicing, ranging, hovering, obstacle avoidance, terrain reconstruction, tracking and the like, and makes full use of multiple cameras configured on the unmanned aerial vehicle.
While the invention has been described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention.
Claims (8)
1. The utility model provides a panorama unmanned aerial vehicle system based on multiplex many meshes, is applied to among the unmanned aerial vehicle, its characterized in that includes:
the vision unit comprises a plurality of cameras, the cameras are arranged on the unmanned aerial vehicle and are used for shooting a panoramic mosaic image with a visual center of 360 degrees, and partial visual angles of two adjacent cameras are overlapped;
the distance measurement unit is connected with the vision unit and acquires the distance between the unmanned aerial vehicle and a target object by adopting a binocular stereo vision measurement method;
the reconstruction unit is connected with the visual unit and used for preprocessing the panoramic stitching image to acquire three-dimensional point cloud data so as to reconstruct the terrain according to the three-dimensional point cloud data;
the hovering unit is used for controlling the unmanned aerial vehicle to hover at a space position matched with the three-dimensional space coordinate;
the obstacle avoidance unit is used for providing a flight path and a safety threshold value, detecting whether an obstacle which is away from the unmanned aerial vehicle on the flight path reaches the safety threshold value or not at regular time, and generating obstacle information and outputting the obstacle information when the obstacle reaches the safety threshold value;
and the control unit is respectively connected with the vision unit, the distance measurement unit, the reconstruction unit, the hovering unit and the obstacle avoidance unit, corrects the flight path according to the obstacle avoidance information, generates the three-dimensional space coordinate, and controls the hovering unit to execute hovering operation according to the three-dimensional space coordinate.
2. The multi-purpose multiplexing based panoramic drone system of claim 1, wherein the vision unit comprises:
the calibration module is used for respectively calibrating the plurality of cameras to acquire internal parameters and external parameters of each camera so as to acquire a mapping relation between coordinates of an image acquired by each camera in a world coordinate system and coordinates in an image coordinate system;
and the processing module is connected with the calibration module, respectively carries out denoising processing on each image, projects all the images to a uniform coordinate system, processes the images by adopting image interpolation values, and carries out matching and splicing on the processed images so as to obtain the panoramic spliced image.
3. The multi-purpose multiplexing based panoramic drone system of claim 1, wherein the ranging unit comprises:
the acquisition module is used for acquiring the position coordinates of the target object;
the positioning module is used for acquiring the current position coordinate of the unmanned aerial vehicle;
and the calculation module is respectively connected with the acquisition module and the positioning module and is used for calculating to acquire the distance between the unmanned aerial vehicle and the target object according to the position coordinate of the target object and the current position coordinate of the unmanned aerial vehicle.
4. The multi-view multiplexing-based panoramic unmanned aerial vehicle system of claim 3, wherein the acquisition module acquires the position coordinates of the target object by controlling any two cameras in the vision units that are adjacent in position to perform binocular stereo vision measurement.
5. The multi-view multiplexing-based panoramic unmanned aerial vehicle system of claim 1, wherein the reconstruction unit extracts feature point pairs in two consecutive panoramic stitched images to match each other in pairs after the panoramic stitched images are preprocessed so as to obtain corresponding point pairs, and obtains the three-dimensional point cloud data by a triangulation method according to the corresponding points so as to reconstruct the terrain according to the three-dimensional point cloud data.
6. The multi-view multiplexing based panoramic drone system of claim 1 or 5, wherein the pre-processing comprises de-noising and/or radiometric correction of the panoramic stitched image.
7. The multi-purpose multiplexing based panoramic drone system of claim 1, wherein the hovering unit comprises:
the detection module is used for detecting characteristic points according to the panoramic mosaic image at preset time intervals so as to obtain effective characteristic points, and obtaining the three-dimensional coordinates of the unmanned aerial vehicle according to the positions of the characteristic points in the panoramic mosaic image and the positions of the characteristic points in a world coordinate system;
the judging module is connected with the detecting module and used for judging whether the difference between the three-dimensional coordinates of the two continuously acquired unmanned aerial vehicles is larger than a preset threshold value or not, and if the difference exceeds the preset threshold value, a correction instruction is output;
and the correction module is connected with the judgment module and used for correcting the current space position of the unmanned aerial vehicle according to the correction instruction.
8. The multi-purpose multiplexing based panoramic drone system of claim 1, further comprising:
and the tracking unit is connected with the control unit and used for detecting the moving target, generating a tracking path according to the detection result and tracking the moving target according to the tracking path.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610502799.3A CN106204443A (en) | 2016-07-01 | 2016-07-01 | A kind of panorama UAS based on the multiplexing of many mesh |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610502799.3A CN106204443A (en) | 2016-07-01 | 2016-07-01 | A kind of panorama UAS based on the multiplexing of many mesh |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106204443A true CN106204443A (en) | 2016-12-07 |
Family
ID=57463891
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610502799.3A Pending CN106204443A (en) | 2016-07-01 | 2016-07-01 | A kind of panorama UAS based on the multiplexing of many mesh |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106204443A (en) |
Cited By (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106774402A (en) * | 2016-12-28 | 2017-05-31 | 湖南省道通科技有限公司 | The method and device positioned to unmanned plane |
| CN106803271A (en) * | 2016-12-23 | 2017-06-06 | 成都通甲优博科技有限责任公司 | A kind of camera marking method and device of vision guided navigation unmanned plane |
| CN107065926A (en) * | 2017-04-12 | 2017-08-18 | 普宙飞行器科技（深圳）有限公司 | Omnidirectional's obstacle avoidance apparatus, head, the control method of head and avoidance obstacle method |
| CN107343177A (en) * | 2017-08-07 | 2017-11-10 | 深圳市华琥技术有限公司 | A kind of filming control method of unmanned plane panoramic video |
| CN107483911A (en) * | 2017-08-25 | 2017-12-15 | 秦山 | A kind of signal processing method and system based on more mesh imaging sensors |
| CN107730584A (en) * | 2017-10-30 | 2018-02-23 | 南京青臣创意数字科技有限公司 | A kind of panoramic space constructing system based on the technology of taking photo by plane |
| CN107833272A (en) * | 2017-10-30 | 2018-03-23 | 南京青臣创意数字科技有限公司 | A kind of panoramic space construction method |
| WO2018134677A1 (en) * | 2017-01-23 | 2018-07-26 | Hangzhou Zero Technology Co., Ltd | Multi-camera system and method of use |
| CN108447097A (en) * | 2018-03-05 | 2018-08-24 | 清华-伯克利深圳学院筹备办公室 | Depth camera scaling method, device, electronic equipment and storage medium |
| CN108873035A (en) * | 2018-05-10 | 2018-11-23 | 四川省冶地工程勘察设计有限公司 | A kind of remote sensing surveying and mapping technology suitable for city suburbs |
| CN108958283A (en) * | 2018-06-28 | 2018-12-07 | 芜湖新尚捷智能信息科技有限公司 | A kind of unmanned plane low latitude automatic obstacle avoiding system |
| CN109164825A (en) * | 2018-08-13 | 2019-01-08 | 上海机电工程研究所 | A kind of independent navigation barrier-avoiding method and device for multi-rotor unmanned aerial vehicle |
| CN109214984A (en) * | 2017-07-03 | 2019-01-15 | 北京臻迪科技股份有限公司 | A kind of image acquiring method and device, calculate equipment at automatic positioning navigation system |
| CN109582032A (en) * | 2018-10-11 | 2019-04-05 | 天津大学 | Quick Real Time Obstacle Avoiding routing resource of the multi-rotor unmanned aerial vehicle under complex environment |
| CN109709977A (en) * | 2017-10-26 | 2019-05-03 | 广州极飞科技有限公司 | The method, apparatus and mobile object of motion track planning |
| CN109767387A (en) * | 2018-12-26 | 2019-05-17 | 北京木业邦科技有限公司 | A kind of forest image acquiring method and device based on unmanned plane |
| WO2019100219A1 (en) * | 2017-11-21 | 2019-05-31 | 深圳市大疆创新科技有限公司 | Output image generation method, device and unmanned aerial vehicle |
| US10358214B2 (en) | 2015-01-04 | 2019-07-23 | Hangzhou Zero Zro Technology Co., Ltd. | Aerial vehicle and method of operation |
| CN110245634A (en) * | 2019-06-20 | 2019-09-17 | 招商局重庆交通科研设计院有限公司 | Multiposition, multi-angle crag deformation judgement and analysis method |
| US10435144B2 (en) | 2016-04-24 | 2019-10-08 | Hangzhou Zero Zero Technology Co., Ltd. | Aerial system propulsion assembly and method of use |
| CN110419209A (en) * | 2017-03-31 | 2019-11-05 | 日本电气株式会社 | Information processing unit, information processing method and message handling program |
| CN110823899A (en) * | 2018-08-10 | 2020-02-21 | 杭州海康机器人技术有限公司 | Fault positioning method, device and system |
| US10824149B2 (en) | 2015-01-04 | 2020-11-03 | Hangzhou Zero Zero Technology Co., Ltd. | System and method for automated aerial system operation |
| US10824167B2 (en) | 2015-01-04 | 2020-11-03 | Hangzhou Zero Zero Technology Co., Ltd. | System and method for automated aerial system operation |
| CN112508865A (en) * | 2020-11-23 | 2021-03-16 | 深圳供电局有限公司 | Unmanned aerial vehicle inspection obstacle avoidance method and device, computer equipment and storage medium |
| CN112598010A (en) * | 2020-11-26 | 2021-04-02 | 厦门大学 | Binocular vision unstructured terrain real-time perception and reconstruction method |
| CN112614171A (en) * | 2020-11-26 | 2021-04-06 | 厦门大学 | Air-ground integrated dynamic environment sensing system for cluster operation of engineering machinery |
| WO2021088684A1 (en) * | 2019-11-07 | 2021-05-14 | 深圳市道通智能航空技术股份有限公司 | Omnidirectional obstacle avoidance method and unmanned aerial vehicle |
| CN115499596A (en) * | 2022-11-18 | 2022-12-20 | 北京中科觅境智慧生态科技有限公司 | Method and device for processing image |
| CN117392328A (en) * | 2023-12-07 | 2024-01-12 | 四川云实信息技术有限公司 | Three-dimensional live-action modeling method and system based on unmanned aerial vehicle cluster |
| CN117710467A (en) * | 2024-02-06 | 2024-03-15 | 天津云圣智能科技有限责任公司 | Unmanned plane positioning method, unmanned plane positioning equipment and aircraft |
| CN118226871A (en) * | 2024-05-23 | 2024-06-21 | 北京数易科技有限公司 | Unmanned aerial vehicle obstacle avoidance method, system and medium based on deep reinforcement learning |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102495522A (en) * | 2011-12-01 | 2012-06-13 | 天津曙光敬业科技有限公司 | Method for manufacturing 360-degree air panoramic interactive roam system based on unmanned helicopter aerial shooting |
| CN104809759A (en) * | 2015-04-03 | 2015-07-29 | 哈尔滨工业大学深圳研究生院 | Large-area unstructured three-dimensional scene modeling method based on small unmanned helicopter |
| CN105203084A (en) * | 2015-07-02 | 2015-12-30 | 汤一平 | 3D panoramic vision device for unmanned aerial vehicle |
| CN105353758A (en) * | 2015-11-10 | 2016-02-24 | 闫夙 | Precise automatic lawnmower distributed beacon laser positioning and trajectory control system |
| CN105654502A (en) * | 2016-03-30 | 2016-06-08 | 广州市盛光微电子有限公司 | Panorama camera calibration device and method based on multiple lenses and multiple sensors |
| CN105704501A (en) * | 2016-02-06 | 2016-06-22 | 普宙飞行器科技（深圳）有限公司 | Unmanned plane panorama video-based virtual reality live broadcast system |
-
2016
- 2016-07-01 CN CN201610502799.3A patent/CN106204443A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102495522A (en) * | 2011-12-01 | 2012-06-13 | 天津曙光敬业科技有限公司 | Method for manufacturing 360-degree air panoramic interactive roam system based on unmanned helicopter aerial shooting |
| CN104809759A (en) * | 2015-04-03 | 2015-07-29 | 哈尔滨工业大学深圳研究生院 | Large-area unstructured three-dimensional scene modeling method based on small unmanned helicopter |
| CN105203084A (en) * | 2015-07-02 | 2015-12-30 | 汤一平 | 3D panoramic vision device for unmanned aerial vehicle |
| CN105353758A (en) * | 2015-11-10 | 2016-02-24 | 闫夙 | Precise automatic lawnmower distributed beacon laser positioning and trajectory control system |
| CN105704501A (en) * | 2016-02-06 | 2016-06-22 | 普宙飞行器科技（深圳）有限公司 | Unmanned plane panorama video-based virtual reality live broadcast system |
| CN105654502A (en) * | 2016-03-30 | 2016-06-08 | 广州市盛光微电子有限公司 | Panorama camera calibration device and method based on multiple lenses and multiple sensors |
Non-Patent Citations (1)
| Title |
|---|
| 苏鲜女 等: "《籒园讲坛 上》", 31 December 2012 * |
Cited By (48)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10824149B2 (en) | 2015-01-04 | 2020-11-03 | Hangzhou Zero Zero Technology Co., Ltd. | System and method for automated aerial system operation |
| US10824167B2 (en) | 2015-01-04 | 2020-11-03 | Hangzhou Zero Zero Technology Co., Ltd. | System and method for automated aerial system operation |
| US10358214B2 (en) | 2015-01-04 | 2019-07-23 | Hangzhou Zero Zro Technology Co., Ltd. | Aerial vehicle and method of operation |
| US11027833B2 (en) | 2016-04-24 | 2021-06-08 | Hangzhou Zero Zero Technology Co., Ltd. | Aerial system propulsion assembly and method of use |
| US10435144B2 (en) | 2016-04-24 | 2019-10-08 | Hangzhou Zero Zero Technology Co., Ltd. | Aerial system propulsion assembly and method of use |
| CN106803271A (en) * | 2016-12-23 | 2017-06-06 | 成都通甲优博科技有限责任公司 | A kind of camera marking method and device of vision guided navigation unmanned plane |
| CN106803271B (en) * | 2016-12-23 | 2020-04-28 | 成都通甲优博科技有限责任公司 | Camera calibration method and device for visual navigation unmanned aerial vehicle |
| CN106774402A (en) * | 2016-12-28 | 2017-05-31 | 湖南省道通科技有限公司 | The method and device positioned to unmanned plane |
| WO2018120350A1 (en) * | 2016-12-28 | 2018-07-05 | 深圳市道通智能航空技术有限公司 | Method and device for positioning unmanned aerial vehicle |
| WO2018134677A1 (en) * | 2017-01-23 | 2018-07-26 | Hangzhou Zero Technology Co., Ltd | Multi-camera system and method of use |
| US10303185B2 (en) | 2017-01-23 | 2019-05-28 | Hangzhou Zero Zero Technology Co., Ltd. | Multi-camera system and method of use |
| US10873689B2 (en) | 2017-03-31 | 2020-12-22 | Nec Corporation | Information processing apparatus, information processing method, and information processing program |
| CN110419209A (en) * | 2017-03-31 | 2019-11-05 | 日本电气株式会社 | Information processing unit, information processing method and message handling program |
| CN107065926A (en) * | 2017-04-12 | 2017-08-18 | 普宙飞行器科技（深圳）有限公司 | Omnidirectional's obstacle avoidance apparatus, head, the control method of head and avoidance obstacle method |
| CN109214984B (en) * | 2017-07-03 | 2023-03-14 | 臻迪科技股份有限公司 | Image acquisition method and device, autonomous positioning navigation system and computing equipment |
| CN109214984A (en) * | 2017-07-03 | 2019-01-15 | 北京臻迪科技股份有限公司 | A kind of image acquiring method and device, calculate equipment at automatic positioning navigation system |
| CN107343177A (en) * | 2017-08-07 | 2017-11-10 | 深圳市华琥技术有限公司 | A kind of filming control method of unmanned plane panoramic video |
| CN107483911A (en) * | 2017-08-25 | 2017-12-15 | 秦山 | A kind of signal processing method and system based on more mesh imaging sensors |
| CN109709977B (en) * | 2017-10-26 | 2022-08-16 | 广州极飞科技股份有限公司 | Method and device for planning movement track and moving object |
| CN109709977A (en) * | 2017-10-26 | 2019-05-03 | 广州极飞科技有限公司 | The method, apparatus and mobile object of motion track planning |
| CN107730584A (en) * | 2017-10-30 | 2018-02-23 | 南京青臣创意数字科技有限公司 | A kind of panoramic space constructing system based on the technology of taking photo by plane |
| CN107833272A (en) * | 2017-10-30 | 2018-03-23 | 南京青臣创意数字科技有限公司 | A kind of panoramic space construction method |
| WO2019100219A1 (en) * | 2017-11-21 | 2019-05-31 | 深圳市大疆创新科技有限公司 | Output image generation method, device and unmanned aerial vehicle |
| CN108447097A (en) * | 2018-03-05 | 2018-08-24 | 清华-伯克利深圳学院筹备办公室 | Depth camera scaling method, device, electronic equipment and storage medium |
| CN108447097B (en) * | 2018-03-05 | 2021-04-27 | 清华-伯克利深圳学院筹备办公室 | Depth camera calibration method and device, electronic equipment and storage medium |
| CN108873035A (en) * | 2018-05-10 | 2018-11-23 | 四川省冶地工程勘察设计有限公司 | A kind of remote sensing surveying and mapping technology suitable for city suburbs |
| CN108958283A (en) * | 2018-06-28 | 2018-12-07 | 芜湖新尚捷智能信息科技有限公司 | A kind of unmanned plane low latitude automatic obstacle avoiding system |
| CN110823899B (en) * | 2018-08-10 | 2022-07-05 | 杭州海康机器人技术有限公司 | Fault positioning method, device and system |
| CN110823899A (en) * | 2018-08-10 | 2020-02-21 | 杭州海康机器人技术有限公司 | Fault positioning method, device and system |
| CN109164825A (en) * | 2018-08-13 | 2019-01-08 | 上海机电工程研究所 | A kind of independent navigation barrier-avoiding method and device for multi-rotor unmanned aerial vehicle |
| CN109582032A (en) * | 2018-10-11 | 2019-04-05 | 天津大学 | Quick Real Time Obstacle Avoiding routing resource of the multi-rotor unmanned aerial vehicle under complex environment |
| CN109582032B (en) * | 2018-10-11 | 2021-10-12 | 天津大学 | Multi-rotor unmanned aerial vehicle rapid real-time obstacle avoidance path selection method in complex environment |
| CN109767387A (en) * | 2018-12-26 | 2019-05-17 | 北京木业邦科技有限公司 | A kind of forest image acquiring method and device based on unmanned plane |
| CN110245634A (en) * | 2019-06-20 | 2019-09-17 | 招商局重庆交通科研设计院有限公司 | Multiposition, multi-angle crag deformation judgement and analysis method |
| WO2021088684A1 (en) * | 2019-11-07 | 2021-05-14 | 深圳市道通智能航空技术股份有限公司 | Omnidirectional obstacle avoidance method and unmanned aerial vehicle |
| CN112508865B (en) * | 2020-11-23 | 2024-02-02 | 深圳供电局有限公司 | Unmanned aerial vehicle inspection obstacle avoidance method, unmanned aerial vehicle inspection obstacle avoidance device, computer equipment and storage medium |
| CN112508865A (en) * | 2020-11-23 | 2021-03-16 | 深圳供电局有限公司 | Unmanned aerial vehicle inspection obstacle avoidance method and device, computer equipment and storage medium |
| CN112614171A (en) * | 2020-11-26 | 2021-04-06 | 厦门大学 | Air-ground integrated dynamic environment sensing system for cluster operation of engineering machinery |
| CN112598010A (en) * | 2020-11-26 | 2021-04-02 | 厦门大学 | Binocular vision unstructured terrain real-time perception and reconstruction method |
| CN112598010B (en) * | 2020-11-26 | 2023-08-01 | 厦门大学 | Unstructured terrain real-time sensing and reconstructing method for binocular vision |
| CN112614171B (en) * | 2020-11-26 | 2023-12-19 | 厦门大学 | Air-ground integrated dynamic environment sensing system for engineering machinery cluster operation |
| CN115499596A (en) * | 2022-11-18 | 2022-12-20 | 北京中科觅境智慧生态科技有限公司 | Method and device for processing image |
| CN117392328A (en) * | 2023-12-07 | 2024-01-12 | 四川云实信息技术有限公司 | Three-dimensional live-action modeling method and system based on unmanned aerial vehicle cluster |
| CN117392328B (en) * | 2023-12-07 | 2024-02-23 | 四川云实信息技术有限公司 | Three-dimensional live-action modeling method and system based on unmanned aerial vehicle cluster |
| CN117710467A (en) * | 2024-02-06 | 2024-03-15 | 天津云圣智能科技有限责任公司 | Unmanned plane positioning method, unmanned plane positioning equipment and aircraft |
| CN117710467B (en) * | 2024-02-06 | 2024-05-28 | 天津云圣智能科技有限责任公司 | Unmanned plane positioning method, unmanned plane positioning equipment and aircraft |
| CN118226871A (en) * | 2024-05-23 | 2024-06-21 | 北京数易科技有限公司 | Unmanned aerial vehicle obstacle avoidance method, system and medium based on deep reinforcement learning |
| CN118226871B (en) * | 2024-05-23 | 2024-08-16 | 北京数易科技有限公司 | Unmanned aerial vehicle obstacle avoidance method, system and medium based on deep reinforcement learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106204443A (en) | A kind of panorama UAS based on the multiplexing of many mesh | |
| KR100912715B1 (en) | Method and apparatus of digital photogrammetry by integrated modeling for different types of sensors | |
| CN108168521A (en) | One kind realizes landscape three-dimensional visualization method based on unmanned plane | |
| Haala et al. | Quality of 3D point clouds from highly overlapping UAV imagery | |
| JP7273927B2 (en) | Image-based positioning method and system | |
| Barazzetti et al. | True-orthophoto generation from UAV images: Implementation of a combined photogrammetric and computer vision approach | |
| EP2910899B1 (en) | A method and a system for building a three-dimensional model from satellite images | |
| CN107492069B (en) | Image fusion method based on multi-lens sensor | |
| CN107504957A (en) | The method that three-dimensional terrain model structure is quickly carried out using unmanned plane multi-visual angle filming | |
| CN108474658B (en) | Ground form detection method and system, unmanned aerial vehicle landing method and unmanned aerial vehicle | |
| CN112634370A (en) | Unmanned aerial vehicle dotting method, device, equipment and storage medium | |
| KR101150510B1 (en) | Method for Generating 3-D High Resolution NDVI Urban Model | |
| Rijsdijk et al. | Unmanned aerial systems in the process of juridical verification of cadastral border | |
| CN110246221A (en) | True orthophoto preparation method and device | |
| WO2019100219A1 (en) | Output image generation method, device and unmanned aerial vehicle | |
| CN109709977B (en) | Method and device for planning movement track and moving object | |
| CN108769569B (en) | 360-degree three-dimensional panoramic observation system and method for unmanned aerial vehicle | |
| US11769225B2 (en) | Image processing apparatus, image processing method, and program | |
| CN108399631B (en) | Scale invariance oblique image multi-view dense matching method | |
| CN110889873A (en) | Target positioning method and device, electronic equipment and storage medium | |
| CN113345084B (en) | Three-dimensional modeling system and three-dimensional modeling method | |
| CN104599281B (en) | A kind of based on the conforming panorama sketch in horizontal linear orientation and remote sensing figure method for registering | |
| Höhle | Oblique aerial images and their use in cultural heritage documentation | |
| CN113566793A (en) | True orthoimage generation method and device based on unmanned aerial vehicle oblique image | |
| CN109712249B (en) | Geographic element augmented reality method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20161207 |
|
| RJ01 | Rejection of invention patent application after publication |