CN111612779A - Water area algae small target detection method and system based on aerial image and CIM - Google Patents
Water area algae small target detection method and system based on aerial image and CIM Download PDFInfo
- Publication number
- CN111612779A CN111612779A CN202010456414.0A CN202010456414A CN111612779A CN 111612779 A CN111612779 A CN 111612779A CN 202010456414 A CN202010456414 A CN 202010456414A CN 111612779 A CN111612779 A CN 111612779A
- Authority
- CN
- China
- Prior art keywords
- water area
- algae
- image
- aerial
- small target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims abstract description 181
- 241000195493 Cryptophyta Species 0.000 title claims abstract description 139
- 238000001514 detection method Methods 0.000 title claims abstract description 21
- 230000011218 segmentation Effects 0.000 claims abstract description 77
- 238000000605 extraction Methods 0.000 claims abstract description 39
- 238000000034 method Methods 0.000 claims abstract description 27
- 238000005516 engineering process Methods 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 7
- 230000009466 transformation Effects 0.000 claims description 15
- 238000012549 training Methods 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000013178 mathematical model Methods 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 238000012800 visualization Methods 0.000 claims description 6
- 230000005855 radiation Effects 0.000 abstract 1
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000009360 aquaculture Methods 0.000 description 2
- 244000144974 aquaculture Species 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 241000894006 Bacteria Species 0.000 description 1
- 241000700605 Viruses Species 0.000 description 1
- 208000034699 Vitreous floaters Diseases 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 239000004927 clay Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000003344 environmental pollutant Substances 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 210000003746 feather Anatomy 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000004060 metabolic process Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 231100000719 pollutant Toxicity 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- G06T3/14—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Abstract
The invention discloses a method and a system for detecting small targets of algae in a water area based on aerial images and CIM. The method comprises the following steps: building a city information model of a three-dimensional digital space; a bayer array having a raw bayer array, containing near-infrared information; carrying out interpolation processing on the original Bayer array; calculating a near infrared band radiation value NIR of the aerial image; calculating the normalized water index NDWI of the aerial photography water area image, combining the NDWI with the RGB data to form four-channel input data, inputting the four-channel input data into an algae small target extraction encoder and a decoder to output an algae small target segmentation probability map; the argmax operation obtains a small algae target segmentation image; and reconstructing aerial water area images and segmentation images of all the cameras, projecting the aerial water area images and the segmentation images into a city information model, and visualizing the city information model by combining a WebGIS technology. The invention solves the problem that the algae detection can not be carried out on the area inside the water area, not only improves the efficiency and the accuracy of the algae target detection, but also provides convenience for supervision and management.
Description
Technical Field
The invention relates to the technical field of artificial intelligence and CIM, in particular to a method and a system for detecting a small target of algae in a water area based on aerial images and CIM.
Background
The algae can adsorb suspended matters in the water body, but can not be settled and can be seen by naked eyes. Therefore, the suspended matters in the water body mainly comprise silt, clay, protozoa, algae, bacteria, viruses, macromolecular organic matters and the like, and are often suspended in the water flow, and the turbidity phenomenon caused by the water is also caused by the substances.
Most algae plants live in water, once pollutants enter a water body and are absorbed by algae, the growth and metabolism of the algae are disordered, so that the composition of the algae in the water body is changed, the visibility and the color of water are changed, and the water quality is polluted under severe conditions.
The algae in the water plays a very important role in aquaculture, determines the success or failure of our culture and simultaneously provides guarantee for high-density intensive culture. Therefore, it is necessary to detect algae in water area in aquaculture and water quality detection.
The existing detection of algae in water is judged based on human eyes, and when the area of the water area is large, the situation of algae or suspended matters around the water area can only be judged artificially, and the existing detection is also stranded for the area inside the water area.
Disclosure of Invention
The invention aims to provide a method and a system for detecting algae in a water area based on aerial images and CIM (common information model), which aim to overcome the problem that algae detection cannot be carried out on the internal area of the water area, improve the efficiency and accuracy of algae target detection and provide convenience for supervision and management.
A method for detecting small targets of algae in a water area based on aerial images and CIM comprises the following steps:
building a three-dimensional digital spatial city information model by superposing BIM information of city buildings, underground facilities and city Internet of things information on the basis of three-dimensional city space geographic information;
shooting a water area in the sky of a city by using an unmanned aerial vehicle, and obtaining an original Bayer array with an infrared cut-off filter and a Bayer array without the infrared cut-off filter and containing near-infrared information through an infrared filter switcher;
carrying out interpolation processing on the original Bayer array to obtain an RGB value of each pixel;
subtracting the original Bayer array from the Bayer array containing the near-infrared information to obtain an aerial water area image near-infrared waveband reflection value NIR;
calculating the normalized water index NDWI of the aerial photography water area image: NDWI ═ (NIR-G)/(NIR + G);
combining the normalized water body index NDWI and the RGB data of the aerial photography water area image to form four-channel input data, and inputting the four-channel input data into an algae small target extraction encoder;
the algae small target extraction encoder performs feature extraction on input data to obtain a feature map;
performing up-sampling and feature extraction on the feature map by using an algae small target extraction decoder, and outputting an algae small target segmentation probability map;
performing argmax operation on the algae segmentation probability map to obtain an algae small target segmentation image;
extracting characteristic points of the aerial photography water area image to be spliced, then carrying out image registration, and finding out the corresponding positions of the characteristic points in the image to be spliced in the reference image;
calculating various parameter values in the mathematical model according to the corresponding relation among the characteristics of the aerial images of the water area to be spliced, and establishing a mathematical transformation model among the aerial images of the water area to be spliced;
converting the aerial photography water area images to be spliced and the small algae target segmentation images in the water area to be spliced into a reference image coordinate system according to the established mathematical conversion model to complete unified coordinate transformation;
stitching, namely combining the pixel values of the overlapped parts and keeping the pixel values which are not overlapped to generate an aerial photography water area image with a larger canvas and a water area algae small target segmentation image;
fusing the superposed areas of the aerial photography water area image with larger canvas and the water area algae small target segmentation image to obtain a spliced and reconstructed aerial photography water area image and a spliced and reconstructed water area algae small target segmentation image;
and projecting the reconstructed aerial photography water area image and the reconstructed water area algae small target segmentation image into a city information model, and visualizing the city information model by combining a WebGIS technology, wherein a visualization result comprises an aerial photography water area image layer and a water area algae small target segmentation layer.
The method also includes training an algae small target extraction encoder and an algae small target extraction decoder:
firstly, marking an aerial photography water area image sample set, setting the value of a pixel point of algae or suspended matters in the aerial photography water area image to be 1, setting the pixel point values of the rest non-algae or suspended matters to be 0, and generating a semantic segmentation label graph corresponding to the aerial photography water area image sample set;
inputting the aerial photography water area image sample set and the corresponding semantic segmentation label map into an algae target extraction encoder and an algae target extraction decoder, and training by adopting a cross entropy loss function.
Projecting the reconstructed aerial photography water area image and the reconstructed water area algae small target segmentation image into the city information model specifically comprises the following steps:
solving a homography matrix according to the position relation between the reconstructed image corner and the corresponding corner of the urban information model ground;
and projecting the reconstructed aerial photography water area image and the reconstructed water area algae small target segmentation image into the city information model according to the homography matrix.
A water area algae small target detection system based on aerial images and CIM can realize the following steps:
building a three-dimensional digital spatial city information model by superposing BIM information of city buildings, underground facilities and city Internet of things information on the basis of three-dimensional city space geographic information;
shooting a water area in the sky of a city by using an unmanned aerial vehicle, and obtaining an original Bayer array with an infrared cut-off filter and a Bayer array without the infrared cut-off filter and containing near-infrared information through an infrared filter switcher;
carrying out interpolation processing on the original Bayer array to obtain an RGB value of each pixel;
subtracting the original Bayer array from the Bayer array containing the near-infrared information to obtain an aerial image near-infrared waveband reflection value NIR;
calculating the normalized water index NDWI of the aerial photography water area image: NDWI ═ (NIR-G)/(NIR + G);
combining the normalized water body index NDWI and the RGB data of the aerial photography water area image to form four-channel input data, and inputting the four-channel input data into an algae small target extraction encoder;
the algae small target extraction encoder performs feature extraction on input data to obtain a feature map;
performing up-sampling and feature extraction on the feature map by using an algae small target extraction decoder, and outputting an algae small target segmentation probability map;
performing argmax operation on the algae segmentation probability map to obtain an algae small target segmentation image;
extracting characteristic points of the aerial photography water area image to be spliced, then carrying out image registration, and finding out the corresponding positions of the characteristic points in the image to be spliced in the reference image;
calculating various parameter values in the mathematical model according to the corresponding relation among the characteristics of the aerial images of the water area to be spliced, and establishing a mathematical transformation model among the aerial images of the water area to be spliced;
converting the aerial photography water area images to be spliced and the small algae target segmentation images in the water area to be spliced into a reference image coordinate system according to the established mathematical conversion model to complete unified coordinate transformation;
stitching, namely combining the pixel values of the overlapped parts and keeping the pixel values which are not overlapped to generate an aerial photography water area image with a larger canvas and a water area algae small target segmentation image;
fusing the superposed areas of the aerial photography water area image with larger canvas and the water area algae small target segmentation image to obtain a spliced and reconstructed aerial photography water area image and a spliced and reconstructed water area algae small target segmentation image;
and projecting the reconstructed aerial photography water area image and the reconstructed water area algae small target segmentation image into a city information model, and visualizing the city information model by combining a WebGIS technology, wherein a visualization result comprises an aerial photography water area image layer and a water area algae small target segmentation layer.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention combines the aerial photography technology to acquire the images of the water area, solves the problem that the algae target in the water area cannot be observed in the prior art, and improves the efficiency and the convenience of algae target detection.
2. The method adopts the deep neural network to detect the algae small target in the water area, uses a large number of samples compared with the traditional identification technology based on the computer vision technology, has better generalization performance, and improves the stability and the accuracy of algae target detection.
3. The method adopts the combination of the normalized water index and the RGB data to train the neural network, the normalized water index can provide richer information for the algae target detection, and the method is favorable for improving the training efficiency of the neural network and the accuracy of the algae target detection.
4. The method is based on the urban information model technology, the detection result is integrated into the urban information model through the image splicing and fusing technology, and the semantic segmentation layer and the aerial image layer are added, so that convenience is provided for intelligent urban information integration; and the city information model is visualized by combining a WebGIS technology, so that the inquiry and supervision of a supervisor are facilitated.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention provides a method and a system for detecting small targets of algae in a water area based on aerial images and CIM. The method comprises the steps of firstly, obtaining near infrared and RGB information through aerial photography and image processing, then generating a normalized water index in remote sensing through a simulation idea, training RGB and NDWI data through a semantic segmentation network to obtain a segmented image of a small algae target in water, and then mapping a detection result to a city information model through image splicing fusion and mapping. FIG. 1 is a flow chart of the method of the present invention. The following description will be made by way of specific examples.
Example 1:
the embodiment provides a method for detecting small targets of algae in a water area based on aerial images and CIM.
The appearance of unmanned aerial vehicle takes photo by plane provides huge facility for each application. The unmanned aerial vehicle has the advantages of simple structure, low use cost and flexibility, can complete the task executed by the piloted plane, and is more suitable for the task which is not suitable to be executed by the piloted plane. Therefore, the invention combines the aerial photography unmanned aerial vehicle to analyze and process the aerial photography image, thereby realizing the algae detection in the water area.
The internal relation between a city information model CIM and an information exchange module is established.
CIM is an organic complex which establishes a three-dimensional urban space model and building information based on urban information data, and mainly comprises GIS data and BIM data of urban roads, buildings, water areas and related infrastructure.
The information exchange module is a data exchange platform based on CIM, mainly comprises a three-dimensional city space model, city information, geographic information and camera perception information, and can update the model and information content in real time along with the continuous advance of the engineering construction progress.
And the CIM is based on three-dimensional urban space geographic information, and overlaps BIM information of urban buildings, underground facilities and urban Internet of things information to construct an urban information model of a three-dimensional digital space. The scene of a city is displayed in Web by combining a CIM city information model with a WebGIS technology, and the system can call an information exchange module to display the latest three-dimensional city space model and city information.
The invention mainly aims at aerial images to detect floaters such as algae small targets and the like in the water area, thereby realizing the environmental management of the water area.
Firstly, shooting a water area by using an unmanned aerial vehicle. Since the ir-cut filter is an essential component of the image sensor, the ir-cut filter switch can be used to control the presence of the ir-cut filter, so that the original bayer array S1 and the bayer array S2 containing the near-infrared information can be obtained by shooting.
Here, there is an infrared cut filter, and the array obtained by imaging is S1; the array obtained by imaging without an infrared cut filter was S2.
The original bayer array S1 is single channel, each pixel includes only a portion of the spectrum, and the RGB values for each pixel must be achieved by interpolation. The Bayer interpolation method is well known and will not be described in detail here.
The infrared cut filter can filter out infrared light within a certain range (the range can be customized), so the reflection information NIR of the near infrared channel can be obtained from S2-S1, namely the value of the S1 array is subtracted from the value of the S2 array.
In the remote sensing image, the normalized water index can effectively reflect the water body information of the image, and the calculation formula is as follows:
NDWI＝(NIR-G)/(NIR+G)
i.e. the sum of the difference ratio of the reflection value of the near infrared band and the reflection value of the green band, NIR is the reflection value of the near infrared band, and G is the reflection value of the green band.
And calculating the green light information (G channel) and the near infrared information (NIR) in the obtained RGB three-channel color picture through the formula to obtain the normalized water index of the aerial image through a simulation thought.
And then, making a data label, setting the value of a pixel point of algae or suspended matters in the aerial image to be 1, and setting the pixel point values of the rest non-algae or suspended matters to be 0.
After all data are obtained, training of the semantic segmentation network is started, and the training process is as follows.
The image data is normalized, and the value range is classified into a [0,1] interval, which is beneficial to the convergence of the network.
The image data and the label data (to be subjected to one-hot encoding) are then fed into the network for training. The algae small target extraction encoder is used for extracting the characteristics of image data, inputting the data after NDWI and RGB are combined (configured), and outputting the data as Feature map; the algae small target extraction decoder plays the roles of up-sampling and Feature extraction, and inputs the Feature map generated by the algae small target extraction encoder and outputs the Feature map as an algae small target segmentation probability map. The loss function uses cross entropy.
Finally, the algae small target segmentation probability map output by the network is subjected to argmax operation to obtain an algae small target segmentation image. Each point with a pixel value of 1 represents an algae small target.
It should be noted that if the aerial image is too large, for example, the resolution of the image is 1024 × 1024, then we need to slice the image, for example, to 4 images with 512 × 512 size, as the input of the model. The input of the model is consistent with the size of the sliced aerial image, and the size of the model cannot be unified by using an interpolation method. There are many cutting methods such as uniform cutting, overlap cutting, and the practitioner can freely select them.
Similarly, when the algae small target segmentation inference is carried out, the aerial images are sliced and finally spliced into a large image (the aerial image algae small target segmentation image), and the size of the large image is consistent with that of the model input image.
The encoder and decoder for extracting the algae small targets recommend a skip level connection structure and combines with block design of lightweight networks such as ShuffleNet and MobileNet so as to more quickly and accurately segment the algae small targets in the water area. Finally, the implementer may also apply common semantic segmentation models, such as FCN, deplab, ICNet, etc., to perform the segmentation of the algae small targets.
Thus, the water area algae small target segmentation image of the aerial image can be obtained.
And then splicing the small algae target segmentation images in the water area, namely, firstly splicing aerial images, wherein the segmentation images are binary images which cannot be directly spliced, and splicing and projection are required to be carried out according to the relation obtained by splicing the aerial images and the projection transformation relation. There are various image stitching methods, such as image stitching based on bottom layer features and image stitching based on regions, and it is proposed herein that image stitching using bottom layer features can achieve more accurate image stitching.
Firstly, feature point extraction is carried out on an aerial image of a water area, the bottom layer features of the image are various, such as ORB, SIFT, SURF and Harris, an implementer can freely select the features, and meanwhile, some improved methods are referred, such as L-ORB and the like.
And then, carrying out image registration, namely finding out the corresponding positions of the feature points in the images to be spliced in the reference image by adopting a certain matching strategy, and further determining the transformation relation between the two images. The matching strategy finds matching feature points, such as by performing a similarity metric.
And then, calculating parameter values in the mathematical model according to the corresponding relation between the image characteristics so as to establish a mathematical transformation model of the two images. This step is to solve the homography matrix.
Further, the images to be spliced are converted into a coordinate system of the reference image according to the established mathematical conversion model, and unified coordinate transformation is completed.
And splicing the water area algae small target segmentation graph according to the coordinate transformation relation of the aerial photography water area image.
Stitching is then performed to generate an image of a larger canvas by merging overlapping portions of pixel values and keeping non-overlapping pixel values.
And finally, fusing the overlapping area of the image of the larger canvas to obtain a spliced and reconstructed water area image and a water area algae small target segmentation image. There are various fusion methods, such as feathering (feather) fusion algorithm, pyramid (pyramid) fusion algorithm, etc.
The spliced and reconstructed aerial photography water area image and the water area algae small target segmentation image can be obtained through the method, and then the spliced and reconstructed aerial photography water area image and the water area algae small target segmentation image are projected onto the CIM for visualization.
The projection needs to calculate a homography matrix of the aerial image to the CIM ground, at least 4 groups of coordinate points are needed, namely at least 4 angular points in the water area algae small target segmentation image needing to be spliced and reconstructed and 4 angular points on the CIM ground, the solving process is well known and is not repeated herein, wherein the angular points are suggested to be selected manually so as to obtain a more accurate result, and the angular points between the two images need to be in one-to-one correspondence.
And after the spliced and reconstructed algae small target segmentation image is obtained, projecting the algae small target segmentation image to the CIM ground through the homography matrix obtained through calculation.
Finally, in order to visually present the detection result of the aerial image, the invention integrates the CIM city information model into a system developed by WebGIS by combining with WebGIS technology, updates the city space three-dimensional model in real time by calling an information exchange module, and performs data visualization of aerial image algae small target detection at a Web end and displays the distribution condition of algae small targets in a water area.
Example 2:
the invention also provides a water area algae small target detection system based on aerial images and CIM, which can realize the following steps: building a three-dimensional digital spatial city information model by superposing BIM information of city buildings, underground facilities and city Internet of things information on the basis of three-dimensional city space geographic information; shooting a water area in the sky of a city by using an unmanned aerial vehicle, and obtaining an original Bayer array with an infrared cut-off filter and a Bayer array without the infrared cut-off filter and containing near-infrared information through an infrared filter switcher; carrying out interpolation processing on the original Bayer array to obtain an RGB value of each pixel; subtracting the original Bayer array from the Bayer array containing the near-infrared information to obtain an aerial image near-infrared waveband reflection value NIR; calculating the normalized water index NDWI of the aerial photography water area image: NDWI ═ (NIR-G)/(NIR + G); combining the normalized water body index NDWI and the RGB data of the aerial photography water area image to form four-channel input data, and inputting the four-channel input data into an algae small target extraction encoder; the algae small target extraction encoder performs feature extraction on input data to obtain a feature map; performing up-sampling and feature extraction on the feature map by using an algae small target extraction decoder, and outputting an algae small target segmentation probability map; performing argmax operation on the algae small target segmentation probability map to obtain an algae small target segmentation image; extracting characteristic points of the aerial photography water area image to be spliced, then carrying out image registration, and finding out the corresponding positions of the characteristic points in the image to be spliced in the reference image; calculating various parameter values in the mathematical model according to the corresponding relation among the characteristics of the aerial images of the water area to be spliced, and establishing a mathematical transformation model among the aerial images of the water area to be spliced; converting the aerial photography water area images to be spliced and the small algae target segmentation images in the water area to be spliced into a reference image coordinate system according to the established mathematical conversion model to complete unified coordinate transformation; stitching, namely combining the pixel values of the overlapped parts and keeping the pixel values which are not overlapped to generate an aerial photography water area image with a larger canvas and a water area algae small target segmentation image; fusing the superposed areas of the aerial photography water area image with larger canvas and the water area algae small target segmentation image to obtain a spliced and reconstructed aerial photography water area image and a spliced and reconstructed water area algae small target segmentation image; and projecting the reconstructed aerial photography water area image and the reconstructed water area algae small target segmentation image into a city information model, and visualizing the city information model by combining a WebGIS technology, wherein a visualization result comprises an aerial photography water area image layer and a water area algae small target segmentation layer.
The above embodiments are merely preferred embodiments of the present invention, which should not be construed as limiting the present invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (4)
1. A method for detecting small targets of algae in a water area based on aerial images and CIM is characterized by comprising the following steps:
building a three-dimensional digital spatial city information model by superposing BIM information of city buildings, underground facilities and city Internet of things information on the basis of three-dimensional city space geographic information;
shooting a water area in the sky of a city by using an unmanned aerial vehicle, and obtaining an original Bayer array with an infrared cut-off filter and a Bayer array without the infrared cut-off filter and containing near-infrared information through an infrared filter switcher;
carrying out interpolation processing on the original Bayer array to obtain an RGB value of each pixel;
subtracting the original Bayer array from the Bayer array containing the near-infrared information to obtain an aerial water area image near-infrared waveband reflection value NIR;
calculating the normalized water index NDWI of the aerial photography water area image: NDWI ═ (NIR-G)/(NIR + G);
combining the normalized water body index NDWI and the RGB data of the aerial photography water area image to form four-channel input data, and inputting the four-channel input data into an algae small target extraction encoder;
the algae small target extraction encoder performs feature extraction on input data to obtain a feature map;
performing up-sampling and feature extraction on the feature map by using an algae small target extraction decoder, and outputting an algae small target segmentation probability map;
performing argmax operation on the algae segmentation probability map to obtain an algae small target segmentation image;
extracting characteristic points of the aerial photography water area image to be spliced, then carrying out image registration, and finding out the corresponding positions of the characteristic points in the image to be spliced in the reference image;
calculating various parameter values in the mathematical model according to the corresponding relation among the characteristics of the aerial images of the water area to be spliced, and establishing a mathematical transformation model among the aerial images of the water area to be spliced;
converting the aerial photography water area images to be spliced and the small algae target segmentation images in the water area to be spliced into a reference image coordinate system according to the established mathematical conversion model to complete unified coordinate transformation;
stitching, namely combining the pixel values of the overlapped parts and keeping the pixel values which are not overlapped to generate an aerial photography water area image with a larger canvas and a water area algae small target segmentation image;
fusing the superposed areas of the aerial photography water area image with larger canvas and the water area algae small target segmentation image to obtain a spliced and reconstructed aerial photography water area image and a spliced and reconstructed water area algae small target segmentation image;
and projecting the reconstructed aerial photography water area image and the reconstructed water area algae small target segmentation image into a city information model, and visualizing the city information model by combining a WebGIS technology, wherein a visualization result comprises an aerial photography water area image layer and a water area algae small target segmentation layer.
2. The method of claim 1, further comprising training an algae micro-object extraction encoder, an algae micro-object extraction decoder:
firstly, marking an aerial photography water area image sample set, setting the value of a pixel point of algae or suspended matters in the aerial photography water area image to be 1, setting the pixel point values of the rest non-algae or suspended matters to be 0, and generating a semantic segmentation label graph corresponding to the aerial photography water area image sample set;
inputting the aerial photography water area image sample set and the corresponding semantic segmentation label map into an algae target extraction encoder and an algae target extraction decoder, and training by adopting a cross entropy loss function.
3. The method of claim 1, wherein projecting the reconstructed aerial water area image and the reconstructed water area algae small target segmentation image into the city information model specifically comprises:
solving a homography matrix according to the position relation between the reconstructed image corner and the corresponding corner of the urban information model ground;
and projecting the reconstructed aerial photography water area image and the reconstructed water area algae small target segmentation image into the city information model according to the homography matrix.
4. A water area algae small target detection system based on aerial images and CIM is characterized by comprising the following steps:
building a three-dimensional digital spatial city information model by superposing BIM information of city buildings, underground facilities and city Internet of things information on the basis of three-dimensional city space geographic information;
shooting a water area in the sky of a city by using an unmanned aerial vehicle, and obtaining an original Bayer array with an infrared cut-off filter and a Bayer array without the infrared cut-off filter and containing near-infrared information through an infrared filter switcher;
carrying out interpolation processing on the original Bayer array to obtain an RGB value of each pixel;
subtracting the original Bayer array from the Bayer array containing the near-infrared information to obtain an aerial image near-infrared waveband reflection value NIR;
calculating the normalized water index NDWI of the aerial photography water area image: NDWI ═ (NIR-G)/(NIR + G);
combining the normalized water body index NDWI and the RGB data of the aerial photography water area image to form four-channel input data, and inputting the four-channel input data into an algae small target extraction encoder;
the algae small target extraction encoder performs feature extraction on input data to obtain a feature map;
performing up-sampling and feature extraction on the feature map by using an algae small target extraction decoder, and outputting an algae small target segmentation probability map;
performing argmax operation on the algae segmentation probability map to obtain an algae small target segmentation image;
extracting characteristic points of the aerial photography water area image to be spliced, then carrying out image registration, and finding out the corresponding positions of the characteristic points in the image to be spliced in the reference image;
calculating various parameter values in the mathematical model according to the corresponding relation among the characteristics of the aerial images of the water area to be spliced, and establishing a mathematical transformation model among the aerial images of the water area to be spliced;
converting the aerial photography water area images to be spliced and the small algae target segmentation images in the water area to be spliced into a reference image coordinate system according to the established mathematical conversion model to complete unified coordinate transformation;
stitching, namely combining the pixel values of the overlapped parts and keeping the pixel values which are not overlapped to generate an aerial photography water area image with a larger canvas and a water area algae small target segmentation image;
fusing the superposed areas of the aerial photography water area image with larger canvas and the water area algae small target segmentation image to obtain a spliced and reconstructed aerial photography water area image and a spliced and reconstructed water area algae small target segmentation image;
and projecting the reconstructed aerial photography water area image and the reconstructed water area algae small target segmentation image into a city information model, and visualizing the city information model by combining a WebGIS technology, wherein a visualization result comprises an aerial photography water area image layer and a water area algae small target segmentation layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010456414.0A CN111612779A (en) | 2020-05-26 | 2020-05-26 | Water area algae small target detection method and system based on aerial image and CIM |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010456414.0A CN111612779A (en) | 2020-05-26 | 2020-05-26 | Water area algae small target detection method and system based on aerial image and CIM |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111612779A true CN111612779A (en) | 2020-09-01 |
Family
ID=72205641
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010456414.0A Withdrawn CN111612779A (en) | 2020-05-26 | 2020-05-26 | Water area algae small target detection method and system based on aerial image and CIM |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111612779A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112052811A (en) * | 2020-09-11 | 2020-12-08 | 郑州大学 | Pasture grassland desertification detection method based on artificial intelligence and aerial image |
| CN112488020A (en) * | 2020-12-10 | 2021-03-12 | 西安交通大学 | Water environment pollution condition detection and evaluation device based on unmanned aerial vehicle aerial photography data |
| CN112860834A (en) * | 2021-02-05 | 2021-05-28 | 深圳力维智联技术有限公司 | WeBGIS-based third-party map docking device and method |
-
2020
- 2020-05-26 CN CN202010456414.0A patent/CN111612779A/en not_active Withdrawn
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112052811A (en) * | 2020-09-11 | 2020-12-08 | 郑州大学 | Pasture grassland desertification detection method based on artificial intelligence and aerial image |
| CN112488020A (en) * | 2020-12-10 | 2021-03-12 | 西安交通大学 | Water environment pollution condition detection and evaluation device based on unmanned aerial vehicle aerial photography data |
| CN112488020B (en) * | 2020-12-10 | 2023-09-19 | 西安交通大学 | Water environment pollution condition detection and evaluation device based on unmanned aerial vehicle aerial photographing data |
| CN112860834A (en) * | 2021-02-05 | 2021-05-28 | 深圳力维智联技术有限公司 | WeBGIS-based third-party map docking device and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111062905B (en) | Infrared and visible light fusion method based on saliency map enhancement | |
| CN111612779A (en) | Water area algae small target detection method and system based on aerial image and CIM | |
| CN101883291B (en) | Method for drawing viewpoints by reinforcing interested region | |
| CN110853037A (en) | Lightweight color point cloud segmentation method based on spherical projection | |
| CN112183788A (en) | Domain adaptive equipment operation detection system and method | |
| CN110796009A (en) | Method and system for detecting marine vessel based on multi-scale convolution neural network model | |
| CN110110682B (en) | Semantic stereo reconstruction method for remote sensing image | |
| CN105139350A (en) | Ground real-time reconstruction processing system for unmanned aerial vehicle reconnaissance images | |
| CN116883610A (en) | Digital twin intersection construction method and system based on vehicle identification and track mapping | |
| CN111599007B (en) | Smart city CIM road mapping method based on unmanned aerial vehicle aerial photography | |
| CN115527123A (en) | Land cover remote sensing monitoring method based on multi-source feature fusion | |
| CN112365467A (en) | Foggy image visibility estimation method based on single image depth estimation | |
| CN114998545A (en) | Three-dimensional modeling shadow recognition system based on deep learning | |
| CN109035307A (en) | Setting regions target tracking method and system based on natural light binocular vision | |
| Gao et al. | Large-scale synthetic urban dataset for aerial scene understanding | |
| CN116543227A (en) | Remote sensing image scene classification method based on graph convolution network | |
| CN113298177A (en) | Night image coloring method, device, medium, and apparatus | |
| CN112052811A (en) | Pasture grassland desertification detection method based on artificial intelligence and aerial image | |
| CN111626971B (en) | Smart city CIM real-time imaging method with image semantic perception | |
| CN108447042A (en) | The fusion method and system of urban landscape image data | |
| González et al. | Realistic correction of sky-coloured points in Mobile Laser Scanning point clouds | |
| CN117237553A (en) | Three-dimensional map mapping system based on point cloud image fusion | |
| CN111726535A (en) | Smart city CIM video big data image quality control method based on vehicle perception | |
| CN115170810B (en) | Visible light infrared image fusion target detection example segmentation method | |
| CN116977866A (en) | Lightweight landslide detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200901 |