CN118097721B - Wetland bird recognition method and system based on multi-source remote sensing observation and deep learning - Google Patents

Abstract

The invention discloses a wetland bird identification method and system based on multi-source remote sensing observation and deep learning, wherein the method comprises the following steps: acquiring a plurality of non-repeated orthographic images of birds covering a certain wetland area in overlooking shooting by an unmanned aerial vehicle; selecting a video frame with highest similarity with each orthographic image, and unifying the size to form a plurality of image pairs; manually labeling a plurality of groups of image pairs to obtain a training sample set and a verification sample set; inputting the training sample set into a bird recognition model for model training, and optimizing the bird recognition model by using the verification sample set to obtain a bird recognition model with high fitting degree; and re-acquiring an orthographic image and a video frame of the wetland area, and inputting a bird recognition model with high fitting degree after processing to obtain a bird recognition result. According to the bird recognition method, the bird features of the side view shot by the camera and the bird features of the orthographic image shot by the unmanned aerial vehicle are fused, so that the problem of reduced bird recognition accuracy caused by shielding of birds in a single-view image is solved.

Description

Wetland bird identification method and system based on multi-source remote sensing observation and deep learning

Technical Field

The present invention relates to the field of remote sensing image processing and geospatial information mining, and specifically to a wetland bird identification method and system based on multi-source remote sensing observation and deep learning.

Background technique

Bird identification is a necessary prerequisite for bird protection. Bird images can effectively distinguish different types of birds. With the rise of artificial intelligence, deep learning methods have begun to be applied to bird identification. Deep learning models identify bird species by learning bird features on images. However, birds live in pairs in wetlands. If the flocks are monitored from a single perspective, there will be mutual occlusion between birds in the captured images, making it impossible for deep learning models to accurately learn the features of the occluded birds, resulting in a decrease in recognition accuracy.

At present, there are two main devices for acquiring bird images: cameras and drones. The side view of birds taken by bird observation cameras can more clearly depict the side profile and detailed features of the bird, making it easier to identify the bird species; however, due to the problem of mutual occlusion between birds in a flock, the obscured birds are prone to identification errors; while in the orthophotos taken by drones, there is usually no occlusion problem between birds, but due to viewing angle and distance issues, the orthophotos can only show the overall shape of the bird, and it is difficult to determine its category based on the orthophotos alone.

Summary of the invention

In view of this, the purpose of the present invention is to provide a wetland bird identification method and system based on multi-source remote sensing observation and deep learning, which solves the problem of decreased bird identification accuracy caused by occlusion of birds in single-perspective images by fusing the bird features of the side view taken by the camera and the bird features of the orthographic image taken by the drone.

The technical solution adopted by the present invention is as follows: a wetland bird identification method based on multi-source remote sensing observation and deep learning, the steps are as follows:

Step S1, obtaining multiple orthophoto images of birds covering a wetland area taken by a drone from a bird's-eye view and synthesizing them into a panoramic orthophoto image, and then evenly cutting the panoramic orthophoto image to obtain multiple orthophoto images without duplication;

Step S2, based on the coordinate information and time information of the multiple non-repetitive orthophoto images obtained in step S1, intercept multiple video frames recorded by multiple cameras in the same range corresponding to each non-repetitive orthophoto image at the same time, perform similarity matching on the multiple video frames corresponding to each non-repetitive orthophoto image, select a video frame with the highest similarity to each non-repetitive orthophoto image to form an image pair, unify the size of the orthophoto image and the video frame in each group of image pairs, and finally form multiple groups of image pairs with the same size corresponding to the temporal and spatial positions;

Step S3, manually labeling the image pairs with the same size corresponding to the multiple groups of spatiotemporal positions obtained in step S2 to obtain a training sample set and a verification sample set;

Step S4, inputting the training sample set into the bird recognition model for model training, and optimizing the bird recognition model using the verification sample set to obtain a bird recognition model with a high degree of fit;

Step S5, reacquire multiple non-repetitive orthophoto images and video frames covering the wetland area, and input them into the bird recognition model with high fitting degree obtained in step S4 after being processed in steps S2 and S3 to obtain the bird recognition result.

Furthermore, in step S1, specifically:

The time period when the camera in a wetland area is working normally and the birds are resting is selected, and the route planning and flight mission of the wetland area are carried out by using a drone with RTK turned on. Multiple overlapping orthophotos covering the birds in the wetland area are obtained, and a panoramic orthophoto with coordinate information is generated by using image processing tools. Then, the panoramic orthophoto with coordinate information is evenly cut and obtained. non-repetitive orthophoto images, and extract the coordinate information and shooting time information of each orthophoto image.

Furthermore, in step S2, specifically:

Step S21, the wetland area has Each camera is at a fixed height, and it monitors the wetland area in all directions and takes clear photos of bird details;

Step S22, based on the result of step S1 Orthophotos without duplication, select one of the orthophotos , , according to the orthophoto The coordinate information and shooting time information, from Capture the same range and time from the video images recorded by the cameras Video frames , ;

Step S23: Video frames With orthophoto Use the image similarity matching algorithm to perform similarity matching and obtain the orthophoto image. The video frame with the highest similarity , , forming a set of image pairs , for image pairs Perform uniform size operation;

Step S24, loop through steps S22-S23 to find the image that matches the orthophoto , ,..., The video frame with the highest similarity , ,..., , and finally get A pair of images of the same size corresponding to a group of spatiotemporal positions , ,..., , is the first orthophoto, is the second orthophoto, For the Orthophoto images, is the video frame with the highest similarity to the first orthophoto image, is the video frame with the highest similarity to the second orthophoto image, For the The video frame with the highest similarity corresponding to the orthophoto.

Furthermore, in step S3, specifically:

Step S31, using data annotation software to annotate the data obtained in step S24 Annotate the image pairs of the same size corresponding to the group's temporal and spatial positions to obtain the position of each bird in the image pair and its category label;

Step S32: mark the The image pairs of the same size corresponding to the temporal and spatial positions are divided into a training sample set and a validation sample set, and the The training sample set and The validation sample set, .

Furthermore, in step S4, the bird recognition model includes an input layer, a convolution layer, a feature fusion layer, a fully connected layer, and an output layer. The training sample set is input into the bird recognition model for model training, and the verification sample set is used to optimize the bird recognition model, specifically:

Step S41, inputting the training sample set obtained in step S31 into the convolution layer through the input layer of the bird recognition model for feature extraction, thereby obtaining a first feature map of birds in the orthophoto image and a second feature map of birds in the video frame;

The training sample set is A pair of images of the same size corresponding to a group of spatiotemporal positions , ,..., , input into the convolutional layer of the bird recognition model for feature extraction, and obtain Group feature maps of the same size , ,..., ;in, is the first feature map of birds in the first orthophoto, is the first feature map of birds in the second orthophoto, For the The first characteristic map of birds in the orthophotos, is the second feature map of birds in the first video frame, is the second feature map of birds in the second video frame, For the The second feature map of birds in the video frame;

Step S42, weighted fusion of the first bird feature map of the orthophoto image and the second bird feature map of the video frame using the feature fusion layer of the bird recognition model to obtain a bird feature fusion map;

According to formula (1), the first feature map of birds in the orthophoto image and the second feature map of birds in the video frame are weightedly fused;

(1);

In the formula, For the A fusion map of bird features. For the The first characteristic map of birds in the orthophoto, Indicates The second feature map of birds in the video frame, , For the The weighting coefficient of the first feature map of birds in the orthophoto, For the The weighted coefficient of the second feature map of birds in each video frame, , and + =1;

Weighting coefficient of the first feature map of birds in the orthophoto and the weighting coefficient of the second feature map of birds in the video frame It is obtained through the attention mechanism. The specific steps are as follows:

Step S421, the bird first feature map of the orthophoto image and the bird second feature map of the video frame are input into the pooling layer in the feature fusion layer, and global average pooling operations are performed on each of them to obtain the global average pooling result of the bird first feature map , the global average pooling result of the second feature map of birds ;

Step S422: Global average pooling result of the first feature map of birds And the global average pooling result of the second feature map of birds After connection, it is input into the fully connected layer of the feature fusion layer to obtain the attention weight of the first feature map of the bird and the attention weight of the second feature map of birds ; The activation function used in the fully connected layer is Function, as shown in formula (2) and formula (3);

(2);

(3);

In the formula, , and , are the weight and bias of the fully connected layer respectively. The weight of the fully connected layer is initialized using a random initialization algorithm, and the bias of the fully connected layer is initialized to a constant;

Step S423: The attention weight of the first feature map of birds and the attention weight of the second feature map of birds pass The function is normalized to obtain the weighted coefficient of the first characteristic map of birds and the weighting coefficient of the second characteristic map of birds ; See formula (4);

(4);

Step S424, the step S41 obtained Group feature maps of the same size , ,..., The operation of steps S421-S423 is repeated to obtain Group feature map weighting coefficient , ,..., ,in The first feature map of the bird in the first orthophoto The weighting coefficient of The first feature map of the bird in the second orthophoto The weighting coefficient of For the First feature map of birds in orthophotos The weighting coefficient of The second feature map of the bird in the first video frame The weighting coefficient of The second feature map of the bird in the second video frame The weighting coefficient of For the The second feature map of birds in the video frame The weighting coefficient of Group feature maps of the same size , ,..., and Group feature map weighting coefficient , ,..., According to formula (1), weighted fusion is performed respectively, and finally the Bird feature fusion map , ,..., ,in represents the first bird feature fusion map, represents the second bird feature fusion map, Indicates A fusion map of bird features;

Step S43, inputting the bird feature fusion map into the fully connected layer of the bird recognition model to obtain a pre-trained bird recognition model, and optimizing the bird recognition model using the verification sample set in step S3 to obtain an optimized bird recognition model;

Step S431, the Bird feature fusion map , ,..., Connect and input into the fully connected layer of the bird recognition model for training to obtain a pre-trained bird recognition model, where This is the first bird feature fusion map. This is the second bird feature fusion map. For the A fusion map of bird features;

Step S432, using the verification sample set in step S3 to optimize the bird recognition model, specifically using a stochastic gradient descent method and a back propagation mechanism to minimize the loss function and obtain a bird recognition model with a high degree of fit;

The loss function uses the cross entropy loss function, as shown in formula (5);

(5);

In the formula, is the cross entropy loss function, is the number of samples, is the number of categories, It is The true label vector of the category The value of the element, when the sample The prediction result is the category If it is correct, it takes 1, otherwise it takes 0; is the model output sample Predict its category as category The probability value of .

Furthermore, in step S5, specifically:

During the time period when the camera in a wetland area described in step S1 is working normally and the birds are resting, the drone is reused to plan routes and perform flight missions in the wetland area. The acquired orthophotos and video frames are processed in steps S2 and S3 to obtain multiple groups of image pairs of the same size corresponding to the time and space positions. After inputting the bird recognition model with a high degree of fit obtained in step S4, the bird recognition results are obtained.

Furthermore, the present invention adopts another technical solution: a wetland bird identification system based on multi-source remote sensing observation and deep learning, which mainly includes the following modules:

Image acquisition module: The image acquisition module uses a drone to obtain multiple orthophoto images covering a wetland, and intercepts multiple corresponding video frames shot by a camera based on the orthophoto images;

Image processing module: The image processing module performs similarity matching and size unification operations on the images collected by the image acquisition module to obtain multiple groups of image pairs with the same size corresponding to the time and space positions;

Model training module: The model training module includes sample annotation of multiple groups of image pairs of the same size corresponding to the time and space positions obtained by the image processing module, and the division of training sample sets and verification sample sets, and then inputting the training sample sets into the bird recognition model for model training, and using the verification sample sets to optimize the model to obtain a bird recognition model with a high degree of fit; the model training module stops working after the model training and optimization are completed;

Bird recognition module: The bird recognition module inputs multiple groups of image pairs of the same size corresponding to the time and space positions obtained by the image processing module, and uses the bird recognition model with high fitting degree obtained by the model training module to obtain the bird recognition result.

The beneficial effects of the present invention are as follows: (1) The present invention complements the advantages of the camera and the drone by fusing the bird features of the side view taken by the camera and the bird features of the orthophoto image taken by the drone, thereby solving the problem of decreased bird recognition accuracy caused by occlusion of birds in a single-view image; (2) The present invention fuses the bird features of the orthophoto image with the bird features of the video frame through an attention mechanism, which is conducive to the deep learning model to comprehensively and deeply learn the bird features and improve the accuracy and confidence of bird recognition; (3) The present invention can be widely used in the identification of wetland birds, which is of great significance to the construction of a wetland bird species library and the protection of wetland birds.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart of the present invention.

FIG. 2 is a schematic diagram of a bird identification model of the present invention.

FIG. 3 is a flow chart of bird identification model training and optimization according to the present invention.

FIG. 4 is a schematic diagram of the system construction of the present invention.

Detailed ways

Referring to FIG1 , the technical solution adopted by the present invention is as follows: a wetland bird identification method based on multi-source remote sensing observation and deep learning, the steps of which are as follows:

Step S1, obtaining multiple orthophoto images of birds covering a wetland area taken by a drone from a bird's-eye view and synthesizing them into a panoramic orthophoto image, and then evenly cutting the panoramic orthophoto image to obtain multiple orthophoto images without duplication;

Step S2, based on the coordinate information and time information of the multiple non-repetitive orthophoto images obtained in step S1, intercept multiple video frames recorded by multiple cameras in the same range corresponding to each non-repetitive orthophoto image at the same time, perform similarity matching on the multiple video frames corresponding to each non-repetitive orthophoto image, select a video frame with the highest similarity to each non-repetitive orthophoto image to form an image pair, unify the size of the orthophoto image and the video frame in each group of image pairs, and finally form multiple groups of image pairs with the same size corresponding to the temporal and spatial positions;

Step S3, manually labeling the image pairs with the same size corresponding to the multiple groups of spatiotemporal positions obtained in step S2 to obtain a training sample set and a verification sample set;

Step S4, inputting the training sample set into the bird recognition model for model training, and optimizing the bird recognition model using the verification sample set to obtain a bird recognition model with a high degree of fit;

Step S5, reacquire multiple non-repetitive orthophoto images and video frames covering the wetland area, and input them into the bird recognition model with high fitting degree obtained in step S4 after being processed in steps S2 and S3 to obtain the bird recognition result.

Furthermore, in step S1, specifically:

The time period when the camera in a wetland area is working normally and the birds are resting is selected, and the route planning and flight mission of the wetland area are carried out by using a drone with RTK turned on. Multiple overlapping orthophotos covering the birds in the wetland area are obtained, and a panoramic orthophoto image with coordinate information is generated by using image processing tools (such as PhotoScan). The panoramic orthophoto image with coordinate information is then evenly cut to obtain non-repetitive orthophoto images, and extract the coordinate information and shooting time information of each orthophoto image.

Furthermore, in step S2, specifically:

Step S21, the wetland area has Cameras, each with a fixed height of 1.7m, monitor the wetland area in all directions and take clear photos of bird details;

Step S22, based on the result of step S1 Orthophotos without duplication, select one of the orthophotos , , according to the orthophoto The coordinate information and shooting time information, from Capture the same range and time from the video images recorded by the cameras Video frames , ;

Step S23: Video frames With orthophoto Using image similarity matching algorithms (such as feature matching algorithms Perform similarity matching to obtain orthophoto images The video frame with the highest similarity , , forming a set of image pairs , for image pairs Perform uniform size operation;

Step S24, loop through steps S22-S23 to find the image that matches the orthophoto , ,..., The video frame with the highest similarity , ,..., , and finally get A pair of images of the same size corresponding to a group of spatiotemporal positions , ,..., , is the first orthophoto, is the second orthophoto, For the Orthophoto images, is the video frame with the highest similarity to the first orthophoto image, is the video frame with the highest similarity to the second orthophoto image, For the The video frame with the highest similarity corresponding to the orthophoto.

Furthermore, in step S3, specifically:

Step S31, using data labeling software (such as LabelImg) to label the data obtained in step S24 Annotate the image pairs of the same size corresponding to the group's temporal and spatial positions to obtain the position of each bird in the image pair and its category label;

Step S32: mark the The image pairs of the same size corresponding to the temporal and spatial positions are divided into a training sample set and a validation sample set, and the The training sample set and The validation sample set, .

Further, referring to FIG2 , in step S4, the bird recognition model includes an input layer, a convolution layer, a feature fusion layer, a fully connected layer, and an output layer. The training sample set is input into the bird recognition model for model training, and the bird recognition model is optimized using the verification sample set. Referring to FIG3 , specifically:

Step S41, inputting the training sample set obtained in step S31 into the convolution layer through the input layer of the bird recognition model for feature extraction, thereby obtaining a first feature map of birds in the orthophoto image and a second feature map of birds in the video frame;

The training sample set is A pair of images of the same size corresponding to a group of spatiotemporal positions , ,..., , which is input into the convolutional layer of the bird recognition model for feature extraction, and we get Group feature maps of the same size , ,..., ;in, is the first feature map of birds in the first orthophoto, is the first feature map of birds in the second orthophoto, For the The first characteristic map of birds in the orthophotos, is the second feature map of birds in the first video frame, is the second feature map of birds in the second video frame, For the The second feature map of birds in the video frame;

Step S42, weighted fusion of the first bird feature map of the orthophoto image and the second bird feature map of the video frame using the feature fusion layer of the bird recognition model to obtain a bird feature fusion map;

According to formula (1), the first feature map of birds in the orthophoto image and the second feature map of birds in the video frame are weightedly fused;

(1);

In the formula, For the A fusion map of bird features. For the The first characteristic map of birds in the orthophoto, Indicates The second feature map of birds in the video frame, , For the The weighting coefficient of the first feature map of birds in the orthophoto, For the The weighted coefficient of the second feature map of birds in each video frame, , and + =1;

Furthermore, the weighting coefficient of the first feature map of the bird in the orthophoto is and the weighting coefficient of the second feature map of birds in the video frame It is obtained through the attention mechanism. The specific steps are as follows:

Step S421, the bird first feature map of the orthophoto image and the bird second feature map of the video frame are input into the pooling layer in the feature fusion layer, and global average pooling operations are performed on each of them to obtain the global average pooling result of the bird first feature map , the global average pooling result of the second feature map of birds ;

Step S422: Global average pooling result of the first feature map of birds And the global average pooling result of the second feature map of birds After connection, it is input into the fully connected layer of the feature fusion layer to obtain the attention weight of the first feature map of the bird and the attention weight of the second feature map of birds ; The activation function used in the fully connected layer is Function, as shown in formula (2) and formula (3);

(2);

(3);

In the formula, , and , are the weight and bias of the fully connected layer respectively. The weight of the fully connected layer is initialized using a random initialization algorithm, and the bias of the fully connected layer is initialized to a constant;

Step S423: The attention weight of the first feature map of birds and the attention weight of the second feature map of birds pass The function is normalized to obtain the weighted coefficient of the first characteristic map of birds and the weighting coefficient of the second characteristic map of birds ; See formula (4);

(4);

Step S424, the step S41 obtained Group feature maps of the same size , ,..., The operation of steps S421-S423 is repeated to obtain Group feature map weighting coefficient , ,..., ,in The first feature map of the bird in the first orthophoto The weighting coefficient of The first feature map of the bird in the second orthophoto The weighting coefficient of For the First feature map of birds in orthophotos The weighting coefficient of The second feature map of the bird in the first video frame The weighting coefficient of The second feature map of the bird in the second video frame The weighting coefficient of For the The second feature map of birds in the video frame The weighting coefficient of Group feature maps of the same size , ,..., and Group feature map weighting coefficient , ,..., According to formula (1), weighted fusion is performed respectively, and finally the Bird feature fusion map , ,..., ,in represents the first bird feature fusion map, represents the second bird feature fusion map, Indicates A fusion map of bird features;

Step S43, inputting the bird feature fusion map into the fully connected layer of the bird recognition model to obtain a pre-trained bird recognition model, and optimizing the bird recognition model using the verification sample set in step S3 to obtain an optimized bird recognition model;

Step S431: the Bird feature fusion map , ,..., Connect and input into the fully connected layer of the bird recognition model for training to obtain a pre-trained bird recognition model, where This is the first bird feature fusion map. This is the second bird feature fusion map. For the A fusion map of bird features;

Step S432, using the verification sample set in step S3 to optimize the bird recognition model, specifically using a stochastic gradient descent method and a back propagation mechanism to minimize the loss function and obtain a bird recognition model with a high degree of fit;

The loss function uses the cross entropy loss function, as shown in formula (5);

(5);

Where L is the cross entropy loss function, is the number of samples, is the number of categories, It is The true label vector of the category The value of the element, when the sample The prediction result is the category If it is correct, it takes 1, otherwise it takes 0; is the model output sample Predict its category as category The probability value of .

Furthermore, in step S5, specifically:

During the time period when the camera in a wetland area described in step S1 is working normally and the birds are resting, the drone is reused to plan routes and perform flight missions in the wetland area. The acquired orthophotos and video frames are processed in steps S2 and S3 to obtain multiple groups of image pairs of the same size corresponding to the time and space positions. After inputting the bird recognition model with a high degree of fit obtained in step S4, the bird recognition results are obtained.

Furthermore, the present invention also provides a technical solution: Referring to FIG. 4 , a wetland bird identification system based on multi-source remote sensing observation and deep learning mainly includes the following modules:

Image acquisition module: The image acquisition module uses a drone to obtain multiple orthophoto images covering a wetland, and intercepts multiple corresponding video frames shot by a camera based on the orthophoto images;

Image processing module: The image processing module performs similarity matching and size unification operations on the images collected by the image acquisition module to obtain multiple groups of image pairs with the same size corresponding to the time and space positions;

Model training module: The model training module includes sample annotation of multiple groups of image pairs of the same size corresponding to the time and space positions obtained by the image processing module, and the division of training sample sets and verification sample sets, and then inputting the training sample sets into the bird recognition model for model training, and using the verification sample sets to optimize the model to obtain a bird recognition model with a high degree of fit; the model training module stops working after the model training and optimization are completed;

Bird recognition module: The bird recognition module inputs multiple groups of image pairs of the same size corresponding to the time and space positions obtained by the image processing module, and uses the bird recognition model with high fitting degree obtained by the model training module to obtain the bird recognition result.

The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for ordinary technicians in this field, some modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the attached claims.

Claims (6)

1. A wetland bird identification method based on multi-source remote sensing observation and deep learning, characterized in that the steps are as follows: Step S1, obtaining multiple orthophoto images of birds covering a wetland area taken by a drone from a bird's-eye view and synthesizing them into a panoramic orthophoto image, and then evenly cutting the panoramic orthophoto image to obtain multiple orthophoto images without duplication; Step S2, based on the coordinate information and time information of the multiple non-repetitive orthophoto images obtained in step S1, intercept multiple video frames recorded by multiple cameras at the same time in the same range corresponding to each non-repetitive orthophoto image, perform similarity matching on the multiple video frames corresponding to each non-repetitive orthophoto image, select a video frame with the highest similarity to each non-repetitive orthophoto image to form an image pair, unify the size of the orthophoto image and the video frame in each group of image pairs, and finally form multiple groups of image pairs with the same size corresponding to the temporal and spatial positions; Step S3, manually labeling the image pairs with the same size corresponding to the multiple groups of spatiotemporal positions obtained in step S2 to obtain a training sample set and a verification sample set; Step S4, inputting the training sample set into the bird recognition model for model training, and optimizing the bird recognition model using the verification sample set to obtain a bird recognition model with a high degree of fit; Step S5, reacquiring a plurality of non-repetitive orthophoto images and video frames covering the wetland area, and inputting the bird recognition model with a high degree of fit obtained in step S4 after being processed in steps S2 and S3 to obtain a bird recognition result; In step S4, the bird recognition model includes an input layer, a convolution layer, a feature fusion layer, a fully connected layer, and an output layer. The training sample set is input into the bird recognition model for model training, and the verification sample set is used to optimize the bird recognition model. Specifically, Step S41, inputting the training sample set obtained in step S31 into the convolution layer through the input layer of the bird recognition model for feature extraction, thereby obtaining a first feature map of birds in the orthophoto image and a second feature map of birds in the video frame; The training sample set is A pair of images of the same size corresponding to a group of spatiotemporal positions , ,..., , input into the convolutional layer of the bird recognition model for feature extraction, and obtain Group feature maps of the same size , ,..., ;in, is the first feature map of birds in the first orthophoto, is the first feature map of birds in the second orthophoto, For the The first characteristic map of birds in the orthophoto, is the second feature map of birds in the first video frame, is the second feature map of birds in the second video frame, For the The second feature map of birds in the video frame; Step S42, weighted fusion is performed on the first feature map of birds in the orthophoto image and the second feature map of birds in the video frame using the feature fusion layer of the bird recognition model to obtain a bird feature fusion map; According to formula (1), the first feature map of birds in the orthophoto image and the second feature map of birds in the video frame are weightedly fused; (1); In the formula, For the A fusion map of bird features. For the The first characteristic map of birds in the orthophotos, Indicates The second feature map of birds in the video frame, , For the The weighting coefficient of the first feature map of birds in the orthophoto, For the The weighted coefficient of the second feature map of birds in each video frame, , and + =1; Weighting coefficient of the first feature map of birds in the orthophoto and the weighting coefficient of the second feature map of birds in the video frame It is obtained through the attention mechanism. The specific steps are as follows: Step S421, the bird first feature map of the orthophoto image and the bird second feature map of the video frame are input into the pooling layer in the feature fusion layer, and global average pooling operations are performed on each of them to obtain the global average pooling result of the bird first feature map , the global average pooling result of the second feature map of birds ; Step S422: Global average pooling result of the first feature map of birds And the global average pooling result of the second feature map of birds After connection, it is input into the fully connected layer of the feature fusion layer to obtain the attention weight of the first feature map of the bird and the attention weight of the second feature map of birds ; The activation function used in the fully connected layer is Function, as shown in formula (2) and formula (3); (2); (3); In the formula, , and , are the weight and bias of the fully connected layer respectively. The weight of the fully connected layer is initialized using a random initialization algorithm, and the bias of the fully connected layer is initialized to a constant; Step S423: The attention weight of the first feature map of birds and the attention weight of the second feature map of birds pass The function is normalized to obtain the weighted coefficient of the first characteristic map of birds and the weighting coefficient of the second characteristic map of birds ; See formula (4); (4); Step S424, the step S41 obtained Group feature maps of the same size , ,..., The operation of steps S421-S423 is repeated to obtain Group feature map weighting coefficient , ,..., ,in The first feature map of the bird in the first orthophoto The weighting coefficient of The first feature map of the bird in the second orthophoto The weighting coefficient of For the First feature map of birds in orthophotos The weighting coefficient of The second feature map of the bird in the first video frame The weighting coefficient of The second feature map of the bird in the second video frame The weighting coefficient of For the Second feature map of birds in video frames The weighting coefficient of Group feature maps of the same size , ,..., and Group feature map weighting coefficient , ,..., According to formula (1), weighted fusion is performed respectively, and finally the Bird feature fusion map , ,..., ,in represents the first bird feature fusion map, represents the second bird feature fusion map, Indicates A fusion map of bird features; Step S43, inputting the bird feature fusion map into the fully connected layer of the bird recognition model to obtain a pre-trained bird recognition model, and optimizing the bird recognition model using the verification sample set in step S3 to obtain an optimized bird recognition model; Step S431, the Bird feature fusion map , ,..., Connect and input into the fully connected layer of the bird recognition model for training to obtain a pre-trained bird recognition model, where This is the first bird feature fusion map. This is the second bird feature fusion map. For the A fusion map of bird features; Step S432, using the verification sample set in step S3 to optimize the bird recognition model, specifically using a stochastic gradient descent method and a back propagation mechanism to minimize the loss function and obtain a bird recognition model with a high degree of fit; The loss function uses the cross entropy loss function, as shown in formula (5); (5); In the formula, is the cross entropy loss function, is the number of samples, is the number of categories, It is The true label vector of the category The value of the element, when the sample The prediction result is the category If it is correct, it takes 1, otherwise it takes 0; is the model output sample Predict its category as category The probability value of . 2. The wetland bird identification method based on multi-source remote sensing observation and deep learning according to claim 1 is characterized in that: in step S1, specifically: The time period when the camera in a wetland area is working normally and the birds are resting is selected, and the route planning and flight mission of the wetland area are carried out by using a drone with RTK turned on. Multiple overlapping orthophotos covering the birds in the wetland area are obtained, and a panoramic orthophoto image with coordinate information is generated by using image processing tools. Then, the panoramic orthophoto image with coordinate information is evenly cut and obtained. non-repetitive orthophoto images, and extract the coordinate information and shooting time information of each orthophoto image. 3. The wetland bird identification method based on multi-source remote sensing observation and deep learning according to claim 2 is characterized in that: in step S2, specifically: Step S21, the wetland area has Each camera is at a fixed height, and the cameras monitor the wetland area in all directions and take clear photos of bird details; Step S22, based on the result of step S1 Orthophotos without duplication, select one of the orthophotos , , according to the orthophoto The coordinate information and shooting time information, from Capture the same range and time from the video images recorded by the cameras Video frames , ; Step S23: Video frames With orthophoto Use the image similarity matching algorithm to perform similarity matching and obtain the orthophoto image. The video frame with the highest similarity , , forming a set of image pairs , for image pairs Perform uniform size operation; Step S24, loop through steps S22-S23 to find the image that matches the orthophoto , ,..., The video frame with the highest similarity , ,..., , and finally get A pair of images of the same size corresponding to a group of spatiotemporal positions , ,..., , is the first orthophoto, is the second orthophoto, For the Orthophoto images, is the video frame with the highest similarity to the first orthophoto image, is the video frame with the highest similarity to the second orthophoto image, For the The video frame with the highest similarity corresponding to the orthophoto image. 4. The wetland bird identification method based on multi-source remote sensing observation and deep learning according to claim 3 is characterized in that: in step S3, specifically: Step S31, using data annotation software to annotate the data obtained in step S24 Annotate the image pairs of the same size corresponding to the group's temporal and spatial positions to obtain the position of each bird in the image pair and its category label; Step S32: mark the The image pairs of the same size corresponding to the temporal and spatial positions are divided into a training sample set and a validation sample set, and the The training sample set and The validation sample set, . 5. The wetland bird identification method based on multi-source remote sensing observation and deep learning according to claim 4 is characterized in that: in step S5, specifically: During the time period when the camera in a wetland area described in step S1 is working normally and the birds are resting, the drone is reused to plan routes and perform flight missions in the wetland area. The acquired orthophotos and video frames are processed in steps S2 and S3 to obtain multiple groups of image pairs of the same size corresponding to the time and space positions. After inputting the bird recognition model with a high degree of fit obtained in step S4, the bird recognition results are obtained. 6. A wetland bird identification system based on multi-source remote sensing observation and deep learning, applied to the wetland bird identification method based on multi-source remote sensing observation and deep learning as described in claim 5, characterized in that it mainly includes the following modules: Image acquisition module: The image acquisition module uses a drone to obtain multiple orthophoto images covering a wetland, and intercepts multiple corresponding video frames shot by a camera based on the orthophoto images; Image processing module: The image processing module performs similarity matching and size unification operations on the images collected by the image acquisition module to obtain multiple groups of image pairs with the same size corresponding to the time and space positions; Model training module: The model training module includes sample annotation of multiple groups of image pairs of the same size corresponding to the time and space positions obtained by the image processing module, and the division of training sample sets and verification sample sets, and then inputting the training sample sets into the bird recognition model for model training, and using the verification sample sets to optimize the model to obtain a bird recognition model with a high degree of fit; the model training module stops working after the model training and optimization are completed; Bird recognition module: The bird recognition module inputs multiple groups of image pairs of the same size corresponding to the time and space positions obtained by the image processing module, and uses the bird recognition model with high fitting degree obtained by the model training module to obtain the bird recognition result.