CN118097463A

CN118097463A - Lodging area identification method and system based on crop remote sensing image

Info

Publication number: CN118097463A
Application number: CN202410147602.3A
Authority: CN
Inventors: 李丽; 张恒; 傅玉祥; 何书专; 钟悦; 邹幸洁
Original assignee: Nanjing Ningqi Intelligent Computing Chip Research Institute Co ltd
Current assignee: Nanjing Ningqi Intelligent Computing Chip Research Institute Co ltd
Priority date: 2024-02-01
Filing date: 2024-02-01
Publication date: 2024-05-28

Abstract

The application discloses a lodging area identification method and system based on crop remote sensing images, which belong to the technical field of image processing and comprise the following steps: remote sensing image acquisition is carried out on a crop field through an unmanned plane; performing image stitching, radiometric calibration and image cutting pretreatment on the acquired remote sensing images; extracting features of the processed remote sensing image to obtain color features, texture features and vegetation index features; and identifying crop lodging areas in the images based on a semantic segmentation algorithm of the convolutional neural network. Aiming at the problem of low recognition precision of the crop lodging area in the prior art, the application uses the unmanned aerial vehicle to acquire the high-definition remote sensing image of the target area, realizes the semantic segmentation of the image by constructing the convolutional neural network model, and can automatically recognize the crop lodging area. The learning capability of the model on the fine features is improved by adopting an attention mechanism and multi-scale feature fusion, and the recognition accuracy of the crop lodging area is improved.

Description

Lodging area identification method and system based on crop remote sensing image

Technical Field

The application relates to the technical field of image processing, in particular to a lodging area identification method and system based on crop remote sensing images.

Background

Agriculture is an indispensable part of human life, and crops are one of main grain crops, and improvement of yield and quality is critical to grain safety. However, crop lodging is a common problem, especially when subjected to extreme weather events or abnormal conditions during the growing period. Lodging causes the inability of crops to grow normally, affecting yield and quality. Therefore, timely and accurate identification of crop lodging areas is critical to taking effective countermeasures. Traditional crop lodging recognition methods are often limited by manual visual inspection or simple image processing technology, and have low recognition accuracy, so that the requirements of agricultural production are difficult to meet. With the rapid development of remote sensing technology and deep learning, the application of the advanced technology to the agricultural field becomes an effective way for improving the lodging recognition precision of crops.

In the prior art, the identification accuracy of the lodging area of crops is challenging. The traditional image processing method generally cannot fully consider the influence of complex terrains, vegetation and illumination conditions on the image, so that the recognition result is inaccurate. Therefore, a more advanced and accurate method is urgently needed to cope with this problem.

The Chinese patent application, application number CN202210658126.2, publication day 2022, 9 and 13, discloses a crop lodging area extraction method and device based on unmanned aerial vehicle images, wherein the method comprises the following steps: 1, acquiring RGB image data of crops in a field, and performing splicing, cutting and labeling to obtain a research area diagram and label data; 2, obtaining a crop digital surface model image by utilizing RGB images of a crop research area, and fusing the crop RGB images and the digital surface model image to obtain a crop RGB-digital surface model image; 3, constructing a crop lodging segmentation model based on deep learning, wherein the crop lodging segmentation model comprises a space path, a context path and a feature fusion module, and the context path comprises ResNet networks and pyramid segmentation attentions; and 4, training a crop lodging segmentation model through crop RGB-digital surface model image data, and segmenting a lodging area of a crop image, so that the lodging area is estimated by using a segmentation result. In this approach, however, resNet networks and pyramid segmentation attention are used as context paths. The selection of the network structure has an influence on the complex feature extraction of the lodging area, and if the selected network structure is not suitable for the lodging scene of crops, the identification accuracy may be low.

Disclosure of Invention

1. Technical problem to be solved

Aiming at the problem of low recognition precision of the crop lodging area in the prior art, the application provides a lodging area recognition method and system based on a crop remote sensing image, which are used for carrying out semantic segmentation and the like by collecting the remote sensing image and a convolutional neural network through an unmanned aerial vehicle, so that the recognition precision of the crop lodging area is improved.

2. Technical proposal

The aim of the application is achieved by the following technical scheme.

An aspect of embodiments of the present disclosure provides a lodging area identification method based on crop remote sensing images, including: collecting remote sensing images of crops; performing image stitching, radiometric calibration and image cutting pretreatment on the acquired remote sensing images; extracting features of the processed remote sensing image to obtain color features, texture features and vegetation index features; and identifying crop lodging areas in the images based on a semantic segmentation algorithm of the convolutional neural network.

Furthermore, the convolutional neural network adopts PSPNet networks; the encoder adopts ResNet of pre-training as the encoder to extract the characteristics of the input image; the fusion module comprises convolution layers of pooling cores with different sizes, and performs multi-scale pooling on the characteristics extracted by the encoder to obtain characteristic diagrams with different scales; the decoder comprises an n-level up-sampling layer and an n-level convolution layer, wherein the i-level up-sampling layer amplifies the i-level feature image to the size of an i+1-level feature image, and the i-level convolution layer fuses the feature image output by the i-level up-sampling layer with the feature image output by the i-level up-sampling layer.

Further, the method comprises the following steps: extracting features of an input image through a convolution layer to serve as a first feature map; performing multi-scale pooling on the first feature map by using pooling check of different sizes to obtain pooling feature maps of different scales as a second feature map; feature fusion is carried out on the second feature graphs with different scales, convolution processing is carried out on the feature graphs with the minimum scale, up-sampling or down-sampling and splicing processing are carried out on the feature graphs except the feature graphs with the minimum scale, and a spliced feature graph is obtained; fusing the spliced feature images by adopting an attention mechanism to obtain a fused feature image; up-sampling the fusion feature map through deconvolution to obtain an up-sampling feature map; and performing dot multiplication, splicing and convolution operation on the up-sampling feature map and the minimum scale feature map, and outputting an image containing a lodging area prediction result.

Further, feature fusion is carried out on the second feature graphs with different scales, and the method comprises the following steps: taking the second feature images with different scales as input, and inputting the second feature images into a fusion module; performing first feature extraction on the second feature map through the first convolution layer, and extracting feature maps with different scales to serve as a first feature map; carrying out bidirectional space separation convolution on the first feature map by adopting a second convolution layer, and extracting feature maps with different scales to serve as a second feature map, wherein the bidirectional space separation convolution comprises transverse one-dimensional convolution and longitudinal one-dimensional convolution; splicing the second feature images extracted through the bidirectional space separation convolution to obtain spliced feature images; and carrying out convolution operation on the spliced feature images by adopting a third convolution layer, and outputting the feature images after fusion.

Further, when the transverse one-dimensional convolution is performed, the length of the convolution kernel is set to be larger than the width of the convolution kernel, and the convolution kernel is used for extracting transverse features of the image; when longitudinal one-dimensional convolution is performed, the width of the convolution kernel is set to be larger than the length of the convolution kernel, and the convolution kernel is used for extracting longitudinal features of an image.

Further, remote sensing image acquisition is carried out on a crop field through an unmanned aerial vehicle, and the method comprises the following steps: the unmanned aerial vehicle collects remote sensing images of the crop fields through the multispectral camera; the multispectral camera performs RGB three-channel splitting on the acquired remote sensing image to respectively acquire three single-channel images of red, green and blue; and carrying out channel-division storage on the acquired red, green and blue three single-channel images so as to reduce the storage capacity.

Further, image stitching is performed on the acquired remote sensing images, and the method comprises the following steps: the acquired single-channel images are arranged and combined according to time sequence; carrying out image fusion processing on the part of the adjacent images with an overlapping area, wherein the overlapping area represents a commonly covered ground surface area generated by shooting at the same geographic position; and (5) splicing to generate an integral remote sensing image covering the target crop field.

Further, the method for performing radiation calibration on the collected remote sensing image comprises the following steps: collecting illumination parameters when the unmanned aerial vehicle acquires a remote sensing image; constructing an image radiometric calibration model by utilizing the collected illumination parameters; and correcting the illumination information of each pixel in the spliced integral remote sensing image through the constructed image radiation calibration model so as to eliminate the influence of different illumination conditions.

Further, the method for clipping the collected remote sensing image comprises the following steps: using the integral remote sensing image after radiometric calibration as an input image; acquiring a field area and a background area of a target crop in an input image through an image segmentation algorithm; calculating the boundary of the target crop field area through a boundary detection algorithm, and obtaining the polygon area coordinates of the target crop field area; and cutting and storing the input image according to the acquired polygonal region coordinates to serve as a target region.

Another aspect of the embodiments of the present disclosure further provides a lodging area identifying system based on a remote sensing image of a crop, for executing a lodging area identifying method based on a remote sensing image of a crop of the present application, including: the image acquisition module acquires remote sensing images of the crop fields; the preprocessing module is used for splicing, radiating and calibrating the collected remote sensing images and cutting the images; the feature extraction module is used for extracting features of the preprocessed remote sensing image to obtain color features, texture features and vegetation index features respectively; the recognition module is used for carrying out semantic segmentation on the remote sensing image after feature extraction based on a pre-trained convolutional neural network model and recognizing a crop lodging area in the remote sensing image, wherein the convolutional neural network model adopts a PSPNet structure; PSPNet the structure comprises: an encoder for feature extraction; the fusion module is used for multi-scale feature fusion; and a decoder for pixel prediction and up-sampling to the original resolution.

3. Advantageous effects

Compared with the prior art, the application has the advantages that:

the unmanned aerial vehicle is adopted to acquire multispectral crop remote sensing images, and the semantic segmentation algorithm of the convolutional neural network is combined, so that the high-precision identification of the crop lodging area is realized; the adoption of the multi-scale pooling and feature fusion module is beneficial to capturing the feature information of different scales in the crop image, and improves the sensitivity to the lodging area; the feature map is fused by using the attention mechanism, so that the model is facilitated to pay more attention to the key part of the lodging area, and the accuracy and the robustness of the model are improved.

The acquired images are arranged in time sequence and fused, so that image splicing gaps caused by unmanned aerial vehicle track change or other factors can be eliminated, inconsistency in the images is reduced, and noise possibly introduced is reduced; aiming at the part of the adjacent images with the overlapping area, the image fusion processing is carried out, so that the information of the overlapping area is more smoothly transited, and possible noise and discontinuity are reduced; and identifying the field area of the target crop by an image segmentation algorithm, and cutting the area according to a boundary detection algorithm to obtain an image subset containing the target area. The clipping operation is helpful to remove irrelevant areas, concentrate on analyzing target areas and improve the efficiency and stability of subsequent algorithms.

The illumination parameters and the radiometric calibration model are introduced, the whole remote sensing image is corrected, the influence of different illumination conditions on the identification result is eliminated, and the robustness of the system is improved; the illumination parameters are collected, an image radiometric calibration model is constructed, and the illumination information of each pixel in the whole remote sensing image can be corrected. This helps to obtain more consistent images under different lighting conditions, improving the stability and consistency of subsequent algorithms to image features.

By introducing a bi-directional spatial separation convolution, feature extraction can be performed simultaneously in both the lateral and longitudinal directions. The parallel feature extraction mechanism is helpful for the model to more comprehensively understand the structure and details in the image, and the fine granularity recognition capability of the crop lodging area is enhanced; the combination of the lateral and longitudinal convolutions enables the model to more sharply capture subtle lateral and longitudinal features in the image, including texture, shape and orientation information of the crop lodging area; the use of the bidirectional spatial separation convolution enhances the perception of the model to the image space structure; by convolving in the transverse and longitudinal directions, respectively, the model can better maintain the spatial resolution of the image, improve the sensitivity to fine structures, and facilitate more accurate division of the crop lodging area.

Splitting an image into RGB channels and storing it in separate channels means that the information for each channel can be stored separately. This helps to reduce storage requirements while retaining important color information. Storing each channel of an image instead of the entire RGB image can significantly reduce storage requirements. This is particularly important for remote sensing image data of large-scale farmland, and reduces the burden of data management and transmission; the split channel storage allows independent analysis of each color channel, making subsequent image processing and analysis more flexible. This has a positive impact on the tasks of vegetation index calculation, vegetation health assessment, etc.

In summary, the unmanned aerial vehicle is used for collecting high-definition remote sensing images of the target area, image semantic segmentation is achieved by constructing a convolutional neural network model, and the crop lodging area can be automatically identified. The learning capability of the model on the fine features is improved by adopting an attention mechanism and multi-scale feature fusion, and the recognition accuracy of the crop lodging area is improved.

Drawings

Fig. 1 is a schematic diagram of a lodging area identification method based on a remote sensing image of crops according to an embodiment of the application;

fig. 2 is a schematic diagram of a system architecture of an automatic wheat remote sensing image acquisition unmanned aerial vehicle according to an embodiment of the present application;

FIG. 3 is a PSPNet model diagram of an embodiment of the present application;

fig. 4 is a diagram showing a wheat lodging area recognition result according to an embodiment of the present application.

Detailed Description

The application will now be described in detail with reference to the drawings and the accompanying specific examples.

Fig. 1 is a schematic diagram of a lodging area identification method based on a remote sensing image of crops according to an embodiment of the application, which includes the following steps: collecting data; preprocessing data; constructing a data set; and (5) extracting characteristics. The unmanned aerial vehicle multispectral camera is used for carrying out low-altitude flight shooting on the crop field, and remote sensing image data of different time phases are obtained. The obtained image data includes a visible light image and a near infrared image. And performing preprocessing such as image stitching, image cutting, image registration and the like on the acquired original remote sensing image data, removing an invalid region, and constructing an orthographic image which precisely covers the target field. And performing atmospheric correction, denoising treatment and the like to obtain a preprocessed high-quality image. And selecting a representative sample from the preprocessed image, and extracting an image block construction data set. The constructed dataset includes normal growth samples and lodging samples, wherein the lodging samples cover varying degrees of mild and severe lodging conditions. And extracting characteristic information expressing the growth state of crops based on the constructed sample data set, wherein the characteristic information comprises characteristics such as color, texture, morphology and the like. The extracted features become key criteria for crop lodging identification.

Fig. 2 is a schematic diagram of an automatic wheat remote sensing image acquisition unmanned aerial vehicle system according to an embodiment of the present application, including a quad-rotor unmanned aerial vehicle, a multispectral camera mounted thereon, a ground base station, a server, and a client. Wherein: the four-rotor unmanned aerial vehicle automatically cruises according to the set route and parameters to execute a field cruising task; the multispectral camera carried by the four-rotation unmanned aerial vehicle is used for shooting remote sensing images in the field; the ground base station is used for parking the four-rotor unmanned aerial vehicle, providing a charging interface for the unmanned aerial vehicle, receiving remote sensing data acquired by the multispectral camera through a wireless transceiver in the cabin, and then transmitting the remote sensing data to the server through a wireless network; the server is used for collecting, storing and processing the remote sensing images acquired by the multispectral camera; the client is used for sending control instructions to the unmanned aerial vehicle and the ground base station and displaying the processing result of the server on the remote sensing image to the user.

Specifically, the crop lodging monitoring system comprises a quadrotor unmanned aerial vehicle, a multispectral camera carried by the quadrotor unmanned aerial vehicle, a ground base station, a server and a client. Wherein: four rotor unmanned aerial vehicle: the remote sensing image collecting device is used for automatically cruising in the field of the target crops according to the preset route and flight parameters, and completing the remote sensing image collecting task. The unmanned aerial vehicle realizes automatic tracking of the cruising route through the autonomous flight control system. Multispectral camera: the device is arranged on an airborne cradle head of a quad-rotor unmanned aerial vehicle and comprises two imaging spectrum sensors of visible light and near infrared. In the cruising process of the unmanned aerial vehicle, multi-angle shooting is carried out on the crop field below, and original image data of different spectrums are obtained. Ground base station: the device is arranged on the open ground beside a crop field and is used for sending control instructions and positioning signals to realize remote intelligent control of the four-rotor unmanned aerial vehicle. And may receive image data transmitted by the multispectral camera. And (3) a server: the method has high-speed operation processing capability, is used for storing and preprocessing received original images, constructing a deep learning model for monitoring crop lodging, and realizing intelligent recognition and positioning of crop lodging areas in the images. Client side: the operation management software of the monitoring system is installed and can be used for planning a route and setting parameters, and monitoring the working state of the unmanned aerial vehicle and the image recognition result in real time.

Specifically, ground basic station is used for parking four rotor unmanned aerial vehicle, provides the interface that electric power charges for unmanned aerial vehicle, satisfies its task rotation requirement of charging. And a high-sensitivity wireless signal transceiver is also arranged in the ground base station and is used for receiving the original remote sensing image data transmitted by the multispectral camera in real time in the flight process. The ground base station transmits the received image data to a server side of the monitoring system through a wireless network transmission system. The server has high-speed storage and data processing capabilities. It can receive and store a large amount of raw remote sensing image data transmitted from a ground base station. And a series of preprocessing operations such as image format conversion, image clipping, image stitching and the like can be performed. The server also performs feature extraction and training sample selection on the preprocessed image data to construct a deep learning model for monitoring and identifying crop lodging. Finally, the trained model is utilized to realize intelligent identification and positioning of the image lodging area of the crop field. And the processing result of the server transmits feedback to the client of the monitoring system, and visual monitoring results such as crop lodging occurrence areas, damage degrees and the like are provided for users.

Specifically, man-machine interaction software of the crop lodging monitoring system is installed in the client device, so that a friendly man-machine interface can be provided for a user. The client can send remote control instructions such as take-off and landing, route modification and the like to the unmanned aerial vehicle, so that the unmanned aerial vehicle task can be accurately controlled. The client may also send control instructions related to the transmission of image data to the ground base station. In addition, the client can receive the processed crop remote sensing images from the server and crop lodging recognition result data obtained based on the images. The client software visually presents the processing result, and clearly displays the processing result to a user in the forms of images, curves, reports and the like, for example, lodging area distribution in the images is defined by using different colors. Through the client, a user can intuitively know the growth condition and lodging occurrence condition of the field crops, evaluate the severity of loss and take prevention and control measures in time.

Unmanned aerial vehicle system of crops remote sensing image automatic acquisition includes following steps: setting information such as a cruising route, a flying height, a flying time, a data acquisition area and the like on a client by a user; the ground base station will receive the instruction and automatically open the cabin door through the internal transmission device. Then, the quadrotor unmanned aerial vehicle can take off from the cabin and go to execute a flight task; in the flight process of the four-rotor unmanned aerial vehicle, the ground base station can conduct real-time information interaction with the four-rotor unmanned aerial vehicle. The airborne infrared spectrum camera adjusts the steering engine angle at a preset coordinate position according to a preset instruction to shoot wheat Tian Yaogan images; after the cruising task is completed, the four-rotor unmanned aerial vehicle automatically returns to the cabin to be automatically charged, so that the four-rotor unmanned aerial vehicle is ready for the next cruising task. Meanwhile, the four-rotor unmanned aerial vehicle transmits the collected remote sensing data to a ground base station in the cabin; the base station transmits the data to the server, and the processing result of the data by the server is fed back to the client platform, so that data support is provided for agricultural production.

Specifically, the unmanned aerial vehicle system workflow for automatically collecting the remote sensing images of crops comprises the following steps: the user plans and sets the cruising route, flying height, flying time, remote sensing data acquisition area range and other key task parameters of the unmanned aerial vehicle through client software. After receiving the flight task instruction sent by the client, the ground base station automatically opens the maintenance cabin door by an unmanned aerial vehicle transmitting device in the ground base station to perform unmanned aerial vehicle transmitting preparation. After confirming the parameters, the user sends a flight task starting instruction to the unmanned aerial vehicle on the ground base station and the load through the client. The transmission device in the ground base station pushes the unmanned aerial vehicle fixed on the transmitting device out of the maintenance cabin door of the ground base station. After the self-checking program of the unmanned aerial vehicle is started, the flight control system automatically unlocks the propeller, and the unmanned aerial vehicle immediately starts to take off according to preset parameters. After the unmanned aerial vehicle comes to the upper air of the target area, the unmanned aerial vehicle starts to automatically cruise according to the set route and altitude, and the airborne multispectral imaging system starts a remote sensing image acquisition task. After the collection is completed, the unmanned aerial vehicle automatically returns to the navigation, and lands in a maintenance cabin of the ground base station. And the ground base station closes the maintenance cabin door to finish one flight task.

Specifically, after the system self-checking and the screw propeller unlocking are completed, the quadrotor unmanned aerial vehicle takes off from the maintenance cabin of the ground base station, flies to a preset target area, and starts to execute a flight task of remote sensing image acquisition. The communication system in the ground base station will be in radio communication with the quad-rotor unmanned aerial vehicle in the aircraft throughout the flight mission. On the one hand, the ground base station can receive flight telemetry data transmitted back by the unmanned aerial vehicle in real time, including parameters such as GPS position, altitude, speed, electric quantity and the like of the unmanned aerial vehicle, and monitor the flight state of the unmanned aerial vehicle. On the other hand, if the flight track or the return instruction needs to be modified, the ground base station can also transmit the instruction in real time to control the flight of the unmanned aerial vehicle. After the unmanned aerial vehicle finishes data acquisition and returns to the voyage, the communication system of the ground base station can be switched to be communicated with the video link of the unmanned aerial vehicle, so that ground operators can monitor the landing process of the unmanned aerial vehicle in real time, and safe and stable landing of the unmanned aerial vehicle is ensured.

Specifically, in the middle of flight, an onboard infrared spectrum camera can adjust the angle of the camera by controlling the rotation of a steering engine when reaching a designated coordinate position according to a preset shooting instruction, and multi-angle shooting is performed on a wheat field below to acquire remote sensing image data. The infrared spectrum imaging can compensate the influence of illumination conditions, and clear wheat Tian Dewu information is ensured to be obtained. After the four-rotor unmanned aerial vehicle smoothly finishes a complete cruising shooting task, the four-rotor unmanned aerial vehicle automatically flies back to a ground base station and accurately lands in a maintenance cabin. At this time, the charging interface of the ground base station can automatically charge the unmanned aerial vehicle battery. After the electric quantity is restored to the set value, the quadrotor unmanned aerial vehicle is ready for the next cruising shooting task.

Specifically, after the quadrotor unmanned aerial vehicle is safely landed in the maintenance cabin of the ground base station, the quadrotor unmanned aerial vehicle can transmit the original remote sensing image data acquired in the flight task to the storage equipment of the ground base station through the data transmission interface. After receiving the image data transmitted by the unmanned aerial vehicle, the ground base station can immediately transmit the data to a server at the rear end through network equipment for storage. The server intelligently analyzes or identifies the growth state, the pest and disease damage condition and the like of crops according to a preset algorithm model based on the original data acquired by the sensor, and generates result data through processing. And finally, the server transmits the processed result data to a client platform of the monitoring system in a feedback manner, and the result data are displayed to a user through a software interface. The method provides data support for subsequent agricultural production management, crop condition judgment, prevention and control strategy formulation and the like.

Data preprocessing, comprising the following steps: preprocessing the collected remote sensing image, wherein the preprocessing comprises image stitching, radiometric calibration and image cutting; carrying out characteristic value and screening on the preprocessed image data, and obtaining parameters such as color characteristics, texture characteristics, vegetation indexes and the like through the Maltese distance calculation separability and random forest characteristic screening; finally, the lodging area is extracted through the improved PSPNet.

Specifically, the data preprocessing module processes the acquired original remote sensing image, and mainly comprises the following steps: and (3) image stitching, namely stitching the images shot by the unmanned aerial vehicle segment by segment, constructing a complete orthographic image, and ensuring the whole coverage of a target area. Adopting a SIFT feature matching algorithm to realize accurate splicing of overlapping areas; and (3) radiometric calibration, namely constructing a radiometric calibration model according to illumination conditions, sensor parameters and the like during shooting, compensating image pixel values and eliminating illumination influence. The reliability of subsequent processing and analysis is ensured. And (3) cutting the image, automatically detecting and cutting the whole image based on longitude and latitude coordinates, and extracting a local sub-image accurately containing the target wheat field. The calculation amount of subsequent processing is reduced, and the efficiency is improved.

Specifically, the preprocessed remote sensing image data enters a feature extraction module to be processed, and the following features are mainly obtained: color features, which adopt pixel statistical features of RGB, HSV and other color spaces, describe wheat Tian Zhengti color information. Texture features, namely extracting texture features such as variance, entropy value, angular second moment and the like, and representing texture information of wheat Tian Debiao. And calculating indexes such as NDVI, RDVI and the like, and reflecting the growth states of wheat straw, leaves and the like. When the above features are extracted, the separability analysis and feature screening are performed, which specifically includes: and calculating the separability of each feature by using the mahalanobis distance, and deleting the redundant related features. And screening and filtering the features by using a random forest algorithm, and selecting the features with obvious target recognition effect. The feature set obtained after screening can be used as a key criterion for subsequent modeling training.

Specifically, on the basis of feature extraction, the system uses an improved PSPNet network model to extract the lodging area of crops. The model extracts multi-scale features through the pyramid pooling module, and global information under different RECEPTIVE FIELD is fully utilized. And attention mechanisms are introduced into the network, so that the key local features can be focused, and the recognition capability of the model on the lodging area is improved. On the basis of training sample labeling and network parameter tuning, the PSPNet model can carry out semantic segmentation on an input crop remote sensing image, the output result is pixel-level classification of an original image, and the lodging occurrence area in the image is clearly marked. By applying the improved PSPNet model, the system realizes automatic and accurate extraction of crop lodging and provides support for subsequent damage assessment and prevention and control measures.

When a certain area is designated as an object of image acquisition, it is difficult to obtain an image of higher resolution because when a scene image of the unmanned aerial vehicle is used to cover all ground objects. And when crops appear lodging disasters, most of crops lodge in a large area, so that images of the multi-view unmanned aerial vehicle are required to be spliced, and high-definition crop image data are obtained. In addition, due to the influence of wind, atmosphere and the like, the GPS information acquired by the unmanned aerial vehicle image is deviated, so that the spliced image must be calibrated through a ground control point. Radiometric calibration is required to obtain relative spectral reflectance, and geometric correction of the acquired image is required to account for distortion of the image acquired by the optical sensor. For images acquired by the track edges of the unmanned aerial vehicle, the phenomena of discontinuity, dislocation and the like can occur after the images are spliced, and the images are needed to be cut, so that a complete research area is left.

Specifically, when a local area is designated as an object for acquiring a remote sensing image, if a single-view image shot by an unmanned aerial vehicle at high altitude is directly used, because the distance is long, it is difficult to obtain an image with higher resolution, and the ground feature details of the area cannot be clearly captured. In addition, when crops are subjected to lodging disasters, the crops often show a distribution form of large-area connection sheets. Therefore, in order to obtain high-quality crop lodging monitoring images, image acquisition is required for a target area in a low-altitude multi-view flight mode. Specifically, the low-altitude flight route of the unmanned aerial vehicle can be planned in advance, and repeated shooting can be carried out on the target area for a plurality of times from different angles and different altitudes. Therefore, partial images with different angles can be obtained, and the images are spliced together by utilizing the characteristic information of the public part through a robust image matching algorithm to construct a high-resolution integral image. The spliced image contains more abundant and complete ground feature details of the target area, and can clearly reflect the lodging conditions of crops at different degrees for subsequent monitoring and identification.

Specifically, after the image of the multi-view unmanned aerial vehicle is obtained, the following further pretreatment is required: based on the image calibration of ground control points, the unmanned aerial vehicle flies and can be influenced by atmospheric conditions such as wind direction, air current and the like, and certain deviation exists in the recorded image GPS information. In order to eliminate the deviation, clear ground control points need to be marked in the image, and geometric correction of the image is realized through two-dimensional conversion by utilizing longitude and latitude coordinates of the ground control points, so that the space accuracy of subsequent analysis is ensured. In order to eliminate the influence caused by different illumination conditions and sensor parameters, a radiometric calibration model needs to be established, and the digital value of an image is converted into the standard spectral reflectivity to be used as the basis for subsequent analysis and interpretation. And (3) correcting geometric distortion, wherein the optical image has geometric distortion such as prism distortion, sleeper distortion and the like, so that distortion coefficient estimation and correction are required to be carried out to ensure the authenticity of the image, and an orthographic image with accurate scale is obtained.

Specifically, when multi-view image stitching is performed, attention is required to the problem of images acquired by the edges of the unmanned aerial vehicle route. The edge images are often discontinuous and misplaced with the adjacent images during splicing, and cannot be closely matched with the whole image. After splicing, necessary cutting is needed to be carried out on the images, the edge image parts shot at the two sides of the route and in the front-back direction are removed, and only the image area with complete and continuous center position is reserved. After cutting, we can obtain a remote sensing image which accurately contains the ground object information in the whole research area. The range of image clipping needs to be determined according to the actual flight trajectory and shooting coverage, so that not only is the inclusion of all research areas ensured, but also the introduction of invalid edge areas is avoided. The cut image provides a reliable data basis for accurately identifying the growth condition of crops in the follow-up process.

Image stitching determines transformation parameters between images by calculating similarity measurement, transforms a plurality of acquired images to the same coordinate system, and achieves the effect of optimal matching on a pixel layer. The gray level-based matching takes image gray level information as a processing object, and the matching is performed through the idea of calculating and optimizing extremum. The gray level-based matching method does not relate to the extraction process of the segmentation value characteristic value of the image, so the method has the characteristics of high precision, strong robustness and the like.

Specifically, the image stitching module needs to integrate a plurality of partial images into one panoramic image. Feature points among images are detected first, and feature point descriptors are built. Similarity measurement between feature point descriptors is calculated, and geometric transformation model parameters between images are determined, including rotation, translation, scaling and the like. All images are converted into a unified target coordinate system based on the obtained conversion parameters. At the pixel level, a weighted fusion method is adopted to realize smooth transition between images. Through the processing, the accurate splicing of a plurality of images can be realized, a seamless panoramic remote sensing image with high resolution can be output, and the subsequent requirement for the fine monitoring of the growth condition of crops is met.

Specifically, when multi-view image stitching is performed, one common matching method is image gray scale-based matching. The multi-view image is converted to a gray scale image and the color information is converted to a single gray scale channel. And extracting image blocks in the overlapping region, calculating gray scale correlation among the image blocks, and constructing a gray scale similarity matrix. And finding a maximum point in the similarity matrix through an optimization algorithm, and determining the best matching image block. And determining a geometric transformation model between the two images according to the matching points, and realizing the alignment between the images. The method has high calculation efficiency by matching the gray texture information of the image without considering complex color change. And the gray scale reflects the structural outline of the scene, so that the geometric relationship between the images is determined more accurately.

Specifically, based on the matching of the image gray information, the method is directly established on the basis of the image gray value, so that error accumulation caused by intermediate steps such as image segmentation and feature extraction is avoided, and the matching accuracy is improved. The gray level of the image is insensitive to illumination change and noise interference, so that the matching result has certain robustness. Only gray level information is calculated, the operation amount is small, and the calculation speed is high. In conclusion, the image splicing method based on gray level matching is high in accuracy and reliable, is suitable for remote sensing image splicing tasks with high accuracy requirements, and can effectively serve for monitoring the growth condition of crops.

The radiation calibration can only perform the comparison in the same scene image by using the pixel value, so as to enhance the distinction between lodging and non-lodging crop images. Scaling is a relatively worthwhile process of transforming measured values of remote sensing data into physical quantities such as absolute brightness, surface reflectivity, or surface temperature. More comparable telemetry data can be provided by radiometric calibration, which can be radiometric and reflectance calibrated by using ENVI5.3 software.

Specifically, the pixel value (DN value) in the original digital image is greatly affected by the shooting condition, and can only be used for contrast analysis inside the same scene image. In order to enhance the spectral reflectance differences of lodged and unbent areas of a crop field photographed at different times and under different lighting conditions, radiometric calibration is required. The purpose of radiation calibration is to convert the DN value to a normalized reflectivity, eliminating the effects due to changes in conditions such as solar radiation angle, sensor gain, etc. Such that there is comparability between images acquired at different times and with other regions of interest. The pixel value of the remote sensing image subjected to radiometric calibration can directly reflect the intrinsic spectral characteristics of the earth surface. This provides the basis for subsequent use of quantitative indicators to enhance the comparative analysis of the lodged and unbent areas of the crop.

In particular, radiometric calibration (Radiometric Calibration) is a process of converting the original digital value (DN value) of a remote sensing image into a quantitative reflectance value or temperature value related to a physical quantity of the earth's surface. The DN value for each band is converted to absolute radiance or irradiance of the scene. The method further converts the image into directional reflectivity or emissivity of the earth surface by using observation geometric conditions, atmospheric parameters and the like during image acquisition. The temperature of the surface component or the brightness temperature of the wave band which can be converted into the standardized wave band can be selected. By scaling, the influence of the change of shooting conditions can be eliminated, so that the quantitative reflection value or the temperature value of the image reflects the spectral characteristics or the thermal properties of the earth surface essence, and a foundation is provided for the subsequent information extraction and analysis.

Specifically, the remote sensing image processing software ENVI5.3 can be utilized to carry out radiometric calibration on an original image obtained by the unmanned aerial vehicle. And collecting auxiliary data required by calibration of unmanned aerial vehicle shooting parameters such as illumination conditions, detector parameters and the like. The original image is imported in ENVI and a "Radiometric Calibration (radiometric scaling)" module is selected. According to the interface prompt, the type (reflectivity or irradiance) of radiation calibration is sequentially selected, and various calibration parameters are input. Clicking "OK" results in a scaled transformed image. The "Convert Reflectance (convert to reflectivity)" module may be further selected to yield a normalized reflectivity result. The software can effectively improve the automation degree of radiometric calibration, output quantitative result images and provide a data basis for the subsequent analysis of the growth conditions of crops.

The data set construction comprises the following steps: image labeling is carried out on the preprocessed data; cutting the marked image data in blocks; to amplify the training samples, the image data is amplified. An extended dataset is obtained by flipping, geometrically transforming, translating, rotating, cropping, adding noise, and color space transforming the image.

Specifically, on the preprocessed remote sensing image, the areas where the crops are lodged and not lodged in the crop fields are manually marked by using the experience knowledge of professionals and are used as samples of different types. And according to the labels, carrying out block cutting on the images, extracting small images containing different categories, and generating an initial sample set. The sample set is amplified through operations such as rotation, overturning, noise adding and the like, so that the sample size is increased, and the model robustness is improved. Through the steps, the crop lodging image data set containing rich positive and negative samples is constructed, and can be used for subsequent model training and verification, so that the intelligent level of crop lodging monitoring is improved.

Specifically, to expand the data set, multiple transformation and amplification are performed on the original image, which specifically includes: and (3) turning: the image is flipped in the horizontal or vertical direction. Geometric transformation: and performing zooming, stretching and other operations. Translation: the image is moved in different directions. And (3) rotation: the image is rotated at different angles. Cutting: a local region of the image is truncated. Adding noise: gaussian noise and the like are added. Color space transformation: converting to HSV, YIQ and other spaces. The number of the amplified image samples can reach several times of that of the original image, and the robustness of the model is improved. Fig. 3 shows a pre-processed manually annotated crop remote sensing image. Wherein the red area is marked as lodging crops, and the green area is normal growing crops.

FIG. 3 is a PSPNet model diagram of an embodiment of the present application: performing convolution operation on an input image to obtain a feature map; the feature map is subjected to four times of pooling operation through pooling cores with different sizes, so that a multi-scale feature map is obtained, wherein the larger the pooling core is, the larger the feature scale of the obtained feature map is; respectively inputting the feature graphs with different scales into a feature integration module; performing convolution operation on the feature map with the minimum scale; and performing splicing operation on the characteristic diagrams of the other three scales.

Specifically, the structural flow of the improved PSPNet network is as follows: the input image is convolved to extract a primary feature map. The primary feature map is subjected to four-dimensional spatial pyramid pooling by using pooling cores (such as 1×1, 2×2,3×3 and 6×6) with different sizes respectively, so as to obtain a multi-scale pooling feature map. And up-sampling the pooled feature maps with different scales to a certain size, splicing the pooled feature maps together, and fusing global information under different RECEPTIVE FIELD. And (3) carrying out channel compression on the spliced characteristic images by using 1X 1 convolution, and then fusing the spliced characteristic images with the characteristic images obtained in the step (1). The fusion features pass through a convolution classifier to realize classification prediction of each pixel.

Specifically, when multi-scale space pooling is performed, the pooling kernel size corresponds to the information scale size contained in the feature map: the larger the pooling kernel, the more global and macroscopic the semantic information contained by the resulting feature map. The smaller the pooling kernel, the more local and detailed the information the feature map contains. Inputting the pooled feature graphs with different scales into a feature integration module for processing: convolving the feature map with the smallest pooling kernel (e.g., 1 x 1) enhances its feature extraction capability for the detail. And interpolating and splicing the feature graphs with other larger kernels and integrating semantic information under different scales. And finally, fusing the scale feature graphs to form feature representation fused with global and local information.

In the feature graphs with three scales, the feature graph with the largest scale needs to be subjected to up-sampling operation before splicing operation, and the feature graph with the smallest scale needs to be subjected to down-sampling operation; performing attention fusion operation on the splicing result; deconvolution is carried out on the attention fusion result, so that the deconvolution result is the same as the feature diagram of the convolution result in size; performing point multiplication operation on the convolution result and the deconvolution result; deconvolution is carried out on the attention fusion result, so that the deconvolution result is the same as the feature diagram of the dot product result in size; performing splicing operation on the point multiplication result and the deconvolution result; and (3) sequentially carrying out convolution and Sigmoid activation function operation on the splicing results to obtain an output image.

Specifically, before feature maps of different scales are spliced, sampling adjustment is required to be performed to the same size: up-sampling the feature map with the largest scale (e.g. 6 x 6 pooling) enlarges its spatial resolution. Downsampling the feature map of smallest scale (e.g., 1 x 1 pooling) reduces its resolution. The intermediate scale feature map (e.g., 2 x2, 3 x 3 pooling) maintains the original resolution. And then splicing the three scale characteristic diagrams after sampling adjustment in the channel dimension. To enhance feature expression, a attention mechanism is further applied to the stitched features to perform weighted fusion of the multi-scale features. And finally, up-sampling the characteristic diagram to the same space size as the original convolution result by adopting a deconvolution operation.

Specifically, the convolution result of the primary feature extraction and the feature map after deconvolution of the feature integration module are subjected to dot product operation, so that feature complementation is realized. Deconvolution is also performed on the feature map obtained by attention fusion, and the feature map is mapped to the same spatial dimension as the dot product result. And splicing the two characteristic diagrams in the channel dimension. And carrying out convolution operation on the spliced feature map to further extract the features. And finally outputting the classification result of each pixel by the convolution result through a Sigmoid activation function. Through the processing, the model realizes the effective fusion of different layers of characteristics, so that the classification decision makes use of global and local information simultaneously.

The feature integration module comprises the following steps: sequentially performing convolution, reLU activation function and batch normalization operation on the input image; and carrying out space-separated cavity convolution on the output result twice in parallel, and then respectively carrying out ReLU and batch normalization operation in sequence. Wherein the spatially separated hole convolution comprises a transverse one-dimensional convolution kernel and a longitudinal one-dimensional convolution. The difference between the two space-separated cavity convolutions is that one transverse one-dimensional convolution is firstly performed and then one longitudinal one-dimensional convolution is performed; and the longitudinal one-dimensional convolution is firstly carried out for the other time, and then the transverse one-dimensional convolution is carried out; performing splicing operation on batch normalization results of the cavity convolution of the two times of space separation; and carrying out convolution, reLU and batch normalization operations on the splicing operation results in sequence.

Specifically, the processing flow of the feature integration module is as follows: the input image is convolved to extract the primary features. The convolution result is subjected to nonlinear conversion of the ReLU activation function. The activation results are subjected to a batch normalization (Batch Normalization) operation. The batch normalized output is input in parallel to two hole convolutions (Atrous Convolution) with hole rates of 6 and 12, respectively. The results generated by the two hole convolutions are respectively subjected to a ReLU activation function and batch normalization. And splicing the two normalized characteristic diagrams on the channel to be used as the final output of the module. The multi-scale feature extraction and fusion provided by the cavity convolution improves the robustness of feature expression and provides information support for subsequent identification.

Specifically, the space separation cavity convolution in the feature integration module is realized by adopting two one-dimensional convolutions with different sequences: the first cavity convolution firstly carries out transverse one-dimensional convolution, namely the convolution kernel size is 1xk, and transverse information is extracted by sliding on an input feature map. Then longitudinal one-dimensional convolution is carried out, the kernel size is kx1, and longitudinal information is extracted. The operation sequence of the second cavity convolution is longitudinal one-dimensional convolution and transverse one-dimensional convolution. The two different-order spatial separation operations can more comprehensively and uniformly extract the longitudinal and transverse information in the feature map, and the feature expression capacity is improved. And the same cavity rate is used for the two operations, so that the consistency of the extracted spatial scale can be ensured, and the subsequent feature fusion is facilitated.

Specifically, feature graphs obtained by batch normalization after the two times of space separation and cavity convolution are spliced in the channel dimension. And carrying out convolution operation, reLU activation function and batch normalization on the spliced characteristic graphs successively. And convolution extracts new expression of splicing characteristics, and ReLU introduces nonlinearity, so that the numerical stability is improved by batch normalization. Through the convolution-activation-normalization processing, two groups of cavity features are extracted and fused. And finally, outputting the convolved batch normalization result as a final characteristic representation of the module.

In order to evaluate the quality of the model, 2 evaluation indexes, namely, pixel Accuracy (PA) and average cross ratio (Mean Intersection over Union, MIoU) are adopted in the embodiment. The pixel accuracy can intuitively reflect the accuracy of the model in extracting the lodging area, and the average intersection ratio intuitively reflects the accuracy of the model in segmentation. The area correctly divided into lodging crops is marked as true positive rate (True positives, TP); the area correctly divided into non-lodging crops is marked as true negative (True negatives, TN); the area that is misclassified as lodging crop is scored as false positive (False positives, FP); the area misclassified as non-lodging crop was noted as false negative (FALSE NEGATIVES, FN).

Specifically, to evaluate the effect of the crop lodging recognition model, two evaluation indexes are adopted: pixel Accuracy (PA), PA denotes the proportion of the number of pixels that the model correctly predicts as positive samples to all of the positive samples predicted. The accuracy of the model in extracting the lodging area of the crops is intuitively reflected. Average intersection ratio (Mean Intersection over Union, mIoU), mIoU calculates the average of intersection and union ratios of each category prediction region and truth region. It embodies the accurate segmentation capability of the model for different categories. The two indexes are integrated, so that the recognition and positioning effect of the model on the lodging area of the crops can be comprehensively evaluated. Higher PA values indicate more accurate extraction, higher mIoU values represent finer segmentation. The method is used as an important judgment basis for evaluating the performance of the model.

Specifically, when calculating the pixel accuracy and average intersection ratio, comparing the prediction type of each pixel with the real type by the model, the method can be divided into the following four cases: true positive (True Positives, TP): the model correctly predicts the pixels as the number of samples of the lodged crop. True negative (True Negatives, TN): the model correctly predicts the number of samples for which the pixel is a non-lodging crop. False positives (False Positives, FP): the model mispredicted pixels are the number of samples of lodged crop. False negative (FALSE NEGATIVES, FN): the model mispredicted pixels are the number of samples of the non-lodging crop. According to the four conditions, an evaluation index can be calculated, and the recognition and segmentation effects of the model can be evaluated from different angles.

Fig. 4 is a diagram of a wheat lodging area recognition result according to an embodiment of the present application, where the unmanned aerial vehicle image has the characteristics of large scale and high resolution, so that the data scale is very large, which is undoubtedly a huge consumption of storage space and calculation power. Therefore, the embodiment disassembles the original RGB image on the color channel, disassembles the original 3-channel data into single-channel data according to red, green and blue, reduces the data volume to one third of the original data volume, and inputs the single-channel data into the network for testing. The accuracy rate is reduced by 0.53% on average and the cross ratio is reduced by 0.74% on average. It can be seen that the model still has excellent performance under the condition of remarkably reducing the data quantity.

In particular, the remote sensing image of the crop field obtained by the unmanned aerial vehicle has the characteristics of large scale and high resolution, which makes the data volume very huge. Large-scale data puts extremely high demands on memory space and computational effort in image processing. In order to reduce data redundancy and improve processing efficiency, the scheme adopts the following measures: the original RGB image is subjected to channel decomposition, and independent red, green and blue channels are extracted, so that the data volume can be reduced by 2/3. Processing and analysis are carried out on the single-channel image, so that the storage and calculation amount is greatly reduced. And the processing results of all channels are fused, so that the extraction accuracy is ensured. And a more efficient model structure design, such as a module for saving the calculation amount, such as cavity convolution, is used. And the calculation power level is improved by using distributed processing means such as cloud computing. Through data volume control and calculation power extension, the intelligent analysis of large-scale unmanned aerial vehicle image can be supported to this scheme, realizes the monitoring of efficient crops lodging.

Specifically, the original RGB image is extracted into three color channels of red, green and blue, and the data volume of each channel is 1/3 of the original image. And respectively inputting the single-channel image data of the red, green and blue channels into a segmentation network for testing. And fusing network test results of each channel to obtain a final lodging area segmentation result of the crop field. The channel decomposition test reduces the data size by 2/3 compared to using 3-channel RGB images directly. The storage occupation and the calculated amount are greatly reduced while the accuracy of the result is ensured. The single-channel network is more suitable for large-scale remote sensing image processing. Through channel decomposition, data volume control in unmanned aerial vehicle image processing is realized, storage and calculation burden are lightened, and processing efficiency is improved.

Specifically, compared with the original 3-channel RGB image, the single-channel input is adopted for testing, and the results of all evaluation indexes are as follows: the pixel accuracy is reduced by 0.53% on average. The average cross-over is reduced by 0.74% compared with the average. It can be seen from the combination of the changes of the two indexes that the segmentation effect of the model is only slightly reduced under the condition that the data quantity is reduced by 3 times. This shows that the model can still maintain excellent recognition and segmentation performance on the premise of greatly reducing data redundancy. The channel decomposition reduces the storage and calculation requirements, is more suitable for large-scale remote sensing image processing, and has good engineering application value.

The foregoing has been described schematically the application and embodiments thereof, which are not limiting, but are capable of other specific forms of implementing the application without departing from its spirit or essential characteristics. The drawings are also intended to depict only one embodiment of the application, and therefore the actual construction is not intended to limit the claims, any reference number in the claims not being intended to limit the claims. Therefore, if one of ordinary skill in the art is informed by this disclosure, a structural manner and an embodiment similar to the technical scheme are not creatively designed without departing from the gist of the present application, and all the structural manners and the embodiment are considered to be within the protection scope of the present patent. In addition, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" preceding an element does not exclude the inclusion of a plurality of such elements. The various elements recited in the product claims may also be embodied in software or hardware. The terms first, second, etc. are used to denote a name, but not any particular order.

Claims

1. A lodging area identification method based on crop remote sensing images comprises the following steps:

Collecting remote sensing images of crops;

performing image stitching, radiometric calibration and image cutting pretreatment on the acquired remote sensing images;

Extracting features of the processed remote sensing image to obtain color features, texture features and vegetation index features;

and identifying crop lodging areas in the images based on a semantic segmentation algorithm of the convolutional neural network.

2. The crop remote sensing image-based lodging area identification method according to claim 1, wherein:

The convolutional neural network adopts PSPNet network;

The encoder adopts ResNet of pre-training as the encoder to extract the characteristics of the input image;

the fusion module comprises convolution layers of pooling cores with different sizes, and performs multi-scale pooling on the characteristics extracted by the encoder to obtain characteristic diagrams with different scales;

The decoder comprises an n-level up-sampling layer and an n-level convolution layer, wherein the i-level up-sampling layer amplifies the i-level feature image to the size of an i+1-level feature image, and the i-level convolution layer fuses the feature image output by the i-level up-sampling layer with the feature image output by the i-level up-sampling layer.

3. The crop remote sensing image-based lodging area identification method according to claim 2, wherein:

The method comprises the following steps:

extracting features of an input image through a convolution layer to serve as a first feature map;

Performing multi-scale pooling on the first feature map by using pooling check of different sizes to obtain pooling feature maps of different scales as a second feature map;

Feature fusion is carried out on the second feature graphs with different scales, convolution processing is carried out on the feature graphs with the minimum scale, up-sampling or down-sampling and splicing processing are carried out on the feature graphs except the feature graphs with the minimum scale, and a spliced feature graph is obtained;

fusing the spliced feature images by adopting an attention mechanism to obtain a fused feature image;

Up-sampling the fusion feature map through deconvolution to obtain an up-sampling feature map;

and performing dot multiplication, splicing and convolution operation on the up-sampling feature map and the minimum scale feature map, and outputting an image containing a lodging area prediction result.

4. The crop remote sensing image-based lodging area identification method according to claim 3, wherein:

And carrying out feature fusion on the second feature graphs with different scales, wherein the feature fusion comprises the following steps of:

Taking the second feature images with different scales as input, and inputting the second feature images into a fusion module;

performing first feature extraction on the second feature map through the first convolution layer, and extracting feature maps with different scales to serve as a first feature map;

Carrying out bidirectional space separation convolution on the first feature map by adopting a second convolution layer, and extracting feature maps with different scales to serve as a second feature map, wherein the bidirectional space separation convolution comprises transverse one-dimensional convolution and longitudinal one-dimensional convolution;

Splicing the second feature images extracted through the bidirectional space separation convolution to obtain spliced feature images;

and carrying out convolution operation on the spliced feature images by adopting a third convolution layer, and outputting the feature images after fusion.

5. The crop remote sensing image-based lodging area identification method according to claim 4, wherein:

When the transverse one-dimensional convolution is carried out, setting the length of the convolution kernel to be larger than the width of the convolution kernel, and extracting transverse features of the image;

when longitudinal one-dimensional convolution is performed, the width of the convolution kernel is set to be larger than the length of the convolution kernel, and the convolution kernel is used for extracting longitudinal features of an image.

6. The crop remote sensing image-based lodging area identification method according to claim 1, wherein:

Remote sensing image acquisition is carried out on a crop field through an unmanned aerial vehicle, and the method comprises the following steps of:

The unmanned aerial vehicle collects remote sensing images of the crop fields through the multispectral camera;

The multispectral camera performs RGB three-channel splitting on the acquired remote sensing image to respectively acquire three single-channel images of red, green and blue;

and carrying out channel-division storage on the acquired red, green and blue three single-channel images so as to reduce the storage capacity.

7. The crop remote sensing image-based lodging area identification method of claim 6, wherein the method comprises the steps of:

image stitching is carried out on the collected remote sensing images, and the method comprises the following steps:

The acquired single-channel images are arranged and combined according to time sequence;

Carrying out image fusion processing on the part of the adjacent images with an overlapping area, wherein the overlapping area represents a commonly covered ground surface area generated by shooting at the same geographic position;

And (5) splicing to generate an integral remote sensing image covering the target crop field.

8. The crop remote sensing image-based lodging area identification method as claimed in claim 7, wherein:

The method for performing radiation calibration on the acquired remote sensing image comprises the following steps of:

Collecting illumination parameters when the unmanned aerial vehicle acquires a remote sensing image;

constructing an image radiometric calibration model by utilizing the collected illumination parameters;

And correcting the illumination information of each pixel in the spliced integral remote sensing image through the constructed image radiation calibration model so as to eliminate the influence of different illumination conditions.

9. The crop remote sensing image-based lodging area identification method of claim 8, wherein:

cutting the collected remote sensing image, comprising the following steps:

using the integral remote sensing image after radiometric calibration as an input image;

Acquiring a field area and a background area of a target crop in an input image through an image segmentation algorithm;

calculating the boundary of the target crop field area through a boundary detection algorithm, and obtaining the polygon area coordinates of the target crop field area;

And cutting and storing the input image according to the acquired polygonal region coordinates to serve as a target region.

10. A lodging area recognition system based on crop remote sensing images for performing the lodging area recognition method based on crop remote sensing images according to any one of claims 1 to 9, comprising:

The image acquisition module acquires remote sensing images of the crop fields;

the preprocessing module is used for splicing, radiating and calibrating the collected remote sensing images and cutting the images;

The feature extraction module is used for extracting features of the preprocessed remote sensing image to obtain color features, texture features and vegetation index features respectively;

The recognition module is used for carrying out semantic segmentation on the remote sensing image after feature extraction based on a pre-trained convolutional neural network model and recognizing a crop lodging area in the remote sensing image, wherein the convolutional neural network model adopts a PSPNet structure;

PSPNet the structure comprises:

An encoder for feature extraction;

The fusion module is used for multi-scale feature fusion;

And a decoder for pixel prediction and up-sampling to the original resolution.