CN112906648A

CN112906648A - Method and device for classifying objects in land parcel and electronic equipment

Info

Publication number: CN112906648A
Application number: CN202110314424.5A
Authority: CN
Inventors: 李世行; 吴海山; 殷磊
Original assignee: WeBank Co Ltd
Current assignee: WeBank Co Ltd
Priority date: 2021-03-24
Filing date: 2021-03-24
Publication date: 2021-06-04

Abstract

The application provides a method, a device, an electronic device, a computer readable storage medium and a computer program product for classifying objects in a parcel; the method comprises the following steps: acquiring a region image; determining an interested area in the region image according to pre-labeling information, and cutting out an image of the interested area from the region image to be used as an image of the land parcel; cutting the image of the land parcel by a preset size based on a plurality of sampling points in the land parcel to obtain a plurality of sub-block images; and acquiring a classification result of the object in the corresponding sub-block image based on the image characteristics of each sub-block image, and determining the object category covered by the parcel based on the classification result of the object in each sub-block image. Through the method and the device, the classification of the objects in the plot can be accurately and quickly carried out, and the object types covered by the plot are obtained.

Description

Method and device for classifying objects in land parcel and electronic equipment

Technical Field

The present application relates to image processing technologies, and in particular, to a method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product for classifying objects in a parcel.

Background

The method is one of key research directions in the aspect of land parcel research, and is based on the technology of satellite remote sensing or unmanned aerial vehicle aerial photography to shoot ground images and perform image processing and analysis.

The image classification processing is an image processing method for distinguishing different types of targets according to different characteristics reflected in image information, and is characterized by utilizing a computer to carry out quantitative analysis on an image, classifying each pixel or area in the image into one of a plurality of categories to replace the visual interpretation of a human.

For the classification processing of objects in a plot, the mode provided by the prior art is mainly a pixel-level method based on rules or machine learning, the traditional characteristic extraction method is low in efficiency and low in precision, a large amount of manual data marks are needed when the deep learning method is used for classifying the objects in the remote sensing image, the cost is high, full-image scanning sampling and classification prediction are needed, and the accuracy is low.

Disclosure of Invention

The embodiment of the application provides a method and a device for classifying objects in a plot, an electronic device, a computer-readable storage medium and a computer program product, which can accurately and quickly classify the objects in the plot to obtain object classes covered by the plot.

The technical scheme of the embodiment of the application is realized as follows:

the embodiment of the application provides a method for classifying objects in a land parcel, which comprises the following steps:

acquiring a region image;

determining an interested area in the region image according to pre-labeling information, and cutting out an image of the interested area from the region image to be used as an image of the land parcel;

cutting the image of the land parcel by a preset size based on a plurality of sampling points in the land parcel to obtain a plurality of sub-block images;

and acquiring a classification result of the object in the corresponding sub-block image based on the image characteristics of each sub-block image, and determining the object category covered by the parcel based on the classification result of the object in each sub-block image.

The embodiment of the application provides a classification device of objects in plots, includes:

the image acquisition module is used for acquiring a region image;

the image cutting module is used for determining an interested area in the region image according to the pre-labeling information and cutting out the image of the interested area from the region image to be used as the image of the land parcel;

the image cutting module is used for cutting the image of the land parcel through a preset size based on a plurality of sampling points in the land parcel to obtain a plurality of sub-block images;

and the land parcel type determining module is used for acquiring a classification result of the corresponding object in the sub-block images based on the image characteristics of the sub-block images and determining the object type covered by the land parcel based on the classification result of the object in the sub-block images.

In the above solution, the apparatus for classifying objects in a parcel further includes: the information acquisition module is used for responding to the marking operation and acquiring manually marked pre-marking information; or acquiring pre-labeled information labeled based on the historical vector.

In the foregoing solution, the image cutting module is further configured to perform the following processing for each land parcel: randomly acquiring a plurality of sampling points in the plot; based on the plurality of sampling points, cutting the image of the land block by a preset size respectively to obtain a plurality of sub-block images corresponding to the land block; the preset size is a regular geometric figure taking the sampling point as a center.

In the foregoing solution, the block type determining module is further configured to perform the following processing for each sub-block image: mapping image features extracted from the sub-block images to probabilities corresponding to a plurality of candidate object classes; and taking the object class corresponding to the probability exceeding the probability threshold as the classification result of the sub-block images.

In the foregoing solution, the block type determining module is further configured to, when the probabilities corresponding to at least two candidate object types exceed the probability threshold, use the object types corresponding to the at least two probabilities exceeding the probability threshold as the classification result of the sub-block image, or use the object type corresponding to the maximum probability as the classification result of the sub-block image.

In the above solution, the obtaining of the classification result of the object in the corresponding sub-block image based on the image feature of each sub-block image is implemented by at least one classification model; the land parcel type determination module is further used for executing the following processing through the same classification model: classifying objects in the corresponding sub-block images respectively based on image features extracted from each sub-block image to obtain classification results of the objects in the sub-block images; alternatively, the following is performed by a plurality of classification models: and classifying the objects in the sub-block images one by one based on the image characteristics of the sub-block images to obtain the classification result of the objects in the sub-block images.

In the above scheme, the block type determining module is further configured to determine and count the number of sub-block images belonging to the same object type in each sub-block image; and determining the object type corresponding to the subblock image with the largest number as the object type covered by the parcel.

In the above solution, the obtaining of the classification result of the object in the corresponding sub-block image based on the image feature of each sub-block image is implemented by at least one classification model, and the apparatus for classifying the object in the parcel further includes: a classification model training module for training the classification model by: initializing model parameters of a classification model; performing class prediction on the sub-block image samples in the training sample set through the classification model to obtain prediction object classes of the sub-block image samples; and determining errors of the pre-marked object class and the predicted object class of the sub-block image samples, and performing back propagation in the classification model based on the errors so as to update the model parameters of the classification model.

In the above scheme, when the obtaining of the classification result of the object in the corresponding sub-block image based on the image features of each sub-block image is implemented by a plurality of classification models, the classification model training module is further configured to extract a training sample subset from the training sample set, where the training sample subsets of each classification model are different or not identical.

In the above scheme, the classification model training module is further configured to determine, based on the plurality of pre-labeled coordinate points, a parcel where each pre-labeled coordinate point is located in the region image sample; cutting an image of the land where each pre-marked coordinate point is located from the region image sample according to a preset size to obtain a plurality of sub-block image samples; and constructing the training sample set based on the plurality of sub-block image samples and the corresponding labeled object classes.

An embodiment of the present application provides an electronic device, including:

a memory for storing executable instructions;

and the processor is used for realizing the classification method of the objects in the parcel provided by the embodiment of the application when the executable instructions stored in the memory are executed.

The embodiment of the application provides a computer-readable storage medium, which stores executable instructions and is used for realizing the classification method of the objects in the parcel provided by the embodiment of the application when being executed by a processor.

The embodiment of the application provides a computer program product, which comprises a computer program, and the computer program is used for realizing the classification method of the objects in the land parcel provided by the embodiment of the application when being executed by a processor.

The embodiment of the application has the following beneficial effects:

the method comprises the steps of cutting the image of the land block in a preset size to obtain a plurality of effective sub-block image samples corresponding to the land block, improving the precision of the samples, predicting the classification of the sub-block images to obtain a plurality of corresponding classification results, determining the object classification covered by the land block according to the predicted classification results, improving the accuracy of land block identification, reducing the processing difficulty compared with a processing mode of dividing and classifying the image pixel by pixel, and improving the identification precision.

Drawings

FIG. 1 is an alternative schematic structural diagram of a system for classifying objects in a parcel as provided by an embodiment of the present application;

fig. 2 is an alternative structural schematic diagram of an electronic device provided in an embodiment of the present application;

FIG. 3A is an alternative flow chart of a method for classifying objects in a parcel as provided in an embodiment of the present application;

FIG. 3B is an alternative flow chart of a method for classifying objects in a parcel as provided in the embodiments of the present application;

FIG. 4 is a schematic flow chart illustrating an alternative method for training a classification model according to an embodiment of the present disclosure;

FIG. 5 is a schematic flow chart illustrating an alternative method for training a classification model according to an embodiment of the present disclosure;

fig. 6A is an alternative schematic diagram of a region image provided in the embodiment of the present application;

FIG. 6B is an alternative schematic diagram of a cut land image provided by an embodiment of the present application;

FIG. 7 is an alternative schematic diagram of a sampling window provided by embodiments of the present application;

FIG. 8A is an alternative diagram of a classification process based on a classification model according to an embodiment of the present application;

FIG. 8B is an alternative diagram of a classification process based on multiple classification models provided by an embodiment of the present application;

FIG. 9 is a flowchart illustrating a method for training a classification model according to an embodiment of the present disclosure;

fig. 10 is a flowchart illustrating a method for classifying objects in a parcel according to an embodiment of the present application.

Detailed Description

In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be described in further detail with reference to the attached drawings, the described embodiments should not be considered as limiting the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

Where similar language of "first/second" appears in the specification, the following description is added, and where reference is made to the term "first \ second \ third" merely for distinguishing between similar items and not for indicating a particular ordering of items, it is to be understood that "first \ second \ third" may be interchanged both in particular order or sequence as appropriate, so that embodiments of the application described herein may be practiced in other than the order illustrated or described herein.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the application.

The current remote sensing crop recognition model is limited by sufficient training samples, semantic segmentation is not easy to directly carry out, and feature extraction is simple and low in precision; when the deep learning model is used for recognizing the objects in the plot, the real data needs to be acquired manually on the spot, the fixed point is arbitrary, the cost is high, sufficient data amount does not exist, the training sample image acquired by directly windowing is seriously fragmented, and the recognition accuracy is low.

Based on this, the embodiments of the present application provide a method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product for classifying objects in a parcel, which can improve the precision and accuracy of parcel identification and classification of objects in a parcel.

First, a description is given of a classification system for objects in a parcel provided in an embodiment of the present application, referring to fig. 1, fig. 1 is an optional architecture schematic diagram of the classification system 100 for objects in a parcel provided in an embodiment of the present application, a terminal 400 is connected to a server 200 through a network 300, the network 300 may be a wide area network or a local area network, or a combination of the two, and data transmission is achieved using a wireless link. In some embodiments, the terminal 400 may be, but is not limited to, a laptop, a tablet, a desktop smart phone, a dedicated messaging device, a portable gaming device, a smart speaker, a smart watch, and the like. The server 200 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, middleware service, a domain name service, a security service, a CDN, a big data and artificial intelligence platform, and the like. The network 300 may be a wide area network or a local area network, or a combination of both. The terminal 400 and the server 200 may be directly or indirectly connected through wired or wireless communication, and the embodiment of the present application is not limited thereto.

And the terminal 400 is configured to acquire the region remote sensing image and send the region remote sensing image to the server 200.

The server 200 is used for receiving the region remote sensing image, determining an interested region in the region image according to pre-labeling information, and cutting out an image of the interested region from the region image to be used as an image of the land; cutting the image of the land parcel by a preset size based on a plurality of sampling points in the land parcel to obtain a plurality of sub-block images; and obtaining a classification result of the corresponding object in the sub-block image based on the image characteristics of each sub-block image, determining the object type covered by the land block based on the classification result of the object in each sub-block image, obtaining a land block classification map of the remote sensing image, and returning the land block classification map to the terminal 400.

In other examples, the terminal 400 may obtain a region remote sensing image, implement the classification method for the objects in the region provided by the present application based on the region remote sensing image, obtain a region classification map of the remote sensing image, and display the region classification map on the terminal interface.

Referring to fig. 2, fig. 2 is an optional schematic structural diagram of an electronic device 500 provided in the embodiment of the present application, in practical applications, the electronic device 500 may be implemented as the terminal 400 or the server 200 in fig. 1, and the electronic device implementing the method for classifying objects in a parcel in the embodiment of the present application is described by taking the electronic device as the server 200 shown in fig. 1 as an example. The electronic device 500 shown in fig. 2 includes: at least one processor 510, memory 550, at least one network interface 520, and a user interface 530. The various components in the electronic device 500 are coupled together by a bus system 540. It will be appreciated that the bus system 540 is used to enable communications among the components. The bus system 540 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 540 in fig. 2.

The Processor 510 may be an integrated circuit chip having Signal processing capabilities, such as a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like, wherein the general purpose Processor may be a microprocessor or any conventional Processor or the like.

The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, and the like. Memory 550 includes one or more storage devices physically located remote from processor 510.

The memory 550 may comprise volatile memory or nonvolatile memory, and may also comprise both volatile and nonvolatile memory. The nonvolatile Memory may be a Read Only Memory (ROM), and the volatile Memory may be a Random Access Memory (RAM). The memory 550 described in embodiments herein is intended to comprise any suitable type of memory.

In some embodiments, memory 550 can store data to support various operations, examples of which include programs, modules, and data structures, or subsets or supersets thereof, as exemplified below.

An operating system 551 including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

a network communication module 552 for communicating to other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including: bluetooth, wireless compatibility authentication (WiFi), and Universal Serial Bus (USB), etc.;

in some embodiments, the classification device for objects in a parcel provided by the embodiments of the present application may be implemented in software, and fig. 2 shows a classification device 555 for objects in a parcel stored in a memory 550, which may be software in the form of programs and plug-ins, and includes the following software modules: the image acquisition module 5551, the image cropping module 5552, the image cutting module 5553, and the parcel type determination module 5554 are logical, and thus may be arbitrarily combined or further divided according to the functions implemented. The functions of the respective modules will be explained below.

In other embodiments, the classification Device of the object in the parcel provided by the embodiments of the present Application may be implemented in hardware, and as an example, the classification Device of the object in the parcel provided by the embodiments of the present Application may be a processor in the form of a hardware decoding processor, which is programmed to execute the classification method of the object in the parcel provided by the embodiments of the present Application, for example, the processor in the form of the hardware decoding processor may employ one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), or other electronic components.

Based on the above description of the classification system and the electronic device for objects in a parcel in the embodiment of the present application, a description is next given of a classification method for objects in a parcel provided in the embodiment of the present application. In some embodiments, the classification method for the objects in the parcel provided by the embodiments of the present application may be implemented by a server alone, or implemented by a server and a terminal in cooperation.

The method for classifying objects in a parcel provided by the embodiment of the present application will be described in conjunction with an exemplary application and implementation of the parcel identification system provided by the embodiment of the present application.

Referring to fig. 3A, fig. 3A is an alternative flowchart of a method for classifying objects in a parcel according to an embodiment of the present application, which will be described with reference to the steps shown in fig. 3A. The steps shown in fig. 3A may be performed by a server (e.g., server 200 shown in fig. 1), and will not be described again below.

In step 101, a region image is acquired.

Here, the regional image may be acquired by any means, such as satellite remote sensing, unmanned aerial vehicle aerial photography. In some embodiments, the regional remote sensing image can be collected by means of unmanned aerial vehicle fixed-point aerial photography, satellite shooting and other ways facing county-level, city-level, province-level or even national agricultural areas.

In step 102, an area of interest in the region image is determined according to the pre-labeling information, and an image of the area of interest is cut out from the region image to serve as an image of the land parcel.

In some embodiments, before step 102, it may further be performed: responding to the marking operation, and acquiring manually marked pre-marking information; or acquiring pre-labeled information labeled based on the historical vector. Here, the pre-labeling information is the land parcel cutting information labeled for the region image. The vector label can be label information of vector data drawn by the remote sensing image based on points, lines, surfaces and the like, and the historical vector label is historical label information in the remote sensing image.

In some embodiments, referring to fig. 6A, fig. 6A is an optional schematic diagram of a region image provided in the embodiments of the present application. In actual implementation, the region of interest is determined by making the land cutting information of the artificial labeling and the history vector labeling of the fine tuning land, and the land of the region of interest is cut in the remote sensing image.

For example, in the manual labeling, an open-source labeling software is used to make a land edge segmentation label, such as labelme, ArcGIS, and other labeling software, and the remote sensing image is segmented by using the label to obtain a remote sensing image including at least one land. Here, in order to distinguish different land parcels which are close to each other, the boundary is intentionally left blank when the artificial marking is performed, or morphological erosion operation is performed on each marked land parcel.

It should be noted that the remote sensing image includes the region of interest and some other regions. Illustratively, the region of interest may be a plant region where plants, crops, etc. are grown, and the region of non-interest may include a small number of other regions such as a destination office area, a road area, and a building area. The objects in the plot image may be plants, crops, etc. planted in the plot.

Referring to fig. 6B, fig. 6B is an alternative schematic diagram of a land image after land cutting provided by an embodiment of the present application, where a white area 62 is an interested area (i.e., an area that needs to be subjected to category identification), and a black area 61 is another area.

In some embodiments, the cut image shown in fig. 6B and the region image shown in fig. 6A are subjected to a mask process to obtain an image of the corresponding region.

In practical implementation, region image acquisition and classification processing for counties, towns and towns are performed, and as the plots are basically not changed or are migrated at plot levels in a long time, plot segmentation processing can be performed through historical vector labeling, and the historical vector labeling can be updated at regular time and loaded for use again after fine adjustment.

Above-mentioned mode through artifical mark or historical vector mark cutting landmass cuts apart a plurality of landmasses fast to in order to discern and classify to every landmass, reduce the complexity of carrying out pixel-by-pixel discernment to the region image, reduce the excessive fragmentation of image that brings of random sampling, distribute the not concentrated problem, promote the categorised precision of landmass.

In step 103, the image of the land parcel is cut by a preset size based on a plurality of sampling points in the land parcel to obtain a plurality of sub-block images.

In some embodiments, step 103 may be implemented by: the following processing is performed for each plot: randomly acquiring a plurality of sampling points in a land parcel; based on the plurality of sampling points, cutting the image of the land block by preset sizes respectively to obtain a plurality of sub-block images corresponding to the land block; the preset size is a regular geometric figure taking the sampling point as the center.

In some embodiments, the number of random points may be determined by a preset rule; for example, the number of random dots is determined to be five ten-thousandth of the total pixels of the tile image, and if the size of the image at this time is 100 × 100, the number of random dots may be determined to be 5.

In actual implementation, a sampling window is constructed in a preset size, and as the plot generally becomes a block and a cube, if the sampling window is too large, other types of information or noise can be introduced, so that the identification precision is reduced; if the sampling window is too small, the information such as the boundary of the land parcel and the like cannot be sufficiently contained, so the setting of the size of the sampling window is usually determined on real regional remote sensing image data through a large number of experimental tests, and generally, a preset size which can wrap the land parcel as much as possible is selected. For example, the preset size may be obtained by performing area pixel clustering processing on the remote sensing image to obtain a mode, or may be set according to image parameters such as spatial resolution of the remote sensing image, which is not limited in the embodiment of the present application.

Illustratively, referring to fig. 7, fig. 7 is an alternative schematic diagram of a sampling window provided in an embodiment of the present application. And generating a sampling window with a fixed size aiming at the sampling point in each land block, and cutting the land block image with the fixed size (namely the preset size) to obtain a sub-block image corresponding to the land block image. The predetermined size is generally a regular geometric figure centered on the sampling point, for example, the regular geometric figure may be a circle, a rectangle, a square, etc.

Here, the fixed size may be represented by a parameter size, and for example, the fixed size may be a radius value of a circle, a length and width value of a rectangle, or a side length value of a square, or the like. Referring to fig. 7, the sampling window 71 shown in fig. 7 is a square sampling window 71 constructed by taking a sampling point as a center, shifting the size/2 at the upper left and shifting the size/2 at the lower right, and based on each sampling point obtained randomly, the sampling window 71 with a preset size is used to cut the land block image, so as to obtain a plurality of sub block images with the size consistent with the size of the sampling window.

In other embodiments, step 103 may also be implemented by: in each land block, a central point of each land block is obtained through a geometric image processing method, and the images of the land blocks are cut by preset sizes by taking the central point as a center, so that a plurality of sub-block images corresponding to the land blocks are obtained.

For example, the image of the land block is cut from different directions with the sampling point as the center and the preset size as the size of the sampling window, and a plurality of sub-block images corresponding to the land block are obtained.

The image is cut according to the preset size, the sampling processing of the block-level image can be adapted, and compared with full-image random sampling and prediction, the accuracy of image identification is improved.

In step 104, a classification result of the object in each sub-block image is obtained based on the image features of the sub-block images.

In some embodiments, the method of step 104 may be implemented by at least one classification model, where the classification model may be a deep neural network model including a plurality of convolutional layers, pooling layers, and output layers, e.g., the classification model may be a residual network model ResNet, a dense connection network model densnet, etc.

In some embodiments, referring to fig. 8A, fig. 8A is an optional schematic diagram of a classification process based on a classification model provided in the embodiments of the present application. The method of step 104 may perform the following by the same classification model: and respectively classifying the objects in the corresponding sub-block images based on the image characteristics extracted from each sub-block image to obtain the classification result of the objects in the sub-block images.

In practical implementation, the same classification model is used for respectively carrying out feature extraction processing on the sub-block image 1 and the sub-block images 2 to the sub-block image n, each sub-block image is classified based on the image features extracted from each sub-block image, and the classification result of the sub-block image 1, the classification result of the sub-block image 2 and the classification result of the sub-block image n are respectively obtained.

In other embodiments, referring to fig. 8B, fig. 8B is an optional schematic diagram of a classification process performed based on a plurality of classification models provided in an embodiment of the present application, where each classification model may be a structure of the classification model shown in fig. 8A. Wherein each classification model corresponds to one sub-block image. The method of step 103 may perform the following by a plurality of classification models: and classifying the objects in the sub-block images one by one based on the image characteristics of the sub-block images to obtain the classification result of the objects in the sub-block images.

In practical implementation, the classification model 1 is used for carrying out feature extraction processing on the sub-block image 1, and the sub-block image 1 is classified based on the image features extracted from the sub-block image 1 to obtain a classification result of the sub-block image 1; performing feature extraction processing on the sub-block images 2 by using the classification model 2, and classifying the sub-block images 2 based on the image features extracted from the sub-block images 2 to obtain classification results of the sub-block images 2; and sequentially utilizing n classification models to classify the n image sub-blocks respectively to obtain corresponding n classification results.

Here, the plurality of classification models are trained by a plurality of training sample subsets, wherein the training sample subsets of each classification model are different or not identical.

The sub-block images are classified one by one through the classification models to obtain the corresponding prediction categories, so that the error identification caused by the deviation of the sub-block image identification through one classification model can be avoided, and the stability of the land block identification is ensured.

Referring to fig. 3B, fig. 3B is a flowchart illustrating a method for classifying objects in a parcel according to an embodiment of the present application, the method in step 104 may be implemented by step 1041-1043, which is described below with reference to step 1041-1043 illustrated in fig. 3B.

In step 1041, feature extraction is performed on each sub-block image to obtain an image feature corresponding to each sub-block image.

Before feature extraction is carried out on the subblock image, operations such as radiation correction, geometric correction, filtering processing and the like are carried out on the subblock image to obtain a preprocessed subblock image. Inputting the preprocessed subblock images into a convolution layer of a classification model for feature extraction to obtain a plurality of feature maps, inputting the corresponding feature maps into a pooling layer for down-sampling processing, reserving stronger image features, and obtaining image features corresponding to the subblock images after multiple times of convolution and pooling.

It should be noted that the image feature is used to indicate the object class of the image sub-block, the classification of the sub-block image may be two or more, the number of candidate object classes is two for the two classification, and the number of candidate object classes is more for the more classification.

In step 1042, mapping the image features extracted from each sub-block image to probabilities corresponding to a plurality of candidate object classes;

in some embodiments, the sub-block images are class predicted by an output layer of the classification model, where the output layer may be a softmax layer. The output layer provided by the embodiment of the application is provided with a plurality of softmax mapping functions, and the image features extracted from the sub-block images are subjected to mapping and normalization processing to obtain the probability corresponding to a plurality of candidate object categories.

Illustratively, the object class of the sub-block image characterizes the class of the object contained in the current image. When the object is a plant, the candidate object categories can be corn, rice and cotton, the sub-block images are subjected to category prediction according to the image characteristics, and the image characteristics are mapped into a first probability value belonging to the corn category, a second probability belonging to the rice category and a third probability belonging to the cotton category.

In step 1043, the object class corresponding to the probability exceeding the probability threshold is used as the classification result of the sub-block image.

It should be noted that the probability threshold represents a threshold of the probability, and for example, the probability threshold may be set to 60%, and for a candidate object class corresponding to a probability exceeding 60%, the candidate object class is generally an object class covering a large area or all of the image sub-blocks.

In some embodiments, when the probability that the at least two candidate object classes correspond exceeds the probability threshold, the at least two object classes corresponding to the probability exceeding the probability threshold are used as the classification result of the sub-block images.

Here, at least two types of objects can be compatible in the sub-block image, the object type corresponding to the probability exceeding the probability threshold is used as the classification result of the current sub-block image, that is, it is determined which type or types of objects are included in the current sub-block image, and when the objects are plants, it is determined which type or types of plants are planted in the land parcel corresponding to the current sub-block image.

For example, in accordance with the above example, of the first probability belonging to the corn category, the second probability belonging to the rice category, and the third probability belonging to the cotton category, where the first probability and the second probability exceed the probability threshold, the "corn" and the "rice" may be used as the object categories of the image sub-blocks, that is, it is determined that two crops, namely, corn and rice, are planted in the region represented by the current sub-block image.

In other embodiments, when the probability corresponding to at least two candidate object classes exceeds the probability threshold, the object class with the highest probability is determined, and the object class corresponding to the highest probability is used as the classification result of the sub-block images.

Here, only the object class of the same class is planted in the sub-block image, and the object class with the highest probability value among the probabilities exceeding the probability threshold is used as the classification result of the sub-block image, that is, it is determined which class the object contained in the current image sub-block is most likely to be.

For example, in connection with the above example, of the first probability belonging to the corn category, the second probability belonging to the rice category, and the third probability belonging to the cotton category, the first probability and the second probability exceed the probability threshold, and the value of the first probability is the largest, so that the "corn" may be used as the object category of the image sub-block, that is, it is determined that the region represented by the current sub-block image is planted with a corn crop.

Through the embodiment, a more complete land parcel identification processing mode is provided for the situation of planting in each land parcel in a non-single type, and accurate identification of various object types in the land parcels is ensured.

In some embodiments, if the probabilities corresponding to the plurality of candidate object classes do not exceed the probability threshold, it is determined that the class prediction fails, it is determined that the object class is not accurately identified in the current image sub-block, the current image sub-block may be re-identified using the classification model, or a training set of the classification model is enriched based on the image sub-block for which the current class prediction fails, so as to update the classification model and improve the identification accuracy of the classification model.

At this time, if the predicted result is wrong compared with the actual result, the training set of the classification model can be enriched based on the image with failed current recognition, so as to update the classification model and improve the recognition accuracy of the classification model.

Through the embodiment, the classification model can fully learn the wrongly recognized images, so that the recognition errors before the classification model are corrected, and the classification precision is improved.

In step 105, the object class covered by the parcel is determined based on the classification result of the object in each sub-block image.

In some embodiments, step 105 may be implemented by a majority voting method; for example: determining the number of sub-block images belonging to the same object class in the plurality of sub-block images; and determining the sub-block image with the largest number as the object type covered by the land parcel, wherein the sub-block image corresponds to the object type.

By means of a majority voting method, the block-level object class prediction is dispersed to the class prediction of a plurality of sample sub-block images, a plurality of prediction classification results are determined, the prediction classification result with the largest number is determined, the error of classification processing is reduced, and the accuracy and precision of block identification are improved.

In some embodiments, after step 105, different coloring processes are performed on each parcel according to different categories according to preset rules, so as to obtain a parcel classification map of the region remote sensing image. The preset rule may be a corresponding relationship between different object categories and colors.

For example, for each land category, coloring labels of different colors are given to pixels of the image area corresponding to the current land to obtain a land classification map. For example, the image area representing the corn crop land is colored yellow, the image area representing the rice crop land is colored green, and the image area representing the cotton crop land is colored red, so that the land classification map carrying the differently colored remote sensing images of the regions is obtained.

In other embodiments, the preset rules may include correspondence between different object categories and tags.

For example, for each category of the land parcel, different labels are given to the image area corresponding to the current land parcel to obtain a land parcel classification map. For example, a 'corn' label is displayed in an image area representing a corn crop land, a 'rice' label is displayed in an image area representing a rice crop land, and a 'cotton' label is displayed in an image area representing a cotton crop land, so that a land classification map of a region remote sensing image carrying the labels is obtained.

In the above embodiment, the remote sensing region image is segmented by the pre-labeling information to obtain an image of a land parcel, the complex region remote sensing image is simplified, multi-point sampling is performed on the land parcel level image, the image of the land parcel is cut by a preset size to obtain a plurality of effective sub-parcel image samples corresponding to the land parcel, the precision of the image samples is effectively improved, the sub-parcel images are subjected to category prediction according to a pre-trained classification model to obtain a plurality of corresponding classification results, the object category covered by the land parcel is determined, the effect of the classification model on multi-prediction is considered, the optimal recognition result is determined, the probability of recognition error is reduced, the accuracy of land parcel recognition is improved, the classification precision is improved, and the application value of object recognition in the land parcel is improved.

Referring to fig. 4, fig. 4 is an alternative flowchart of a training method of a classification model provided in an embodiment of the present application, where the step of step 104 shown in fig. 3A is implemented by at least one classification model that is pre-trained, where the classification model may be any one of the classification models shown in fig. 8A and fig. 8B, and the description is not repeated below. Before the classification model is obtained, the classification model is trained through step 201 and step 203 shown in fig. 4, which will be described in conjunction with the steps.

In step 201, initializing model parameters of a classification model;

in step 202, performing class prediction on the subblock image samples in the training sample set through a classification model to obtain a prediction object class of the subblock image samples;

here, the class prediction is performed on the sub-block image samples to obtain the prediction object class of the sub-block image samples, and the method of classification processing described above in the embodiments of the present application is referred to, and is not repeated here.

In step 203, the error between the pre-labeled object class and the predicted object class of the sub-block image sample is determined, and the back propagation is performed in the classification model based on the error, so as to update the model parameters of the classification model.

Explaining backward propagation, namely inputting image characteristics into an input layer of a neural network model, passing through a hidden layer, finally reaching an output layer and outputting a prediction object type, which is a forward propagation process of the neural network model, calculating a difference between an output result and an actual result due to a difference between the output result and the actual result of the neural network model, reversely propagating the difference from the output layer to the hidden layer until the difference is propagated to the input layer, and adjusting the value of a model parameter according to an error in the process of the backward propagation; the above process is iterated until the loss function converges.

Referring to fig. 5, which is an optional flowchart of the method for training a classification model provided in the embodiment of the present application, the step 103 shown in fig. 3A is implemented by at least one classification model, and before obtaining the classification model, a training sample set is constructed by the

steps

301 and 303 shown in fig. 5 to train the at least one classification model:

in step 301, a land parcel in which each pre-labeled coordinate point is located is determined in the region image sample based on the plurality of pre-labeled coordinate points.

In some embodiments, prior to acquiring the geographical image sample, the area of interest is typically surveyed and sampled in the field, for example, the sampling point and the attributes of the current sampling point (e.g., the type of crop planted) are acquired in the actual plot area, and a GPS instrument is carried to take a fixed-point photograph to record the attributes and coordinate position of the current sampling point as a pre-labeled coordinate point.

In some embodiments, determining the parcel in which each pre-annotated coordinate point is located in the geographical image sample may be achieved by direct visual interpretation: and observing the remote sensing image data through human eyes, distinguishing a plot area where the coordinate point is located, and marking as a constraint according to the manually added point information to obtain a plot where each pre-marked coordinate point is located.

In other embodiments, the determination of the land parcel in which each pre-labeled coordinate point is located in the region image sample may also be implemented by an open-source labeling tool or other clustering models, which is not limited in this application.

It should be noted that, because the sampling point is generally any point in the parcel, an edge may be obtained when the training sample is obtained by using the square frame sampling, which is not enough to represent the feature of the object in the parcel, the parcel is expanded based on the coordinate point by the method of the above embodiment to perform sampling identification on each parcel, so that fragmentation of the image is reduced, and the efficiency of subsequent sampling and the accuracy of image identification are improved.

In step 302, an image of a land where each pre-marked coordinate point is located is cut out from the region image sample by a preset size, so as to obtain a plurality of sub-block image samples.

Here, the predetermined size is a fixed size in the above embodiment of the present application, and the predetermined size is a regular geometric figure centered on the sampling point. The processing of obtaining a plurality of sub-block images by cutting a block image from a region image sample by a preset size refers to the above embodiments of the present application, and is not described herein again.

In step 303, a training sample set is constructed based on the plurality of sub-block image samples and the corresponding labeled object classes.

In some embodiments, the labeled object class of the corresponding land parcel expanded based on the pre-labeled coordinate point is used as a labeled object class corresponding to a plurality of sub-block image samples corresponding to the same land parcel, and the plurality of sub-block image samples and the corresponding pre-labeled object class are used as training samples to construct a training sample set for training at least one classification model.

In some embodiments, a training sample set is constructed, and a plurality of classification models are trained to implement the method of step 104 of the present application, for example, a subset of training samples is extracted from the training sample set for training each classification model, wherein the subset of training samples of each classification model is different or not identical.

In some embodiments, when step 104 is implemented by a plurality of classification models, before training the classification models, it may further be performed: a subset of training samples is extracted from the set of training samples, wherein the subset of training samples for each classification model is different or not identical.

In practical implementation, a plurality of training sample sets can be constructed in a sample replacement extraction manner to train a plurality of classification models. Exemplarily, in the training sample set, randomly selecting a plurality of sub-training samples to construct a training sample subset 1, putting the plurality of sub-training samples back to the training sample set by using the training sample subset 1 and the pre-training classification model 1, continuously randomly selecting a plurality of training samples to construct a training sample subset 2 and a pre-training classification model 2; the samples are sequentially and repeatedly replaced, training samples are randomly extracted, i training sample subsets are obtained, and corresponding i sub-classification models are obtained through pre-training; wherein i is larger than or equal to 1, and the training sample subsets are not identical.

In other embodiments, a plurality of training sample sets may be constructed in a sample non-replacement extraction manner to train a plurality of classification models. Exemplarily, in the training sample set, a plurality of sub-training samples are randomly selected, a training sample subset 1 is constructed, and a classification model 1 is pre-trained; randomly selecting a plurality of sub-training samples from the rest training sample sets to construct a training sample subset 2; pre-training a classification model 2; and sequentially extracting training samples from the residual training sample sets to obtain i training sample subsets, and pre-training to obtain corresponding i sub-classification models, wherein i is larger than or equal to 1, and the training sample subsets are completely different.

It should be noted that, the training of the i classification models refers to the above classification model training method in the embodiment of the present application, and details are not described herein.

Here, by constructing and training a plurality of sample subsets to train a plurality of sub-classification models, the model effect of each model may not be optimal, but the output results of all models are considered comprehensively, so that an output result with higher accuracy is obtained, errors of feature extraction and classification processing by using one classification model are reduced, and the accuracy of sub-block image classification and identification is improved.

Next, an exemplary application of the embodiment of the present application in a practical application scenario will be described.

Taking a scene of crop identification of an interested region as an example, in order to improve the accuracy and efficiency of crop identification, a region-level image of a region remote sensing image needs to be obtained, and the type of crops is distinguished based on the region-level image, so as to obtain a crop classification map.

The current remote sensing crop identification model is mainly a pixel-level method based on rules or machine learning, such as a Mahalanobis distance method, a parallelepiped classification method, a support vector machine and the like, the traditional characteristic extraction method is low in efficiency, is not suitable for crop identification in complex regions, and is low in precision and serious in fragmentation. Although deep learning has advantages in feature extraction, real crop data needs to be acquired manually in the field, fixed points are arbitrary, cost is high, sufficient data volume does not exist, and the deep learning is difficult to be directly used for training of a deep neural network and a semantic segmentation model; although there are some attempts to classify crops by deep learning, the fragmentation of sample image data obtained by the conventional sampling method (e.g. direct windowing to obtain training samples) is serious, and the accuracy of the conventional prediction method (e.g. sliding window center small block full-map prediction) is low because the distribution of the land blocks is not clear.

In the practical application of distinguishing the crop distribution by using the remote sensing images, the number of the remote sensing images is huge, the related areas are complex and may be country-level, provincial-level and city-level remote sensing images and the like, and the embodiment of the application is used for identifying crops in interested areas with smaller ranges, for example, identifying crops in a certain county and city area.

Because the crop plots of the interested regions (such as a certain county city region) can not be changed basically for a long time, the divided plots can be directly marked manually, fine adjustment and reloading are carried out for use, and the crop planting type is identified for each farmland plot on the basis of manual division of the farmland plots, so that the crop classification map is obtained.

Fig. 9 is a flowchart illustrating a training method of a classification model according to an embodiment of the present application, where the classification method of an object in a parcel according to the embodiment of the present application can be implemented by at least one classification model, and referring to fig. 9, fig. 9 illustrates a method in a training phase of the above classification model, which is described with reference to steps 401 to 404 below.

Step 401: coordinates and crop planting types for a field survey are obtained.

In actual implementation, experimenters conduct field investigation on a specific farmland area, take fixed-point pictures by using a Global Positioning System (GPS) instrument, and record coordinate positions of a plurality of sampling points and attributes (crop planting types) of the sampling points.

Step 402: obtaining a region remote sensing image, making a remote sensing image plot-level farmland label, and expanding the position of the coordinate into a whole farmland.

In some embodiments, the coordinates are generally any point in the farmland, and the edges of the farmland plots may be obtained when the training samples are obtained by conventional block sampling, and the non-farmland areas (such as roads, weeds and the like) or other plot areas (such as farmland plots planted with different types of crops) are covered by the training samples, which are not enough to represent the characteristics of the crops of the farmland plots, so that the farmland plots are artificially expanded to make plot-level crop labels according to the coordinate positions and the crop types of the sampling points recorded in step 401.

For example, by manually observing remote sensing image data, a farmland plot area where a coordinate point is located is distinguished, and a farmland plot where each pre-marked coordinate point is located is obtained by using a manually added point information mark as a constraint.

Step 403: based on each farmland plot and the marked crop type, N random points in the farmland plots are taken, and training sample images are cut out.

For example, N sampling points are randomly obtained in a farmland block, a fixed size is used as a preset size, each sampling point is used as a center, an upper left size/2 and a lower right size/2 are shifted to be used as sampling windows, a picture with the size of the size is cut out from a block-level remote sensing image to be used as a training sample image, for example, a 25 × 3 sampling window can be selected for image cutting, and the image with the size of the size is cut out from each sampling point in sequence through the size sampling window to obtain N training sample images corresponding to the corresponding farmland block.

It should be noted that the requirement of the fixed size is preset, and is determined according to a large amount of experimental data and priori knowledge, and generally a small amount of outside of the plot is required to be covered, so that the sampling window can wrap the whole farmland as much as possible.

Step 404: according to the training sample image and the corresponding crop type mark, a crop classification model (namely a classification model) is obtained by using deep learning training, for example, the crop classification model can be a residual error network model ResNet, a dense connection network model DenseNet or a self-defined deep learning model.

In some embodiments, a training sample set is constructed by N training sample images corresponding to a farmland plot and labeled crop types, the training sample set is input into a crop classification model, sub-block image samples in the training sample set are subjected to class prediction through the crop classification model to obtain predicted plant classes of the sub-block image samples, errors between the labeled crop types and the predicted plant classes are determined, and model parameters of the plot classification model are updated according to the errors.

Fig. 10 is a schematic flowchart of a method for classifying objects in a parcel, which is provided in the embodiment of the present application, and referring to fig. 10, a classification model trained based on the embodiment of the present application may classify crop regions in a remote sensing image to obtain a corresponding crop classification map, and a test stage for parcel recognition based on the classification model trained in the embodiment of the present application, which is shown in fig. 10, is described with reference to steps 501 to 506 below.

Step 501, test image data is acquired.

The test image data is a regional remote sensing image set, for example, a regional remote sensing image set of prefecture a, B, city, C, collected by a channel such as unmanned aerial vehicle fixed-point aerial photography or satellite photography.

And step 502, dividing the region of interest farmland plots.

For example, the plot for planting crops is segmented in the region remote sensing image according to manual labeling or historical vector labeling, and an image comprising a plurality of farmland plots is obtained.

It should be noted that, for the remote sensing image collected in prefecture C, B, province a, B, and prefecture B, the crop plot in the region is not updated frequently, i.e., the farmland plot (boundary) is not changed frequently, and manual labeling can be performed, or the history vector label can be called for loading and using, and here, the history vector label can be finely adjusted and repeatedly loaded for use, so that the labeling cost is relatively low, and the history vector label is very fixed.

And 503, taking M random points in each farmland plot, and cutting out a test sample image.

For example, for each farmland plot, M sampling points are randomly obtained, and the image of the farmland plot is cropped to obtain corresponding M sample images (block images) with the fixed size provided above in the embodiment of the present application.

And step 504, carrying out crop classification on the M sample images by using a crop classification model to obtain M classification results.

In some embodiments, a crop classification model is used to perform feature extraction processing on the sample images, so as to obtain image features corresponding to each sample image; and according to the image characteristics, performing category prediction on the sample images, mapping the image characteristics into probabilities corresponding to a plurality of candidate crop categories, taking the crop category which exceeds a probability threshold and is corresponding to the probability with the maximum probability value as a classification result of the sample images, and sequentially performing category prediction on the M sample images by using a crop classification model to obtain M classification results.

It should be noted that the classification result is generally the result of the class with the largest probability, and the sum of the probabilities is generally 100%. Illustratively, if the candidate crop types are "corn", "rice" and "other", for a farmland plot, the crop classification model is used to perform feature extraction and classification processing on M sample images corresponding to the current plot respectively, so as to obtain a probability that the prediction type is "corn" of 20%, a probability that the prediction type is "rice" of 70%, and a probability that the prediction type is "other" of 10%, and since the probability that the prediction type is "rice" is the largest, it can be determined that the type of crop planted in the current sample image is "rice".

And 505, performing majority voting on the M classification results, determining a final classification result, and assigning the result to a corresponding parcel.

In some embodiments, the number of the M classification results belonging to the same classification result is determined, the classification result with the largest number is determined as the classification result of the farmland plots corresponding to the M image samples, and the classification result is assigned to the current plot.

Illustratively, for a farmland plot, if 50 sample images are cut, the crop classification model is used for respectively carrying out feature extraction and classification on the sample images to obtain 40 classification results predicted as rice, 9 classification results predicted as corn and 1 classification result predicted as cotton, and the classification result of the current plot is determined as rice by means of majority voting.

And step 506, repeatedly executing the step 503 to the step 505 to obtain a crop classification map of the test image.

For example, a crop classification model is used to classify and predict a plurality of farmland plots respectively to obtain a classification result of each plot, and different coloring or labeling is performed on each farmland plot according to different crop categories according to a preset rule to obtain a crop classification map of a region remote sensing image. The preset rule can be the corresponding relation between different plant categories and colors or the corresponding relation between different plant categories and labels.

Through the above-mentioned embodiment of this application, a plurality of samples training crops classification model are got in the farmland plot of true data, through the artifical mark of region of interest or to farmland plot historical vector mark fine setting when carrying out crops discernment test, cut apart out the farmland plot, and get every plot and carry out the sample of windowing in multiple spot, utilize crops classification model to carry out classification prediction to a plurality of samples, and carry out the majority vote to a plurality of classification results, assign the result that obtains for every plot, thereby promote the categorised precision of crops, promote crops discernment's using value.

Continuing with the exemplary structure provided by the present application for the sorting apparatus 555 of objects in a parcel as a software module, in some embodiments, as shown in fig. 2, the software module stored in the sorting apparatus 555 of objects in a parcel in the memory 550 may include: the image acquisition module 5551 is used for acquiring a region image; the image cutting module 5552 is configured to determine an area of interest in the region image according to the pre-labeling information, and cut out an image of the area of interest from the region image to serve as an image of the land parcel; the image cutting module 5553 is configured to cut the image of the parcel by a preset size based on a plurality of sampling points in the parcel to obtain a plurality of sub-block images; a parcel category determination module 5554, configured to obtain a classification result of an object in each of the sub-block images based on an image feature of each of the sub-block images, and determine a category of the object covered by the parcel based on the classification result of the object in each of the sub-block images.

In the above solution, the image cutting module 5553 is further configured to perform the following processing for each parcel: randomly acquiring a plurality of sampling points in the plot; based on the plurality of sampling points, cutting the image of the land block by a preset size respectively to obtain a plurality of sub-block images corresponding to the land block; the preset size is a regular geometric figure taking the sampling point as a center.

In the above scheme, the block type determining module 5554 is further configured to perform the following processing for each sub-block image: mapping image features extracted from the sub-block images to probabilities corresponding to a plurality of candidate object classes; and taking the object class corresponding to the probability exceeding the probability threshold as the classification result of the sub-block images.

In the foregoing solution, the land parcel type determining module 5554 is further configured to, when the probabilities corresponding to at least two candidate object types exceed the probability threshold, use the object types corresponding to the at least two probabilities exceeding the probability threshold as the classification result of the sub-block image, or use the object type corresponding to the maximum probability as the classification result of the sub-block image.

In the above solution, the obtaining of the classification result of the object in the corresponding sub-block image based on the image feature of each sub-block image is implemented by at least one classification model; the parcel category determination module 5554 is further configured to perform the following processing by the same classification model: classifying objects in the corresponding sub-block images respectively based on image features extracted from each sub-block image to obtain classification results of the objects in the sub-block images; alternatively, the following is performed by a plurality of classification models: and classifying the objects in the sub-block images one by one based on the image characteristics of the sub-block images to obtain the classification result of the objects in the sub-block images.

In the above solution, the block type determining module 5554 is further configured to determine and count the number of sub-block images belonging to the same object type in each sub-block image; and determining the object type corresponding to the subblock image with the largest number as the object type covered by the parcel.

In the above solution, the obtaining of the classification result of the object in the corresponding sub-block image based on the image feature of each sub-block image is implemented by at least one classification model, and the apparatus for classifying the object in the parcel further includes: a classification model training module 5554 for training the classification model by: initializing model parameters of a classification model; performing class prediction on the sub-block image samples in the training sample set through the classification model to obtain prediction object classes of the sub-block image samples; and determining errors of the pre-marked object class and the predicted object class of the sub-block image samples, and performing back propagation in the classification model based on the errors so as to update the model parameters of the classification model.

In the above solution, when the obtaining of the classification result of the object in the sub-block image based on the image feature of each sub-block image is implemented by a plurality of classification models, the classification model training module 5554 is further configured to extract a training sample subset from the training sample set, where the training sample subset of each classification model is different or not identical.

In the foregoing solution, the classification model training module 5554 is further configured to determine, based on a plurality of pre-labeled coordinate points, a land parcel where each pre-labeled coordinate point is located in a region image sample; cutting an image of the land where each pre-marked coordinate point is located from the region image sample according to a preset size to obtain a plurality of sub-block image samples; and constructing the training sample set based on the plurality of sub-block image samples and the corresponding labeled object classes.

It should be noted that the description of the apparatus in the embodiment of the present application is similar to the description of the method embodiment, and has similar beneficial effects to the method embodiment, and therefore, the description is not repeated.

An embodiment of the present application provides an electronic device, including: a memory for storing executable instructions; and the processor is used for realizing the classification method of the objects in the plot when executing the executable instructions stored in the memory.

Embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to execute the classification method for the objects in the parcel as described above in the embodiment of the present application.

Embodiments of the present application provide a computer-readable storage medium storing executable instructions, which when executed by a processor, will cause the processor to perform the methods provided by embodiments of the present application, for example, the classification method of objects in a parcel as shown in fig. 3A, fig. 3B, and so on.

In some embodiments, the computer-readable storage medium may be memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

In some embodiments, executable instructions may be written in any form of programming language (including compiled or interpreted languages), in the form of programs, software modules, scripts or code, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

By way of example, executable instructions may correspond, but do not necessarily have to correspond, to files in a file system, and may be stored in a portion of a file that holds other programs or data, such as in one or more scripts in a hypertext Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code).

By way of example, executable instructions may be deployed to be executed on one computing device or on multiple computing devices at one site or distributed across multiple sites and interconnected by a communication network.

In summary, according to the embodiments of the present application, the remote sensing region image is segmented based on the pre-labeling information to obtain the image of the land parcel, the complex region remote sensing image is simplified to be a land parcel level region, multi-point sampling is performed on the land parcel level image, the image of the land parcel is cut by a preset size to obtain a plurality of effective sub-parcel image samples corresponding to the land parcel, the precision of the image samples is improved, the sub-parcel images are subjected to category prediction according to the pre-trained classification model to obtain a plurality of corresponding classification results, the object category covered by the land parcel is determined, the effect of the multi-prediction of the classification model is considered, the optimal recognition result is determined, the probability of recognition errors is reduced, the accuracy of the land parcel recognition is improved, so that the precision of the object classification in the land parcel is improved, and the application value of the land parcel recognition is improved.

The above description is only an example of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, and improvement made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method of classifying objects in a parcel, the method comprising:

acquiring a region image;

2. The method of claim 1, wherein prior to determining the region of interest in the region image from pre-annotation information, the method further comprises:

responding to the marking operation, and acquiring manually marked pre-marking information; or

And acquiring pre-labeled information labeled based on the historical vectors.

3. The method of claim 1, wherein the cutting the image of the land block by a preset size based on the plurality of sampling points in the land block to obtain a plurality of sub-block images comprises:

the following processing is performed for each plot:

randomly acquiring a plurality of sampling points in the plot;

based on the plurality of sampling points, cutting the image of the land block by a preset size respectively to obtain a plurality of sub-block images corresponding to the land block;

the preset size is a regular geometric figure taking the sampling point as a center.

4. The method of claim 1, wherein the obtaining a classification result of the object in the corresponding sub-block image based on the image feature of each sub-block image comprises:

performing the following processing for each of the subblock images:

mapping image features extracted from the sub-block images to probabilities corresponding to a plurality of candidate object classes;

and taking the object class corresponding to the probability exceeding the probability threshold as the classification result of the sub-block images.

5. The method according to claim 4, wherein the classifying the object class corresponding to the probability exceeding the probability threshold as the sub-block image comprises:

and when the probabilities corresponding to at least two candidate object categories exceed a probability threshold, taking the object categories corresponding to the at least two candidate object categories exceeding the probability threshold as the classification results of the sub-block images, or taking the object category corresponding to the maximum probability as the classification results of the sub-block images.

6. The method of claim 1,

the obtaining of the classification result of the object in the corresponding sub-block image based on the image features of each sub-block image is realized by at least one classification model;

the obtaining of the classification result of the object in each of the sub-block images based on the image features of each of the sub-block images includes:

the following processing is performed by the same classification model: classifying objects in the corresponding sub-block images respectively based on image features extracted from each sub-block image to obtain classification results of the objects in the sub-block images;

alternatively, the following is performed by a plurality of classification models: and classifying the objects in the sub-block images one by one based on the image characteristics of the sub-block images to obtain the classification result of the objects in the sub-block images.

7. The method of claim 1, wherein determining the object class covered by the parcel based on the classification result of the object in each of the sub-block images comprises:

determining and counting the number of sub-block images belonging to the same object class in each sub-block image;

and determining the object type corresponding to the subblock image with the largest number as the object type covered by the parcel.

8. The method of claim 1,

the obtaining of the classification result of the object in the corresponding sub-block image based on the image feature of each sub-block image is implemented by at least one classification model, and before obtaining the classification result of the object in the corresponding sub-block image based on the image feature of each sub-block image, the method further includes:

training the classification model by:

initializing model parameters of the classification model;

performing class prediction on the sub-block image samples in the training sample set through the classification model to obtain prediction object classes of the sub-block image samples;

and determining errors of the pre-marked object class and the predicted object class of the sub-block image samples, and performing back propagation in the classification model based on the errors so as to update the model parameters of the classification model.

9. The method of claim 8, wherein when the obtaining of the classification result of the object in each of the sub-block images based on the image features of the sub-block images is performed by a plurality of classification models, before training the classification models, the method further comprises:

and extracting training sample subsets from the training sample set, wherein the training sample subsets of each classification model are different or not completely the same.

10. The method of claim 8 or 9, wherein prior to training the classification model, the method further comprises:

determining a land parcel where each pre-marked coordinate point is located in the region image sample based on the plurality of pre-marked coordinate points;

cutting an image of the land where each pre-marked coordinate point is located from the region image sample according to a preset size to obtain a plurality of sub-block image samples;

and constructing the training sample set based on the plurality of sub-block image samples and the corresponding labeled object classes.

11. An apparatus for classifying objects in a parcel, comprising:

the image acquisition module is used for acquiring a region image;

12. An electronic device, comprising:

a memory for storing executable instructions;

a processor for implementing the method of classifying objects in a parcel as claimed in any one of claims 1 to 10 when executing executable instructions stored in the memory.

13. A computer-readable storage medium storing executable instructions for implementing the method of classifying objects in a parcel as claimed in any one of claims 1 to 10 when executed by a processor.

14. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, carries out the method of classification of objects in a parcel as claimed in any one of claims 1 to 10.