CN115100471A - Multi-dimensional information acquisition and early warning method for grain storage pests based on mobile terminal - Google Patents
Multi-dimensional information acquisition and early warning method for grain storage pests based on mobile terminal Download PDFInfo
- Publication number
- CN115100471A CN115100471A CN202210723114.3A CN202210723114A CN115100471A CN 115100471 A CN115100471 A CN 115100471A CN 202210723114 A CN202210723114 A CN 202210723114A CN 115100471 A CN115100471 A CN 115100471A
- Authority
- CN
- China
- Prior art keywords
- pest
- image
- granary
- pests
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/18—Status alarms
- G08B21/182—Level alarms, e.g. alarms responsive to variables exceeding a threshold
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30242—Counting objects in image
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Medical Informatics (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- Quality & Reliability (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Catching Or Destruction (AREA)
Abstract
The invention relates to the field of granary pest monitoring, in particular to a mobile terminal-based multi-dimensional information acquisition and early warning method for stored grain pests, and the technical scheme comprises the following steps: establishing basic information of a granary, defining the name of a target area where pests appear in the granary, and setting a corresponding pest susceptibility grade; acquiring an image of a target area, and performing off-line identification on the image of the target area through an image data set and an image identification algorithm embedded in the mobile terminal to acquire pest species and corresponding quantity information in the image; establishing basic pest situation data according to the basic information of the granary, the corresponding pest susceptibility grade, the obtained pest type and the corresponding quantity information; acquiring temperature and humidity information of a grain pile in the granary, and correlating basic pest situation data corresponding to the temperature and humidity information of the grain pile to obtain multi-dimensional information of grain storage pests in the granary. The invention is suitable for monitoring and managing the pests in the granary.
Description
Technical Field
The invention relates to the field of granary pest damage monitoring, in particular to a mobile terminal-based multi-dimensional information acquisition and early warning method for stored grain pests.
Background
At present, through mobile phone APP software, pest points on the surface of a granary can be selected to shoot pictures and identify pests in the pictures, the quantity and the type information of the pests are obtained, then the quantity and the type information of the pests and the shot position information are packaged and sent to a remote server to be stored, data display is carried out at a mobile phone end, an electronic picture can be embedded into the software, and the type of pest identification is associated with the electronic picture. A user clicks the pest name of the display page or inputs the pest name through the page, so that the detailed pest introduction information of the pest can be inquired, and the searched pest introduction information is pushed to the user. The software and the method can timely and conveniently acquire pest images and pest occurrence quantity and species information of image acquisition points, conveniently acquire pest introduction information, avoid manual searching, save workload and facilitate users to visually know pest conditions in the granary.
The software method does not combine pest monitoring management with a grain pile ecosystem, collected information is not comprehensive enough, flexibility is poor, pest detection and species identification in images are not accurate enough, so that accuracy of pest early warning is affected, limitation is large, and specific conditions of pest damage of grain piles in the granary cannot be comprehensively reflected.
Disclosure of Invention
The invention aims to provide a multi-dimensional information acquisition and early warning method for grain storage pests based on a mobile terminal, which realizes multi-dimensional information acquisition and early warning for the grain storage pests in a granary, more comprehensively reflects the specific conditions of the pests in grain piles in the granary, and improves the accuracy of the information acquisition and early warning for the pests.
The invention adopts the following technical scheme to realize the aim, and the multi-dimensional information acquisition method for the grain storage pests based on the mobile terminal comprises the following steps:
establishing basic information of a granary, defining the name of a target area where pests appear in the granary, and setting a corresponding pest susceptibility grade;
acquiring images of a target area, wherein the images comprise acquired images of stored grain surfaces and images acquired after sampling and insect screening corresponding to different stored grain pile depths;
performing off-line identification on the image of the target area through an image data set and an image identification algorithm embedded in the mobile terminal to obtain pest species and corresponding quantity information in the image;
establishing basic pest situation data according to the basic information of the granary, the corresponding pest easy-to-send grade, the obtained pest species and the corresponding quantity information;
acquiring temperature and humidity information of a grain pile in the granary, and correlating the temperature and humidity information of the grain pile with corresponding basic insect pest situation data to obtain multi-dimensional information of grain storage pests in the granary.
Further, in order to facilitate a user to view the granary grain storage information more comprehensively, the granary basic information comprises: granary coding, granary number, granary type, grain types stored in the granary, moisture content, warehousing time and warehousing time.
Further, in order to more comprehensively reflect the pest information in the granary, the basic pest situation data comprises: granary coding, bin type, target area, pest species and corresponding quantity, grain species and grain moisture.
Further, in order to improve the acquisition efficiency, the target area name includes: the system comprises a warehouse four-angle point, a warehouse door point, an exhaust fan point, a warehouse window point, an entrance inspection door point and a middle point, wherein the warehouse four-angle point, the warehouse door point, the exhaust fan point, the warehouse window point, the entrance inspection door point and the middle point are three-dimensional target areas with different depths.
Further, in order to improve the accuracy of image recognition, the image recognition algorithm specifically includes:
and 5, determining the reliability of the pest category in the image and the number of different kinds of pests in the image according to the predicted pest category and the pest target frame, and taking the average accuracy as an evaluation index of the pest identification performance.
Further, in step 1, before extracting the feature map of the image, performing sliding window clipping processing on the image, and clipping the original pest image collected by the mobile terminal to a corresponding size.
Further, in step 2, when the feature pyramid is constructed, for the scale range of the pest target size, based on the receptive field sizes of different levels of the basic network, the feature layer of a specific level of the basic network is selected, and the feature pyramid is constructed through the operation of a hole convolution, deconvolution or attention mechanism of space and channel. Response characteristics of the neural network are utilized to the maximum extent, characteristics with enough granularity and abstract semantic information are provided for positioning and type identification of pest targets with different scales, effectiveness and difference of extracted characteristics are improved, and effective detection and accurate type identification of the pest targets are achieved.
Further, in step 3, the step length of each feature pyramid level determines a preset target frame scale interval corresponding to the level, the preset target frame scale is set to be n times of the scale interval, n is an integer greater than 1, so as to ensure that the preset frames with different scales have the same density in the image, so that the pest targets with multiple scales can be approximately matched with the preset frames with the same number, the pest targets with different scales can be detected as far as possible, and the missing detection condition of the pest targets is reduced.
Further, in step 4, predicting pest species and pest target frame specifically includes:
respectively connecting a classification sub-network and a boundary frame regression sub-network after each feature pyramid level, wherein the classification sub-network predicts the occurrence probability of each pest in an anchor point generated at each spatial position and predicts the probability that pest targets appearing in a plurality of preset frames of each position unit on a detection layer belong to a plurality of target categories; performing regression prediction on the offset between each preset frame matched with the pest target and the true value of the mark by using a boundary frame regression subnetwork; in the classification and bounding box regression subnetwork, the attention feature extraction of space and channel is carried out on the features through convolution layer and maximum pooling or average pooling operation, and the accuracy of the species prediction and bounding box coordinate regression of the pest target is further improved.
Further, determining the reliability of the pest category in the image and the number of different kinds of pests in the image specifically includes:
and carrying out non-maximum value inhibition operation according to the score of each bounding box, filtering repeated and low-score detection results, determining the credibility of the pest belonging to each category respectively for the pest target obtained by detection in the image, and giving the number of different types of pest targets in the image. Because a large number of simple samples and negative samples may exist in the training process, the method adopts the Focal loss method to reduce the influence of the large number of negative samples and the simple samples on the model training process, and improves the attention of the model on difficult samples so as to improve the overall recognition accuracy of the model; further, regression accuracy of most samples is improved by taking the inverse of the first order norm of the regression loss gradient values as the weight of the regression loss.
Furthermore, images are acquired according to different image acquisition frequencies, and the image acquisition method specifically comprises a personal setting method, an averaging method and a maximum value method; the personal setting method specifically includes: according to different seasons and the management experience of an administrator, the acquisition frequency, the number and the positions of sampling points are set at a server end by the administrator, and the image acquisition is carried out by pushing a notification through an interface.
Further, the averaging method specifically includes:
embedding an effective accumulated temperature generation model of pests into a server, associating effective accumulated temperature parameters K corresponding to different types of pests and starting point temperature of pest development according to an image recognition result, counting to obtain the number of target areas of the same type of pests, and calculating the time length T of the area in a set time L Inner average total accumulated temperature K T1 Then, the number of generations of pest occurrence: n is I ＝K T1 /K，n I The value of (d) is proportional to the frequency or time of acquisition; sampling time interval of T 1 Frequency of f 1 Then T is 1 ＝1/f 1 ＝T L /n I ；
Collecting frequency, the number and the positions of sampling points, and pushing a notice through an interface to collect images.
Further, the maximum method specifically includes:
embedding an effective accumulated temperature generation model of pests into a server, selecting the pest with the greatest harm in the identification result according to the image identification result, associating an effective accumulated temperature parameter K corresponding to the pests and the starting point temperature of pest development, and calculating to obtain the target region of the pests within a set time length T L Inner maximum total accumulated temperature K TM Point, the number of generations of pest occurrence: n is IM ＝K TM K, at maximum total accumulated temperature K TM Collecting information at points with sampling time interval of T 2 Frequency of f 2 Then T is 2 ＝1/f 2 ＝T L /n IM And acquiring the frequency, the number and the positions of sampling points, and pushing and informing through an interface to acquire images.
A multi-dimensional information early warning method for grain storage pests based on a mobile terminal comprises the following steps:
the generation number n of pest occurrence is set according to seasons I And n IM Comparing the value with the results calculated by the averaging method and the maximum method, and giving early warning information of the target area if the results calculated by the averaging method and the maximum method exceed any set value in the season;
or counting the pest growth number and the temperature and humidity data change amplitude of the target area by taking the set time as a basic counting time unit in the target area to obtain the maximum number of the sum of the pest growth number and the temperature and humidity data change amplitude and the corresponding target area, and giving early warning information at the equipment end;
or obtaining an image recognition result of the target area, and performing early warning by converting the image recognition result into the insect population density, wherein the insect population density calculation specifically comprises the following steps:
P d1 the number of identified pests in the grain surface pest picture/picture area is multiplied by A multiplied by grain volume weight;
P d2 the insect population density is (n × P) d1 +m*P d2 ) V (n + m); a is height data, n represents the number of the grain target area shooting points, and m represents the number of the sampling insect screening target area shooting points.
According to the invention, based on a granary ecosystem, granary basic information is established, a target area is defined, an image of the target area is obtained, the image of the target area is identified in an off-line manner through an image data set and an image identification algorithm which are embedded in a mobile terminal, the pest type and the corresponding quantity information in the image are obtained, and the flexibility of image identification is improved; establishing basic pest situation data according to the basic information of the granary, the corresponding pest outbreak level, the obtained pest species and the corresponding quantity information, obtaining the temperature and humidity information of the grain stack in the granary, and correlating the basic pest situation data corresponding to the temperature and humidity information of the grain stack to obtain the multidimensional information of the grain storage pests in the granary, thereby improving the comprehensiveness of the pest information of the granary. Compared with a common method for manually setting an acquisition period or manually judging the experience, the method for setting the data acquisition frequency based on the pest growth and development accumulated temperature model design is more scientific and accurate, reduces the dependence on the experience of a custodian, improves the sensitivity of monitoring the pest occurrence condition and provides data support for implementing accurate, timely and effective prevention and control measures. Setting an alarm threshold value aiming at seasons, dynamically adjusting the early warning threshold value, improving the robustness of the early warning result to the change of the seasons, and reducing the occurrence of false alarm and delayed alarm; the monitoring of the time dimension is increased, the temperature and humidity change of the grain pile and the change of the pest detection result in a period of time are considered, the reliability of early warning can be further improved, more accurate assessment of the pest occurrence and development conditions is given, and more accurate early warning results are made.
Drawings
FIG. 1 is a schematic diagram of the location and semantic definition of a target area according to the present invention;
fig. 2 is a flowchart of an image recognition method according to an embodiment of the present invention.
In the attached drawing, 1 is a bin four-corner point, 2 is a bin window point, 3 is a door point, 4 is an exhaust fan point, 5 is a middle point, and 6 is an entrance inspection door.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The following describes in detail embodiments of the present invention with reference to the drawings.
A multi-dimensional information acquisition method for grain storage pests based on a mobile terminal comprises the following steps:
establishing basic information of a granary, defining the name of a target area in which pests appear in the granary, and setting a corresponding pest susceptibility grade;
acquiring images of a target area, wherein the images comprise acquired images of stored grain surfaces and images acquired after sampling and insect screening corresponding to different stored grain pile depths;
performing off-line identification on the image of the target area through an image data set and an image identification algorithm embedded in the mobile terminal to obtain pest species and corresponding quantity information in the image;
establishing basic pest situation data according to the basic information of the granary, the corresponding pest susceptibility grade, the obtained pest type and the corresponding quantity information;
acquiring temperature and humidity information of a grain pile in the granary, and correlating the temperature and humidity information of the grain pile with corresponding basic insect pest situation data to obtain multi-dimensional information of grain storage pests in the granary.
The basic information mainly includes: the information of the granary code, the granary number, the granary type, the grain type stored in the granary, the moisture content, the warehousing time and the like is uploaded to the server to be stored, so that a user can check the grain storage information in the granary at any time through the terminal equipment. The user can also modify the information through the terminal equipment.
The mobile terminal can be a mobile phone or a specially developed portable embedded device, and the terminal device can communicate with a remote server through a WiFi or 4G/5G network to receive or upload information.
The system administrator or the super administrator can set the granary administrator at the server side, and the granary administrator can distribute the granary quality protection personnel and the granary managed by the granary quality protection personnel.
In an embodiment of the present invention, semantic definitions of 6 types of target regions are provided, as shown by each point label in fig. 1, including a warehouse four-corner point 1, a door point 3, an exhaust fan point 4, a warehouse window point 2, an entrance inspection door point 6, and a middle point 5, where the 6 target regions are three-dimensional target regions with different depths. The level of probability of susceptibility to pest given to these spots may be set, a uniform level may be set, or different levels may be set. Taking a high and large square cabin as an example, the areas are areas which are easy to grow insects and need to be monitored in an important mode, wherein the middle point is a point which is easy to accumulate impurities when grain is fed, and the points are also easy to grow insects. The areas are assigned with unique numbers, the numbers are associated with semantics, levels easy to grow insects and positions in the warehouse, not only can the specific positions of the target areas in the warehouse be determined, but also more associated information of the positions of the target areas is assigned, and the information is necessary for timely positioning pest occurrence and early warning. The northeast corner of the bin may be set as the origin of coordinates so that the location coordinates of the target area are determinable within the bin.
The mobile terminal is embedded with a self-built image data set and an image recognition algorithm, so that the image of the target area can be recognized on the terminal in an off-line manner, and the pest species and quantity information in the image can be obtained without uploading to a server for recognition. The pest identification information and the collected position information can be displayed to a user on the mobile terminal. After the user clicks the name of any pest type on the display interface, the name can be automatically associated with the picture of the pest, and the multi-angle professional introduction information of the pest is pushed to the user.
And the image recognition algorithm gives the information of the recognition result, namely the category of the pests and the accuracy of the pests to which the pests belong. In combination with the above, a piece of basic pest situation data lid (local event data) is defined as: granary coding + bin type + target area + image recognition result (type and quantity) + grain type + grain moisture. Examples of data are shown in table 1.
Table 1 basic insect data LID definition example
Granary coding, granary type, grain type and grain moisture data, and a manager sets the granary coding, granary type, grain type and grain moisture data through a terminal software interface; the target area point determines the position by clicking a position point icon given by a terminal software interface, and fills or selects other information of the target area; and the terminal software takes pictures of the target area to automatically obtain a pest identification result. And sending the basic insect condition LID data to a remote server, and storing the data in a database.
The image recognition method provided by the embodiment of the invention is shown in fig. 2, and comprises the following steps:
and 5, determining the reliability of the pest category in the image and the number of different kinds of pests in the image according to the predicted pest category and the pest target frame, and taking the average accuracy as an evaluation index of the pest identification performance.
In step 1, the resolution of the pest image acquired by the mobile terminal is 2000 × 1000 pixels or more, and the spatial resolution of the pest target in the image is distributed from less than 40 × 40 pixels to 256 × 256 pixels or more. Therefore, before performing recognition model training nuclear test on the pest image, sliding window clipping processing needs to be performed on the pest image, and an original pest image collected by the mobile terminal is clipped to a proper size (the maximum side length is less than 1024 pixels). The image of the pest acquired by the mobile terminal is an RGB (red, green, blue) image, and the RGB image is a color image, so that the feature extraction is performed on the input image by using a convolution neural network with a convolution kernel with a first layer of three channels. The convolutional neural networks for image feature extraction are collectively referred to as a base network, and the base network can be a convolutional neural network model with different layer numbers, such as VGG, ResNet, inclusion, DenseNet or DarkNet.
In the step 2, considering that the spatial resolution of the pest targets in the image is very different, different levels of feature pyramids are constructed for the pest targets in different scale ranges to match the effective receptive fields of the convolutional neural network. When the characteristic pyramid is built, based on the sizes of the receptive fields of different levels of the basic network, the characteristic layer of a specific level of the basic network is selected, and the characteristic pyramid is built through operations such as some extra convolutions, cavity convolutions, deconvolution or attention mechanisms of spaces and channels, so that the response characteristics of the neural network are utilized to the maximum extent, the characteristics with enough granularity and abstract semantic information are provided for positioning and type identification of pest targets with different scales, the effectiveness and the difference of extracted characteristics are improved, and the effective detection and the accurate type identification of the pest targets are realized.
In the step 3, for the effective receptive fields of the neural networks with different feature pyramid levels, a preset frame with a corresponding scale size is generated for the scale size of the pest target possibly appearing in the image. Based on the input image, the step length of each feature pyramid level determines a preset frame scale interval corresponding to the level. The scale of the preset frame can be set to be 4 times of the scale interval of the preset frame, so that the preset frames with different scales have the same density in the image, the pest targets with multiple scales can be approximately matched with the preset frames with the same number, the pest targets with different scales can be detected as far as possible, and the condition of missed detection of the pest targets is reduced. If pest detection is performed on a feature pyramid with a step size of {4, 8, 16, 32, 64}, the base size of the preset box may be set to {16, 32, 64, 128, 256 }. In addition, at each level of pyramid level, preset boxes are generated according to the three aspect ratios {1:2,1:1,2:1}, respectively, and in order to enable denser scale coverage, the preset box sizes of the three aspect ratios are scaled by {20,21/3,22/3} at each level. Thus, 9 preset boxes are formed on each unit of each level. Through the generation of the preset frames in all levels of the pyramid layers, the scale of the generated preset frames covers the pixel range of 16-577 in the input image.
In step 4 above, a classification and bounding box regression subnetwork is connected after each feature pyramid level, respectively, the classification subnetwork predicting the probability of each pest appearing in the anchor points generated at each spatial position. The bounding box regression subnetwork regression predicts the offset between each preset box matched to a pest target and the true value of the signature. The classification sub-network predicts the probability that pest targets appearing in 9 preset boxes of each position unit on the detection layer belong to C target categories, wherein C represents the number of types of main grain storage pests appearing in the granary and is an integer larger than 1. The bounding box regression subnetwork is constructed substantially the same as the classification subnetwork except that it has 4 x 9 linear outputs at each spatial location for regressing the offset between each preset box and the actual label value (ground route) closest to it. The sub-network of each pyramid layer only processes the preset boxes within a certain scale generated at this layer. Through classification and bounding box regression sub-networks, species prediction and bounding box coordinates of pest targets can be obtained. In a classification and bounding box regression sub-network, in the classification sub-network, the attention feature extraction of space and channel is carried out on the features through convolution layer and maximum and average pooling operation, thereby further improving the accuracy of species prediction and bounding box coordinate regression of the pest target.
In the step 5, aiming at the given pest target boundary frame and type identification result of the pest identification model, non-maximum value inhibition operation is carried out according to the score of each boundary frame, and repeated and low-score detection results are filtered. And finally, determining the credibility that the pests belong to each category respectively for the pest targets detected in the image, and giving the number of different types of pest targets in the image. Because a large number of simple samples and negative samples may exist in the training process, the method adopts the Focal loss method to reduce the influence of the large number of negative samples and the simple samples on the model training process, and improves the attention of the model on difficult samples so as to improve the overall identification accuracy of the model; further, regression accuracy of most samples is improved by taking the inverse of the first order norm of the regression loss gradient values as the weight of the regression loss.
The inference process of pest identification involves only a simple feed-forward process of inputting images to the detection model. To increase the detection speed, most of the prediction boxes may be first filtered with a confidence threshold of 0.01 and the top 300 prediction boxes may be retained using IoU a non-maximum suppression operation with a threshold of 0.5. Then, the prediction box with the score higher than 0.05 is selected as the final detection result.
The invention can also carry out pest picture and identification, the mobile terminal is embedded into the grain storage pest picture and supports the intelligent query and display functions of character input or voice input key words. When a user inputs a keyword or the voice of the keyword, a grain storage pest picture identification (a list consisting of characters and pictures) related to the keyword can be obtained, and the picture identification information of any pest is clicked to obtain the detailed information of the picture identification and display. The method mainly comprises the following steps: including information such as pictures, categories, morphological characteristics, life habits, major hazards, distribution conditions, and control measures of pests. The introduction information of the pests is avoided being obtained by looking up books or network resources in a manual mode, the workload is saved, and the learning efficiency is improved.
In one embodiment of the present invention, acquiring temperature and humidity information of a grain pile in a grain bin, and associating basic insect pest situation data corresponding to the temperature and humidity information of the grain pile specifically includes:
the system comprises a background program running on a remote server, a special interface program is designed, grain bulk temperature and humidity sensor array data of a specific granary can be read from a database of an independently running temperature and humidity monitoring grain condition system and stored, basic insect condition data LID of a target area of the system is associated with an average value of the temperature and humidity sensor data in a certain space of the area, and multidimensional insect condition data are obtained, namely: LID + temperature and humidity Data, and defining the multi-dimensional Data as OMDU (one Multiple Data Unit). The definition of the multidimensional data is provided by the invention, and LID data is associated with key ecological factors influencing pest occurrence, namely the relative position in a barn, temperature and humidity information and the temperature and humidity information of the environment where the barn is located, and is used as a basic data set for judging pest occurrence early warning.
The correlation method of the LID data and the target area temperature and humidity sensor data comprises the following steps: the data arrangement of the grain pile temperature and humidity sensor array is carried out according to the grain industry standard specification, a three-dimensional grid graph formed by straight lines between sensor nodes can be formed according to the standard and the total sensor number of the temperature and humidity sensor array, and each side length of the grid is calculated according to the bin type and temperature and humidity sensor arrangement graph. According to the position (shown in fig. 1) defined by the target area, a space range with length, width and height is divided by taking the position of each target area as a center point, and the size of the space range is based on the condition that the space range at least comprises one or more temperature and humidity sensor nodes. The temperature and humidity data in the OMDU is the average of the sensor data for this range. Examples of data are shown in table 2.
TABLE 2 example OMDU Multi-dimensional data
The multi-dimensional data OMDU is formed at the server backend in accordance with the method described above. The invention can continuously obtain the OMDU data through the mobile terminal and the system, and establish the OMDU data set. The data are stored in the server side and used for data visualization display of the terminal and insect condition early warning. And (3) establishing an OMDU data set, and setting and optimizing the level of the easily-grown insects in the target area by data statistics and analysis to provide data support.
The invention comprises a personal setting method, an average method and a maximum value method in the aspect of image acquisition.
Personal setting method: according to different seasons, a manager sets acquisition frequency, the number and the positions of sampling points on a server side through a system manager according to years of management experience, and pushes a notice through a display interface to urge a custodian to acquire information in time.
Average method: embedding an effective accumulated temperature generation model of pests at a server end, associating effective accumulated temperature parameters K corresponding to different types of pests and starting point temperature of pest development according to an image identification result, counting to obtain the number of target regions of the same type of pests, and calculating the T (time per week and month) of the regions in a set time length (such as multiple times per week and multiple times per month) L Inner average total accumulated temperature K T1 Then, the generation number of the pest occurrence: n is I ＝K T1 /K，n I The value of (d) is proportional to the frequency or time of acquisition; sampling time interval of T 1 Frequency of f 1 Then T is 1 ＝1/f 1 ＝T L /n I ；
The collection frequency, the number and the positions of sampling points are informed through interface pushing, a custodian is urged to carry out timely information collection, and thus the collection frequency is different for different target areas where different kinds of pests are found.
Maximum value method: embedding an effective accumulated temperature generation model of pests into a server, selecting the pest with the greatest harm in the identification result according to the image identification result, associating an effective accumulated temperature parameter K corresponding to the pests and the starting point temperature of pest development, and calculating to obtain the target region of the pests within a set time length T L Inner maximum total accumulated temperature K TM Point, the number of generations of pest occurrence: n is IM ＝K TM K, at maximum total accumulated temperature K TM Collecting information at points with sampling time interval of T 2 Frequency of f 2 Then T is 2 ＝1/f 2 ＝T L /n IM And acquiring the frequency and the number and the positions of sampling points, and prompting a custodian to acquire information in time through interface pushing notification.
Compared with a common method for manually setting an acquisition period, the method for setting the image data acquisition frequency based on the pest growth and development accumulated temperature model design is more scientific and accurate, reduces the dependence on the experience of a custodian, improves the sensitivity of monitoring the pest occurrence condition, and provides data support for implementing accurate, timely and effective prevention and control measures.
The invention comprises the following early warning method in the aspect of early warning of insect situations.
The generation number n of pest occurrence is set according to seasons I And n IM Comparing the value with the results calculated by the averaging method and the maximum method, and giving early warning information of the target area if the results calculated by the averaging method and the maximum method exceed any set value in the season; setting an alarm threshold value according to seasons, dynamically adjusting the early warning threshold value, improving robustness of early warning results to four-season changes, and reducing false alarm and delayed alarm;
or counting the pest growth number and the temperature and humidity data change amplitude of the target area by taking the set time as a basic counting time unit in the target area to obtain the maximum number of the sum of the pest growth number and the temperature and humidity data change amplitude and the corresponding target area, and giving early warning information at the equipment end; the monitoring of time dimension is increased, the temperature and humidity change of the grain stack and the change of the pest detection result in a period of time are considered, the reliability of early warning can be further improved, more accurate assessment of pest occurrence and development conditions is given, and more accurate early warning results are made;
or obtaining an image recognition result of the target area, and performing early warning by converting the image recognition result into the insect population density, wherein the insect population density calculation specifically comprises the following steps:
P d1 the number/picture area of pest identification in grain surface pest picture is multiplied by A multiplied by grain volume weight；
P d2 The insect population density is (n × P) d1 +m*P d2 ) V (n + m); a is height data, n represents the number of the grain target area shooting points, and m represents the number of the sampling insect screening target area shooting points.
And (4) giving early warning judgment of serious, general and pest-free grains by combining the calculation result of the population density according to the regulations of national or industrial standards.
In conclusion, the invention realizes multi-dimensional information acquisition and pest condition early warning of pests in grain storage in the granary, more comprehensively reflects the specific situation of pests in grain piles in the granary, and improves the accuracy of pest information acquisition and early warning and the flexibility of pest image recognition. The sensitivity of monitoring the pest occurrence condition is improved, and data support is provided for implementing accurate, timely and effective control measures. The pest situation early warning method has the advantages that the occurrence of false alarm and delayed alarm is reduced, the reliability of the early warning can be further improved, more accurate pest occurrence and development evaluation is given, and more accurate early warning results are given. Meanwhile, effective detection and accurate identification of the species of the pest target are realized, and the condition of missed detection of the pest target is reduced.
Claims (14)
1. A multi-dimensional information acquisition method for grain storage pests based on a mobile terminal is characterized by comprising the following steps:
establishing basic information of a granary, defining the name of a target area where pests appear in the granary, and setting a corresponding pest susceptibility grade;
acquiring images of a target area, wherein the images comprise acquired images of stored grain surfaces and images acquired after sampling and insect screening corresponding to different stored grain pile depths;
performing off-line identification on the image of the target area through an image data set and an image identification algorithm embedded in the mobile terminal to obtain pest species and corresponding quantity information in the image;
establishing basic pest situation data according to the basic information of the granary, the corresponding pest susceptibility grade, the obtained pest type and the corresponding quantity information;
acquiring temperature and humidity information of a grain pile in the granary, and correlating the temperature and humidity information of the grain pile with corresponding basic insect pest situation data to obtain multi-dimensional information of grain storage pests in the granary.
2. The multi-dimensional grain storage pest information collection method based on the mobile terminal of claim 1, wherein the granary basic information comprises: granary coding, granary number, granary type, grain types stored in the granary, moisture content, warehousing time and warehousing time.
3. The multi-dimensional information acquisition method for grain storage pests based on the mobile terminal as claimed in claim 2, wherein the basic pest situation data comprises: granary coding, bin type, target area, pest species and corresponding quantity, grain species and grain moisture.
4. The multi-dimensional grain storage pest information acquisition method based on the mobile terminal as claimed in claim 1, wherein the target area name comprises: the system comprises a warehouse four-angle point, a warehouse door point, an exhaust fan point, a warehouse window point, an entrance inspection door point and a middle point, wherein the warehouse four-angle point, the warehouse door point, the exhaust fan point, the warehouse window point, the entrance inspection door point and the middle point are three-dimensional target areas with different depths.
5. The multi-dimensional grain storage pest information acquisition method based on the mobile terminal as claimed in claim 1, wherein the image recognition algorithm specifically comprises:
step 1, extracting a feature map of an image by using a convolutional neural network according to a pixel value of each pixel point on the image;
step 2, constructing a characteristic pyramid based on characteristic output of different convolutional neural network levels;
step 3, generating preset target frames with corresponding sizes at different feature pyramid levels;
step 4, extracting the characteristics of the area corresponding to the preset frame, and predicting the pest species and the pest target frame;
and step 5, determining the credibility of the pest type in the image and the number of different types of pests in the image according to the predicted pest type and the pest target frame, and taking the average accuracy as an evaluation index of the pest identification performance.
6. The multi-dimensional grain storage pest information acquisition method based on the mobile terminal as claimed in claim 5, wherein in the step 1, before extracting the feature map of the image, the sliding window clipping processing is performed on the image, and the original pest image acquired by the mobile terminal is clipped to a corresponding size.
7. The multi-dimensional grain storage pest information acquisition method based on the mobile terminal as claimed in claim 5, wherein in the step 2, when the feature pyramid is constructed, a feature layer of a specific level of the basic network is selected according to the scale range of the pest target size and the sizes of the receptive fields of different levels of the basic network, and the feature pyramid is constructed through a hole convolution, a deconvolution or an attention mechanism operation of a space and a channel.
8. The multi-dimensional information acquisition method for stored grain pests based on the mobile terminal as claimed in claim 5, wherein in step 3, the step length of each feature pyramid level determines a preset target frame scale interval corresponding to the layer, the preset target frame scale is set to be n times of the scale interval, and n is an integer greater than 1.
9. The multi-dimensional grain storage pest information acquisition method based on the mobile terminal as claimed in claim 5, wherein in the step 4, the predicting of pest species and pest target frame specifically comprises:
respectively connecting a classification sub-network and a boundary frame regression sub-network after each feature pyramid level, wherein the classification sub-network predicts the occurrence probability of each pest in an anchor point generated at each spatial position and predicts the probability that pest targets appearing in a plurality of preset frames of each position unit on a detection layer belong to target categories; the bounding box regression subnetwork regression predicts the offset between each preset box matched to a pest target and the true value of the signature.
10. The multi-dimensional information acquisition method for stored grain pests based on the mobile terminal as claimed in claim 5, wherein the determining of the credibility of the pest categories in the image and the number of different kinds of pests in the image specifically comprises:
and carrying out non-maximum value inhibition operation according to the score of each bounding box, filtering repeated and low-score detection results, determining the credibility of the pest belonging to each category respectively for the pest target obtained by detection in the image, and giving the number of different types of pest targets in the image.
11. The multi-dimensional grain storage pest information acquisition method based on the mobile terminal as claimed in claim 1, wherein images are acquired according to different image acquisition frequencies, and the image acquisition method specifically comprises a personal setting method, an averaging method and a maximum value method; the personal setting method specifically includes: according to different seasons and management experience of an administrator, the acquisition frequency, the number and the positions of sampling points are set at the server end by the administrator, and image acquisition is carried out by pushing and informing through an interface.
12. The multi-dimensional grain storage pest information collection method based on the mobile terminal as claimed in claim 11, wherein the averaging method specifically comprises:
embedding an effective accumulated temperature generation model of pests into a server, associating effective accumulated temperature parameters K corresponding to different types of pests and starting point temperature of pest development according to an image recognition result, counting to obtain the number of target areas of the same type of pests, and calculating the time length T of the area in a set time L Inner average total accumulated temperature K T1 Then, the generation number of the pest occurrence: n is I ＝K T1 /K，n I The value of (d) is proportional to the frequency or time of acquisition; sampling time interval of T 1 Frequency of f 1 Then T is 1 ＝1/f 1 ＝T L /n I ；
Collecting frequency, the number and the positions of sampling points, and pushing a notice through an interface to collect images.
13. The multi-dimensional grain storage pest information collection method based on the mobile terminal as claimed in claim 11, wherein the maximum value method specifically comprises:
embedding an effective accumulated temperature generation model of pests into a server, selecting the pest with the greatest harm in the identification result according to the image identification result, associating an effective accumulated temperature parameter K corresponding to the pests and the starting point temperature of pest development, and calculating to obtain the target region of the pests within a set time length T L Inner maximum total accumulated temperature K TM Point, the number of generations of pest occurrence: n is IM ＝K TM K, at maximum total accumulated temperature K TM Collecting information at points with sampling time interval of T 2 Frequency of f 2 Then T is 2 ＝1/f 2 ＝T L /n IM And acquiring the frequency, the number and the positions of sampling points, and pushing and informing through an interface to acquire images.
14. A multi-dimensional information early warning method for grain storage pests based on a mobile terminal is characterized by comprising the following steps:
the generation number n of pest occurrence is set according to seasons I And n IM Comparing the value with the results calculated by the averaging method and the maximum method, and giving early warning information of the target area if the results calculated by the averaging method and the maximum method exceed any set value in the season;
or counting the pest growth number and the temperature and humidity data change amplitude of the target area by taking the set time as a basic counting time unit in the target area to obtain the maximum number of the sum of the pest growth number and the temperature and humidity data change amplitude and the corresponding target area, and giving early warning information at the equipment end;
or obtaining an image recognition result of the target area, and performing early warning by converting the image recognition result into the insect population density, wherein the insect population density calculation specifically comprises the following steps:
P d1 the number of identified pests in the grain surface pest picture/picture area is multiplied by A multiplied by grain volume weight;
P d2 the insect population density is (n × P) d1 +m*P d2 ) V (n + m); a is height data, n represents the number of the grain target area shooting points, and m represents the number of the sampling insect screening target area shooting points.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210723114.3A CN115100471A (en) | 2022-06-23 | 2022-06-23 | Multi-dimensional information acquisition and early warning method for grain storage pests based on mobile terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210723114.3A CN115100471A (en) | 2022-06-23 | 2022-06-23 | Multi-dimensional information acquisition and early warning method for grain storage pests based on mobile terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115100471A true CN115100471A (en) | 2022-09-23 |
Family
ID=83292788
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210723114.3A Pending CN115100471A (en) | 2022-06-23 | 2022-06-23 | Multi-dimensional information acquisition and early warning method for grain storage pests based on mobile terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115100471A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116310658A (en) * | 2023-05-17 | 2023-06-23 | 中储粮成都储藏研究院有限公司 | Method for establishing grain storage pest image data set based on spherical camera |
| CN116824200A (en) * | 2022-11-24 | 2023-09-29 | 武汉很多鱼钓具有限公司 | Forestry pest intelligent identification detection method based on target detection technology |
| CN117272028A (en) * | 2023-10-19 | 2023-12-22 | 中国铁塔股份有限公司吉林省分公司 | Insect condition monitoring method and system based on situation awareness of Internet of things |
| CN117313984A (en) * | 2023-08-31 | 2023-12-29 | 中国标准化研究院 | Grain condition monitoring method, device and system |
-
2022
- 2022-06-23 CN CN202210723114.3A patent/CN115100471A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116824200A (en) * | 2022-11-24 | 2023-09-29 | 武汉很多鱼钓具有限公司 | Forestry pest intelligent identification detection method based on target detection technology |
| CN116310658A (en) * | 2023-05-17 | 2023-06-23 | 中储粮成都储藏研究院有限公司 | Method for establishing grain storage pest image data set based on spherical camera |
| CN116310658B (en) * | 2023-05-17 | 2023-08-01 | 中储粮成都储藏研究院有限公司 | Method for establishing grain storage pest image data set based on spherical camera |
| CN117313984A (en) * | 2023-08-31 | 2023-12-29 | 中国标准化研究院 | Grain condition monitoring method, device and system |
| CN117272028A (en) * | 2023-10-19 | 2023-12-22 | 中国铁塔股份有限公司吉林省分公司 | Insect condition monitoring method and system based on situation awareness of Internet of things |
| CN117272028B (en) * | 2023-10-19 | 2024-01-26 | 中国铁塔股份有限公司吉林省分公司 | Insect condition monitoring method and system based on situation awareness of Internet of things |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115100471A (en) | Multi-dimensional information acquisition and early warning method for grain storage pests based on mobile terminal | |
| CN110826684B (en) | Convolutional neural network compression method, convolutional neural network compression device, electronic device, and medium | |
| US6819797B1 (en) | Method and apparatus for classifying and querying temporal and spatial information in video | |
| CN103283226B (en) | Produce method, video camera system and the processing system for video of the metadata associated with frame of video | |
| CN110751451B (en) | Laboratory big data management system | |
| CN109284613B (en) | Method, device, equipment and storage medium for identification detection and counterfeit site detection | |
| GB2431793A (en) | Image comparison | |
| US20100158362A1 (en) | Image processing | |
| CN109544538A (en) | Wheat scab disease grade is classified method and device | |
| Ganar et al. | Enhancement of image retrieval by using colour, texture and shape features | |
| CN115660262B (en) | Engineering intelligent quality inspection method, system and medium based on database application | |
| CN114581848A (en) | Detection method, system and device for tobacco storage pests | |
| CN108874910A (en) | The Small object identifying system of view-based access control model | |
| CN109858332A (en) | A kind of human behavior analysis method, device and electronic equipment | |
| CN115311623A (en) | Equipment oil leakage detection method and system based on infrared thermal imaging | |
| CN109993071B (en) | Method and system for automatically identifying and investigating color-changing forest based on remote sensing image | |
| CN109241316A (en) | Image search method, device, electronic equipment and storage medium | |
| CN108985346A (en) | Fusion low layer pictures feature and showing for CNN feature survey image search method | |
| CN105843930A (en) | Video search method and device | |
| CN111612734A (en) | Background clutter characterization method based on image structure complexity | |
| CN113792715B (en) | Granary pest monitoring and early warning method, device, equipment and storage medium | |
| KR101142163B1 (en) | Semantic based image retrieval method | |
| CN114972931A (en) | Goods storage method and device based on knowledge distillation | |
| Paul et al. | Change detection of linear features in temporally spaced remotely sensed images using edge-based grid analysis | |
| CN113591662A (en) | Method, system and storage medium for recognizing smoking calling behavior |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |