CN116649191A - Remote fertilization and irrigation control system and method based on PLC - Google Patents
Remote fertilization and irrigation control system and method based on PLC Download PDFInfo
- Publication number
- CN116649191A CN116649191A CN202310704018.9A CN202310704018A CN116649191A CN 116649191 A CN116649191 A CN 116649191A CN 202310704018 A CN202310704018 A CN 202310704018A CN 116649191 A CN116649191 A CN 116649191A
- Authority
- CN
- China
- Prior art keywords
- feature
- classification
- soil
- state
- plc
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003973 irrigation Methods 0.000 title claims abstract description 99
- 230000002262 irrigation Effects 0.000 title claims abstract description 99
- 238000000034 method Methods 0.000 title claims abstract description 38
- 230000004720 fertilization Effects 0.000 title claims abstract description 28
- 239000002689 soil Substances 0.000 claims abstract description 243
- 238000012544 monitoring process Methods 0.000 claims abstract description 91
- 238000012546 transfer Methods 0.000 claims abstract description 78
- 238000010586 diagram Methods 0.000 claims abstract description 68
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 63
- 239000011159 matrix material Substances 0.000 claims abstract description 49
- 238000005457 optimization Methods 0.000 claims abstract description 39
- 238000009826 distribution Methods 0.000 claims abstract description 25
- 230000007704 transition Effects 0.000 claims description 42
- 230000008859 change Effects 0.000 claims description 39
- 230000004044 response Effects 0.000 claims description 32
- 238000000605 extraction Methods 0.000 claims description 26
- 239000013598 vector Substances 0.000 claims description 26
- 230000004927 fusion Effects 0.000 claims description 24
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 23
- 238000004364 calculation method Methods 0.000 claims description 21
- 230000006870 function Effects 0.000 claims description 15
- 238000011176 pooling Methods 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 6
- 238000011156 evaluation Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 5
- 238000005516 engineering process Methods 0.000 description 8
- 238000003062 neural network model Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 238000001514 detection method Methods 0.000 description 5
- 238000005065 mining Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000003860 storage Methods 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000002411 adverse Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 2
- 238000004220 aggregation Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 238000007477 logistic regression Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000015654 memory Effects 0.000 description 2
- 230000008707 rearrangement Effects 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000012271 agricultural production Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000010792 warming Methods 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01G—HORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
- A01G25/00—Watering gardens, fields, sports grounds or the like
- A01G25/16—Control of watering
- A01G25/167—Control by humidity of the soil itself or of devices simulating soil or of the atmosphere; Soil humidity sensors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
- Y02A40/22—Improving land use; Improving water use or availability; Controlling erosion
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Soil Sciences (AREA)
- Environmental Sciences (AREA)
- Water Supply & Treatment (AREA)
- Feedback Control In General (AREA)
Abstract
A remote fertilization irrigation control system based on PLC and a method thereof are disclosed. Firstly, extracting a plurality of soil surface state monitoring key frames from a soil surface state monitoring video, then, respectively passing the soil surface state monitoring key frames through a convolutional neural network model to obtain a plurality of soil state feature matrixes, then, calculating a transfer matrix between every two adjacent soil state feature matrixes in the soil state feature matrixes to obtain a state transfer time sequence feature diagram composed of a plurality of transfer matrixes, then, passing the state transfer time sequence feature diagram through a time sequence feature extractor to obtain a classification feature diagram, then, carrying out feature distribution optimization on the classification feature diagram, and then, passing through a classifier to obtain a classification result used for indicating whether irrigation is needed, and finally, generating a PLC control instruction to a PLC based on the classification result. In this way, the growth and yield of the crop can be ensured.
Description
Technical Field
The application relates to the field of intelligent control, in particular to a remote fertilization and irrigation control system and a remote fertilization and irrigation control method based on PLC.
Background
With the increase of global warming and drought disasters, water resource management in agricultural production has become particularly important. However, the traditional irrigation decision-making mode is mainly based on experience or a time schedule, lacks of real-time monitoring of crop demands and soil moisture status, cannot adjust the water quantity in real time according to the crop growth condition, and easily causes excessive or insufficient soil moisture content, so that the crop growth is affected.
Accordingly, an intelligent remote fertigation irrigation control system is desired.
Disclosure of Invention
The present application has been made to solve the above-mentioned technical problems. The embodiment of the application provides a remote fertilization and irrigation control system and a remote fertilization and irrigation control method based on a PLC. Firstly, extracting a plurality of soil surface state monitoring key frames from a soil surface state monitoring video, then, respectively passing the soil surface state monitoring key frames through a convolutional neural network model to obtain a plurality of soil state feature matrixes, then, calculating a transfer matrix between every two adjacent soil state feature matrixes in the soil state feature matrixes to obtain a state transfer time sequence feature diagram composed of a plurality of transfer matrixes, then, passing the state transfer time sequence feature diagram through a time sequence feature extractor to obtain a classification feature diagram, then, carrying out feature distribution optimization on the classification feature diagram, and then, passing through a classifier to obtain a classification result used for indicating whether irrigation is needed, and finally, generating a PLC control instruction to a PLC based on the classification result. In this way, the growth and yield of the crop can be ensured.
According to one aspect of the present application, there is provided a PLC-based remote fertigation irrigation control system comprising:
the video acquisition module is used for acquiring a soil surface state monitoring video of a preset time period acquired by the camera;
the key frame extraction module is used for extracting a plurality of soil surface state monitoring key frames from the soil surface state monitoring video;
the soil surface state feature extraction module is used for respectively enabling the plurality of soil surface state monitoring key frames to pass through a convolutional neural network model comprising a depth feature fusion module so as to obtain a plurality of soil state feature matrixes;
the soil surface state relative change module is used for calculating a transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices to obtain a state transfer time sequence feature diagram composed of a plurality of transfer matrices;
the state time sequence change feature extraction module is used for enabling the state transition time sequence feature graph to pass through a time sequence feature extractor based on a three-dimensional convolutional neural network model to obtain a classification feature graph;
the feature optimization module is used for carrying out feature distribution optimization on the classification feature map so as to obtain an optimized classification feature map;
The irrigation evaluation module is used for enabling the optimized classification feature map to pass through a classifier to obtain classification results, wherein the classification results are used for indicating whether irrigation is needed or not; and
and the control module is used for generating a PLC control instruction to a PLC controller based on the classification result, wherein the PLC controller is used for executing the PLC control instruction to control the opening or closing of the water outlet valve.
In the remote fertilization and irrigation control system based on the PLC, the soil surface state characteristic extraction module comprises:
a depth feature extraction unit, configured to input the plurality of soil surface state monitoring key frames into the convolutional neural network model respectively to extract a plurality of shallow feature maps from a shallow layer of the convolutional neural network model and a plurality of deep feature maps from a deep layer of the convolutional neural network model;
the feature fusion unit is used for respectively cascading the shallow feature images and the deep feature images by using the depth feature fusion module of the convolutional neural network model so as to obtain a plurality of soil state feature images; and
and the pooling unit is used for pooling the soil state feature maps along the channel dimension respectively to obtain the soil state feature matrixes.
In the remote fertilization and irrigation control system based on the PLC, the soil surface state relative change module is used for:
calculating a transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices according to the following transfer matrix calculation formula to obtain a plurality of transfer matrices;
the calculation formula of the transfer matrix is as follows:
wherein M is 1 、M 2 Representing each adjacent two of the plurality of soil state feature matrices, M representing a transition matrix between each adjacent two of the plurality of soil state feature matrices,representing matrix multiplication; and
and arranging the plurality of transfer matrixes to obtain the state transfer time sequence characteristic diagram.
In the remote fertilization and irrigation control system based on the PLC, the state time sequence change feature extraction module is used for:
input data are separately processed in forward pass of the layer using the timing feature extractor:
performing three-dimensional convolution processing on the input data based on the three-dimensional convolution check to obtain a convolution characteristic diagram;
carrying out mean pooling treatment on the convolution feature map to obtain a pooled feature map; and
Non-linear activation is carried out on the pooled feature map so as to obtain an activated feature map;
the output of the last layer of the time sequence feature extractor is the classification feature diagram, and the input of the first layer of the time sequence feature extractor is the state transition time sequence feature diagram.
In the remote fertilization and irrigation control system based on the PLC, the characteristic optimization module comprises:
an optimization factor calculation unit for calculating a position information schema attention response factor of each position feature value in the classification feature map to obtain a plurality of position information schema attention response factors; and
and the weighted optimization unit is used for weighted optimization of the position characteristic values of the classification characteristic map by taking the plurality of position information schema attention response factors as weighting coefficients so as to obtain the optimized classification characteristic map.
In the remote fertilization and irrigation control system based on the PLC, the optimization factor calculation unit is used for:
calculating location information schema attention response factors for each location feature value in the classification feature map with the following optimization formula to obtain the plurality of location information schema attention response factors;
wherein, the optimization formula is:
Wherein f i Is the value of each position characteristic in the classification characteristic diagram, (x) i ,y i ,z i ) Position coordinates for respective position feature values of the classification feature map, andis the global mean of all feature values of the classification feature map,/for>Andrepresent the functions of mapping three-dimensional real numbers and two-dimensional real numbers into one-dimensional real numbers, W, H and C are the width, the height and the channel number of the classification characteristic diagram respectively, log represents a logarithmic function based on 2, w i Representing individual ones of the plurality of location information schema attention response factors.
In the remote fertilization and irrigation control system based on the PLC, the irrigation evaluation module comprises:
the feature map unfolding unit is used for unfolding the optimized classification feature map into an optimized classification feature vector according to a row vector or a column vector;
the full-connection coding unit is used for carrying out full-connection coding on the optimized classification feature vector by using a plurality of full-connection layers of the classifier so as to obtain a coding classification feature vector; and
and the classification unit is used for passing the coding classification feature vector through a Softmax classification function of the classifier to obtain the classification result.
According to another aspect of the present application, there is provided a PLC-based remote fertilization irrigation control method comprising:
Acquiring a soil surface state monitoring video of a preset time period acquired by a camera;
extracting a plurality of soil surface state monitoring key frames from the soil surface state monitoring video;
the soil surface state monitoring key frames are respectively processed through a convolutional neural network model comprising a depth feature fusion module to obtain a plurality of soil state feature matrixes;
calculating a transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices to obtain a state transfer time sequence feature diagram composed of a plurality of transfer matrices;
the state transition time sequence feature diagram is passed through a time sequence feature extractor based on a three-dimensional convolutional neural network model to obtain a classification feature diagram;
performing feature distribution optimization on the classification feature map to obtain an optimized classification feature map;
the optimized classification characteristic diagram is passed through a classifier to obtain a classification result, and the classification result is used for indicating whether irrigation is needed; and
and generating a PLC control instruction to a PLC controller based on the classification result, wherein the PLC controller is used for executing the PLC control instruction to control the opening or closing of the water outlet valve.
In the above-mentioned remote fertilization irrigation control method based on PLC, the step of obtaining a plurality of soil state feature matrices by passing the plurality of soil surface state monitoring key frames through a convolutional neural network model including a depth feature fusion module, respectively, includes:
Respectively inputting the plurality of soil surface state monitoring key frames into the convolutional neural network model to extract a plurality of shallow feature maps from the shallow layer of the convolutional neural network model and a plurality of deep feature maps from the deep layer of the convolutional neural network model;
the depth feature fusion module of the convolutional neural network model is used for respectively cascading the shallow feature images and the deep feature images to obtain a plurality of soil state feature images; and
and respectively carrying out pooling treatment on the soil state feature maps along the channel dimension to obtain the soil state feature matrixes.
In the above-mentioned remote fertilization irrigation control method based on PLC, calculating a transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices to obtain a state transfer timing sequence feature diagram composed of a plurality of transfer matrices, including:
calculating a transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices according to the following transfer matrix calculation formula to obtain a plurality of transfer matrices;
the calculation formula of the transfer matrix is as follows:
Wherein M is 1 、M 2 Representing each adjacent two of the plurality of soil state feature matrices, M representing a transition matrix between each adjacent two of the plurality of soil state feature matrices,representing matrix multiplication; and
and arranging the plurality of transfer matrixes to obtain the state transfer time sequence characteristic diagram.
Compared with the prior art, the remote fertilization irrigation control system and the method based on the PLC are characterized in that firstly, a plurality of soil surface state monitoring key frames are extracted from a soil surface state monitoring video, then, the soil surface state monitoring key frames are respectively processed through a convolutional neural network model to obtain a plurality of soil state feature matrixes, then, a transfer matrix between every two adjacent soil state feature matrixes in the soil state feature matrixes is calculated to obtain a state transfer time sequence feature diagram composed of a plurality of transfer matrixes, then, the state transfer time sequence feature diagram is processed through a time sequence feature extractor to obtain a classification feature diagram, then, feature distribution optimization is carried out on the classification feature diagram, and then, classification results used for indicating whether irrigation is needed or not are obtained through a classifier, and finally, a PLC control instruction is generated to a PLC controller based on the classification results. In this way, the growth and yield of the crop can be ensured.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort to a person of ordinary skill in the art. The following drawings are not intended to be drawn to scale, emphasis instead being placed upon illustrating the principles of the application.
Fig. 1 is an application scenario diagram of a PLC-based remote fertigation irrigation control system according to an embodiment of the present application.
Fig. 2 is a block diagram schematic of a PLC-based remote fertigation irrigation control system according to an embodiment of the present application.
Fig. 3 is a block diagram schematic of the soil surface state feature extraction module in a PLC-based remote fertigation irrigation control system according to an embodiment of the present application.
Fig. 4 is a block diagram schematic of the feature optimization module in the PLC-based remote fertigation irrigation control system according to an embodiment of the present application.
FIG. 5 is a block diagram schematic of the irrigation assessment module in a PLC-based remote fertigation irrigation control system according to an embodiment of the present application.
Fig. 6 is a flow chart of a PLC-based remote fertigation irrigation control method according to an embodiment of the present application.
Fig. 7 is a schematic diagram of a system architecture of a PLC-based remote fertigation irrigation control method according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are also within the scope of the application.
As used in the specification and in the claims, the terms "a," "an," "the," and/or "the" are not specific to a singular, but may include a plurality, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that the steps and elements are explicitly identified, and they do not constitute an exclusive list, as other steps or elements may be included in a method or apparatus.
Although the present application makes various references to certain modules in a system according to embodiments of the present application, any number of different modules may be used and run on a user terminal and/or server. The modules are merely illustrative, and different aspects of the systems and methods may use different modules.
A flowchart is used in the present application to describe the operations performed by a system according to embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in order precisely. Rather, the various steps may be processed in reverse order or simultaneously, as desired. Also, other operations may be added to or removed from these processes.
Hereinafter, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.
As described above, the traditional irrigation decision-making mode is mainly based on experience or a time schedule, lacks of real-time monitoring of crop demands and soil moisture status, cannot adjust the water quantity in real time according to the crop growth condition, and easily causes excessive or insufficient soil moisture content, thereby affecting the crop growth. Accordingly, an intelligent remote fertigation irrigation control system is desired.
It should be understood that with the rapid development of computer technology in recent years, the programmable controller (Programmable Logical Control, PLC) uses a microprocessor as a core technology and a base, and effectively combines industrial automatic control technology microcomputer technology and communication network technology, so that the programmable controller is successfully applied to various fields of farm irrigation, food processing, material processing, industrial control processes and the like. Therefore, on the premise of rapid development of network technology, by taking PLC as key hardware equipment, it becomes possible to design a set of fertigation control system which is easy to operate and has strong practicability, and the problems in the existing traditional agricultural fertigation automation technology can be solved.
Based on the above, in the technical scheme of the application, whether watering is needed or not is expected to be judged based on the soil surface state, so that the watering amount is controlled to meet the water demand of crops, and various mold diseases of the crops caused by excessive water content of the soil or excessive air humidity are avoided. The method can detect the change condition of the surface state of the soil by analyzing the monitoring video of the surface state of the soil, thereby realizing self-adaptive irrigation control. However, since the soil surface state monitoring video has a large amount of information and a large amount of noise interference, the state characteristic information about the soil surface is hidden characteristic information with a small scale in the actual monitoring process, and it is difficult to sufficiently capture the state characteristic information. Therefore, in the process, the difficulty is how to excavate the time sequence dynamic characteristic distribution information about the soil surface state in the soil surface state monitoring video, so that the PLC is utilized to carry out real-time accurate control of remote irrigation, and the water demand of crops is met, meanwhile, various mold of the crops caused by excessive water content of the soil or excessive air humidity is avoided, and the growth and the yield of the crops are further ensured.
In recent years, deep learning and neural networks have been widely used in the fields of computer vision, natural language processing, text signal processing, and the like. The development of deep learning and neural networks provides a new solution idea and scheme for mining time sequence dynamic characteristic distribution information about soil surface states in the soil surface state monitoring video.
Specifically, in the technical scheme of the application, firstly, a soil surface state monitoring video of a preset time period is acquired through a camera. Next, it is considered that in the soil surface state monitoring video, the state change characteristic with respect to the soil surface may be represented by a difference between adjacent monitoring frames in the soil surface state monitoring video, that is, a time-series change condition of the soil surface state is represented by image characterization of adjacent image frames. However, in consideration of the small difference of adjacent frames in the soil surface state monitoring video, there is a large amount of data redundancy, and therefore, in order to reduce the amount of calculation and avoid adverse effects of data redundancy on detection, the soil surface state monitoring video is key frame sampled at a predetermined sampling frequency to extract a plurality of soil surface state monitoring key frames from the soil surface state monitoring video.
Then, feature mining of the plurality of soil surface state monitoring key frames is performed using a convolutional neural network model having excellent performance in terms of implicit feature extraction of images, and in particular, in order to enable sufficient expression of the soil surface state features to perform irrigation control more accurately in consideration of extracting hidden features of the respective soil surface state monitoring key frames, shallow features such as color and texture of the soil surface state, which have significance for monitoring of the soil surface state, should be focused more. While convolutional neural networks are coded, as their depth deepens, shallow features become blurred and even buried in noise. Therefore, in the technical scheme of the application, the convolution neural network model comprising the depth feature fusion module is used for processing the plurality of soil surface state monitoring key frames to obtain a plurality of soil state feature matrixes. Compared with a standard convolutional neural network model, the convolutional neural network model disclosed by the application can retain the shallow layer characteristics and the deep layer characteristics of the soil surface state, so that the characteristic information is more abundant, and the characteristics of different depths can be retained, so that the accuracy of the soil surface state detection is improved.
Further, considering that the state of the soil surface has a dynamic change rule in the time dimension, the change rule is weak, namely, the change rule is small-scale implicit change characteristic information, and the change rule is difficult to sufficiently capture through a traditional characteristic extraction mode. Therefore, in the technical scheme of the application, the transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices is calculated to extract the relative difference feature information about the soil surface state under every two adjacent key frames, so as to obtain a state transfer time sequence feature diagram consisting of the plurality of transfer matrices.
Then, since the state of the soil surface has a dynamic change rule in the time dimension, the relative difference characteristic information about the state of the soil surface under every two adjacent key frames also has a time-sequential dynamic change rule in the time dimension. That is, the relative change characteristics of the soil surface state have an association relationship in time series. Therefore, in the technical scheme of the application, the state transition time sequence feature map is further subjected to feature mining in a time sequence feature extractor based on a three-dimensional convolutional neural network model so as to extract time sequence dynamic associated feature information of the relative change feature of the soil surface state in the time dimension, thereby obtaining a classification feature map. In particular, the convolution kernel of the three-dimensional convolution neural network model of the time sequence feature extractor is a three-dimensional convolution kernel, which has W (width), H (height) and C (channel dimension), and in the technical solution of the present application, the channel dimension of the three-dimensional convolution kernel corresponds to the time dimension of the state transition time sequence feature map, so that dynamic change feature information of the relative feature of the soil surface state distribution along the time dimension can be extracted when three-dimensional convolution encoding is performed.
And then, classifying the classifying feature map by a classifier to obtain a classifying result used for indicating whether irrigation is needed. That is, in the technical solution of the present application, the labels of the classifier include a need for irrigation (first label) and no need for irrigation (second label), wherein the classifier determines to which classification label the classification feature map belongs through a soft maximum function. It should be noted that the first tag p1 and the second tag p2 do not include a human-set concept, and in fact, during the training process, the computer model does not have a concept of "whether irrigation is needed", which is only two kinds of classification tags, and the probability that the output feature is under the two classification tags, that is, the sum of p1 and p2 is one. Therefore, the classification result of whether irrigation is needed is actually converted into the classification probability distribution conforming to the natural rule through classifying the labels, and the physical meaning of the natural probability distribution of the labels is essentially used instead of the language text meaning of whether irrigation is needed. It should be understood that, in the technical solution of the present application, the classification label of the classifier is a detection and evaluation label for whether irrigation is needed, so after the classification result is obtained, a PLC control instruction may be generated to a PLC controller based on the classification result, where the PLC controller is configured to execute the PLC control instruction to control opening or closing of the water outlet valve, so as to perform real-time control of remote irrigation.
In particular, in the technical scheme of the application, based on the image space characteristics of each pixel of the plurality of soil surface state monitoring key frames extracted from the soil surface state monitoring video, when the plurality of soil state feature matrices are obtained through a convolutional neural network model comprising a depth feature fusion module, feature values of each position of the plurality of soil state feature matrices have corresponding position attributes, and when a transition matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices is calculated and the state transition time sequence feature map is obtained through a time sequence feature extractor based on a three-dimensional convolutional neural network model, the feature values of each position of the classification feature map also have corresponding position attributes due to the position correlation of domain transition feature calculation and three-dimensional time sequence feature extraction. However, when the classification feature map is passed through a classifier, the classification feature map needs to be expanded into feature vectors, that is, a rearrangement transformation based on position attributes involving feature values of the classification feature map, so in order to promote the feature position information expression effect of each feature value of the classification feature map at the time of arrangement transformation, a position information schema attention response factor of the feature value of each position of the classification feature map is calculated, specifically expressed as:
And->Representing the mapping of three-dimensional and two-dimensional real numbers to one-dimensional real numbers, respectively, for example, a weighted and biased representation may be activated by a nonlinear activation function, W, H and C being the width, height and channel number, respectively, of the classification characteristic diagram, (x) i ,y i ,z i ) For each feature value f of the classification feature map i Position coordinates of (2), and->Is the global average of all feature values of the classification feature map.
Here, the location information schema attention response factor is represented by schema information modeling relative geometric directions and relative geometric distances of respective feature values of the classification feature map with respect to high-dimensional spatial locations of the overall feature distribution, capturing overall shape weights of feature manifolds of the high-dimensional feature distribution while achieving location-wise aggregation of the feature values with respect to the overall feature distribution, such that manifold shapes of the feature map are highly responsive to shape information of the respective sub-manifolds to obtain an arrangement invariance (permutation invariance) property of the high-dimensional feature manifolds of the feature map. In this way, the position information expression effect of the image feature semantics of each feature value of the classification feature map in the arrangement transformation can be improved by weighting each feature value of the classification feature map by the position information schema attention response factor, so that the accuracy of the classification result of the classification feature map obtained by the classifier is improved. Like this, can utilize the PLC controller to carry out the real-time accurate control of long-range watering to when satisfying the demand of crop to water, can not cause soil water content too much again or arouse the crop to take place various moulds because of air humidity is too big, and then guarantee the growth and the output of crop.
Fig. 1 is an application scenario diagram of a PLC-based remote fertigation irrigation control system according to an embodiment of the present application. As shown in fig. 1, in this application scenario, first, a soil surface state monitoring video (e.g., D illustrated in fig. 1) acquired by a camera (e.g., C illustrated in fig. 1) for a predetermined period of time is acquired, then, the soil surface state monitoring video is input into a server (e.g., S illustrated in fig. 1) where a PLC-based remote fertigation irrigation control algorithm is deployed, wherein the server can process the soil surface state monitoring video using the PLC-based remote fertigation irrigation control algorithm to obtain a classification result for indicating whether irrigation is required, and finally, a PLC control instruction is generated to a PLC controller based on the classification result.
Having described the basic principles of the present application, various non-limiting embodiments of the present application will now be described in detail with reference to the accompanying drawings.
Fig. 2 is a block diagram schematic of a PLC-based remote fertigation irrigation control system according to an embodiment of the present application. As shown in fig. 2, a PLC-based remote fertigation irrigation control system 100 according to an embodiment of the present application includes: the video acquisition module 110 is used for acquiring a soil surface state monitoring video of a preset time period acquired by the camera; a key frame extracting module 120, configured to extract a plurality of soil surface state monitoring key frames from the soil surface state monitoring video; the soil surface state feature extraction module 130 is configured to obtain a plurality of soil state feature matrices by passing the plurality of soil surface state monitoring key frames through a convolutional neural network model including a depth feature fusion module; the soil surface state relative change module 140 is configured to calculate a transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices to obtain a state transfer timing sequence feature diagram composed of a plurality of transfer matrices; a state time sequence change feature extraction module 150, configured to pass the state transition time sequence feature map through a time sequence feature extractor based on a three-dimensional convolutional neural network model to obtain a classification feature map; the feature optimization module 160 is configured to perform feature distribution optimization on the classification feature map to obtain an optimized classification feature map; the irrigation evaluation module 170 is configured to pass the optimized classification feature map through a classifier to obtain a classification result, where the classification result is used to indicate whether irrigation is needed; and a control module 180, configured to generate a PLC control instruction to a PLC controller based on the classification result, where the PLC controller is configured to execute the PLC control instruction to control opening or closing of the water outlet valve.
More specifically, in the embodiment of the present application, the video acquisition module 110 is configured to acquire a soil surface state monitoring video acquired by the camera for a predetermined period of time. Whether watering is needed or not is judged based on the soil surface state, so that the watering amount is controlled, the water demand of crops can be met, and various mold diseases of the crops caused by excessive water content of the soil or excessive air humidity are avoided. The method can detect the change condition of the surface state of the soil by analyzing the monitoring video of the surface state of the soil, thereby realizing self-adaptive irrigation control.
More specifically, in an embodiment of the present application, the key frame extracting module 120 is configured to extract a plurality of soil surface state monitoring key frames from the soil surface state monitoring video. In the soil surface state monitoring video, the state change characteristic about the soil surface can be represented by the difference between adjacent monitoring frames in the soil surface state monitoring video, that is, the time-series change condition of the soil surface state is represented by the image representation of the adjacent image frames. However, in consideration of the small difference of adjacent frames in the soil surface state monitoring video, there is a large amount of data redundancy, and therefore, in order to reduce the amount of calculation and avoid adverse effects of data redundancy on detection, the soil surface state monitoring video is key frame sampled at a predetermined sampling frequency to extract a plurality of soil surface state monitoring key frames from the soil surface state monitoring video.
More specifically, in the embodiment of the present application, the soil surface state feature extraction module 130 is configured to obtain a plurality of soil state feature matrices by passing the plurality of soil surface state monitoring key frames through a convolutional neural network model including a depth feature fusion module. Feature mining of the plurality of soil surface state monitoring key frames is performed using a convolutional neural network model having excellent performance in terms of implicit feature extraction of images, and in particular, in order to enable sufficient expression of the soil surface state features to perform irrigation control more accurately in consideration of extracting the hidden features of the respective soil surface state monitoring key frames, shallow features such as colors and textures of the soil surface states, which have important significance for monitoring the soil surface states, should be focused on. While convolutional neural networks are coded, as their depth deepens, shallow features become blurred and even buried in noise. Therefore, in the technical scheme of the application, the convolution neural network model comprising the depth feature fusion module is used for processing the plurality of soil surface state monitoring key frames to obtain a plurality of soil state feature matrixes. Compared with a standard convolutional neural network model, the convolutional neural network model disclosed by the application can retain the shallow layer characteristics and the deep layer characteristics of the soil surface state, so that the characteristic information is more abundant, and the characteristics of different depths can be retained, so that the accuracy of the soil surface state detection is improved.
It should be appreciated that convolutional neural network (Convolutional Neural Network, CNN) is an artificial neural network and has wide application in the fields of image recognition and the like. The convolutional neural network may include an input layer, a hidden layer, and an output layer, where the hidden layer may include a convolutional layer, a pooling layer, an activation layer, a full connection layer, etc., where the previous layer performs a corresponding operation according to input data, outputs an operation result to the next layer, and obtains a final result after the input initial data is subjected to a multi-layer operation.
Accordingly, in one specific example, as shown in fig. 3, the soil surface state feature extraction module 130 includes: a depth feature extraction unit 131, configured to input the plurality of soil surface state monitoring key frames into the convolutional neural network model respectively to extract a plurality of shallow feature maps from a shallow layer of the convolutional neural network model and a plurality of deep feature maps from a deep layer of the convolutional neural network model; the feature fusion unit 132 is configured to respectively concatenate the plurality of shallow feature maps and the plurality of deep feature maps using the depth feature fusion module of the convolutional neural network model to obtain a plurality of soil state feature maps; and a pooling unit 133, configured to pool the plurality of soil state feature maps along the channel dimension to obtain the plurality of soil state feature matrices.
More specifically, in the embodiment of the present application, the relative change module 140 of soil surface state is configured to calculate a transition matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices to obtain a state transition timing feature map composed of a plurality of transition matrices. The state of the soil surface has a dynamic change rule in the time dimension, so that the change rule is weak, namely, the change rule is small-scale implicit change characteristic information, and the condition is difficult to sufficiently capture by a traditional characteristic extraction mode. Therefore, in the technical scheme of the application, the transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices is calculated to extract the relative difference feature information about the soil surface state under every two adjacent key frames, so as to obtain a state transfer time sequence feature diagram consisting of the plurality of transfer matrices.
Accordingly, in one specific example, the soil surface state relative change module 140 is configured to: calculating a transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices according to the following transfer matrix calculation formula to obtain a plurality of transfer matrices; the calculation formula of the transfer matrix is as follows:
Wherein M is 1 、M 2 Representing each adjacent two of the plurality of soil state feature matrices, M representing a transition matrix between each adjacent two of the plurality of soil state feature matrices,representing matrix multiplication; and arranging the plurality of transfer matrixes to obtain the state transfer timing characteristic diagram.
More specifically, in an embodiment of the present application, the state timing change feature extraction module 150 is configured to pass the state transition timing feature map through a timing feature extractor based on a three-dimensional convolutional neural network model to obtain a classification feature map. Since the state of the soil surface has a dynamic change rule in the time dimension, the relative difference characteristic information about the state of the soil surface under every two adjacent key frames also has a time sequence dynamic change rule in the time dimension. That is, the relative change characteristics of the soil surface state have an association relationship in time series. Therefore, in the technical scheme of the application, the state transition time sequence feature map is further subjected to feature mining in a time sequence feature extractor based on a three-dimensional convolutional neural network model so as to extract time sequence dynamic associated feature information of the relative change feature of the soil surface state in the time dimension, thereby obtaining a classification feature map.
In particular, the convolution kernel of the three-dimensional convolution neural network model of the time sequence feature extractor is a three-dimensional convolution kernel, which has W (width), H (height) and C (channel dimension), and in the technical solution of the present application, the channel dimension of the three-dimensional convolution kernel corresponds to the time dimension of the state transition time sequence feature map, so that dynamic change feature information of the relative feature of the soil surface state distribution along the time dimension can be extracted when three-dimensional convolution encoding is performed.
Accordingly, in one specific example, the state timing change feature extraction module 150 is configured to: input data are separately processed in forward pass of the layer using the timing feature extractor: performing three-dimensional convolution processing on the input data based on the three-dimensional convolution check to obtain a convolution characteristic diagram; carrying out mean pooling treatment on the convolution feature map to obtain a pooled feature map; performing nonlinear activation on the pooled feature map to obtain an activated feature map; the output of the last layer of the time sequence feature extractor is the classification feature diagram, and the input of the first layer of the time sequence feature extractor is the state transition time sequence feature diagram.
More specifically, in the embodiment of the present application, the feature optimization module 160 is configured to perform feature distribution optimization on the classification feature map to obtain an optimized classification feature map.
Accordingly, in one specific example, as shown in fig. 4, the feature optimization module 160 includes: an optimization factor calculation unit 161 for calculating a positional information schema attention response factor for each positional feature value in the classification feature map to obtain a plurality of positional information schema attention response factors; and a weighted optimization unit 162 for weighting and optimizing each position feature value of the classification feature map with the plurality of position information map attention response factors as weighting coefficients to obtain the optimized classification feature map.
In the technical scheme of the application, based on the image space characteristics of each pixel of the plurality of soil surface state monitoring key frames extracted from the soil surface state monitoring video, when the plurality of soil state feature matrixes are obtained through a convolution neural network model comprising a depth feature fusion module, feature values of each position of the plurality of soil state feature matrixes have corresponding position attributes, and when a transition matrix between every two adjacent soil state feature matrixes in the plurality of soil state feature matrixes is calculated and the state transition time sequence feature diagram is obtained through a time sequence feature extractor based on a three-dimensional convolution neural network model, the feature values of each position of the classification feature diagram also have corresponding position attributes due to the position correlation of domain transition feature calculation and three-dimensional time sequence feature extraction. However, when the classification feature map is passed through a classifier, it is necessary to develop the classification feature map as a feature vector, that is, a rearrangement conversion based on position attributes involving feature values of the classification feature map, and thus in order to promote the feature position information expression effect of each feature value of the classification feature map at the time of arrangement conversion, a position information map attention response factor of the feature value of each position of the classification feature map is calculated.
Accordingly, in a specific example, the optimization factor calculating unit 161 is configured to: calculating location information schema attention response factors for each location feature value in the classification feature map with the following optimization formula to obtain the plurality of location information schema attention response factors; wherein, the optimization formula is:
wherein f i Is the value of each position characteristic in the classification characteristic diagram, (x) i ,y i ,z i ) Position coordinates for respective position feature values of the classification feature map, andis the global mean of all feature values of the classification feature map,/for>Andrepresent the functions of mapping three-dimensional real numbers and two-dimensional real numbers into one-dimensional real numbers, W, H and C are the width, the height and the channel number of the classification characteristic diagram respectively, log represents a logarithmic function based on 2, w i Representing individual ones of the plurality of location information schema attention response factors.
Here, the position information schema attention response factor is represented by schema information modeling relative geometric directions and relative geometric distances of respective feature values of the classification feature map with respect to high-dimensional spatial positions of the overall feature distribution, capturing overall shape weights of feature manifolds of the high-dimensional feature distribution while achieving position-wise aggregation of the feature values with respect to the overall feature distribution, so that manifold shapes of the feature map are highly responsive to shape information of the respective sub-manifolds to obtain an arrangement invariance property of the high-dimensional feature manifolds of the feature map. In this way, the position information expression effect of the image feature semantics of each feature value of the classification feature map in the arrangement transformation can be improved by weighting each feature value of the classification feature map by the position information schema attention response factor, so that the accuracy of the classification result of the classification feature map obtained by the classifier is improved. Like this, can utilize the PLC controller to carry out the real-time accurate control of long-range watering to when satisfying the demand of crop to water, can not cause soil water content too much again or arouse the crop to take place various moulds because of air humidity is too big, and then guarantee the growth and the output of crop.
More specifically, in the embodiment of the present application, the irrigation evaluation module 170 is configured to pass the optimized classification feature map through a classifier to obtain a classification result, where the classification result is used to indicate whether irrigation is needed. After the classification result is obtained, a PLC control instruction can be generated to a PLC controller based on the classification result, wherein the PLC controller is used for executing the PLC control instruction to control the opening or closing of a water outlet valve so as to perform real-time control of remote irrigation.
It should be appreciated that the role of the classifier is to learn the classification rules and classifier using a given class, known training data, and then classify (or predict) the unknown data. Logistic regression (logistics), SVM, etc. are commonly used to solve the classification problem, and for multi-classification problems (multi-class classification), logistic regression or SVM can be used as well, but multiple bi-classifications are required to compose multiple classifications, but this is error-prone and inefficient, and the commonly used multi-classification method is the Softmax classification function.
Accordingly, in one specific example, as shown in fig. 5, the irrigation assessment module 170 includes: a feature map expansion unit 171 for expanding the optimized classification feature map into optimized classification feature vectors according to row vectors or column vectors; a full-connection encoding unit 172, configured to perform full-connection encoding on the optimized classification feature vector by using multiple full-connection layers of the classifier to obtain an encoded classification feature vector; and a classification unit 173, configured to pass the encoded classification feature vector through a Softmax classification function of the classifier to obtain the classification result.
More specifically, in the embodiment of the present application, the control module 180 is configured to generate a PLC control instruction to a PLC controller based on the classification result, where the PLC controller is configured to execute the PLC control instruction to control opening or closing of the water outlet valve.
In summary, the PLC-based remote fertilization irrigation control system 100 according to the embodiment of the present application is illustrated, firstly, a plurality of soil surface state monitoring key frames are extracted from a soil surface state monitoring video, then, the plurality of soil surface state monitoring key frames are respectively passed through a convolutional neural network model to obtain a plurality of soil state feature matrices, then, a transition matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices is calculated to obtain a state transition time sequence feature map composed of a plurality of transition matrices, then, the state transition time sequence feature map is passed through a time sequence feature extractor to obtain a classification feature map, then, after feature distribution optimization is performed on the classification feature map, the classification feature map is passed through a classifier to obtain a classification result for indicating whether irrigation is required, and finally, a PLC control instruction is generated to a PLC controller based on the classification result. In this way, the growth and yield of the crop can be ensured.
As described above, the PLC-based remote fertigation and irrigation control system 100 according to the embodiment of the present application may be implemented in various terminal devices, for example, a server or the like having the PLC-based remote fertigation and irrigation control algorithm according to the embodiment of the present application. In one example, the PLC-based remote fertigation irrigation control system 100 according to embodiments of the present application may be integrated into the terminal device as a software module and/or hardware module. For example, the PLC-based remote fertigation irrigation control system 100 according to embodiments of the present application may be a software module in the operating system of the terminal device, or may be an application developed for the terminal device; of course, the PLC-based remote fertigation irrigation control system 100 according to embodiments of the present application may also be one of numerous hardware modules of the terminal device.
Alternatively, in another example, the PLC-based remote fertigation control system 100 and the terminal device according to the embodiment of the present application may be separate devices, and the PLC-based remote fertigation control system 100 may be connected to the terminal device through a wired and/or wireless network and transmit the interactive information in a agreed data format.
Fig. 6 is a flow chart of a PLC-based remote fertigation irrigation control method according to an embodiment of the present application. As shown in fig. 6, the PLC-based remote fertilization and irrigation control method according to an embodiment of the present application includes: s110, acquiring a soil surface state monitoring video of a preset time period acquired by a camera; s120, extracting a plurality of soil surface state monitoring key frames from the soil surface state monitoring video; s130, the soil surface state monitoring key frames are respectively processed through a convolutional neural network model comprising a depth feature fusion module to obtain a plurality of soil state feature matrixes; s140, calculating a transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices to obtain a state transfer time sequence feature diagram composed of the plurality of transfer matrices; s150, the state transition time sequence feature map passes through a time sequence feature extractor based on a three-dimensional convolutional neural network model to obtain a classification feature map; s160, performing feature distribution optimization on the classification feature map to obtain an optimized classification feature map; s170, the optimized classification characteristic diagram is passed through a classifier to obtain a classification result, and the classification result is used for indicating whether irrigation is needed; and S180, generating a PLC control instruction to a PLC controller based on the classification result, wherein the PLC controller is used for executing the PLC control instruction to control the opening or closing of the water outlet valve.
Fig. 7 is a schematic diagram of a system architecture of a PLC-based remote fertigation irrigation control method according to an embodiment of the present application. As shown in fig. 7, in the system architecture of the PLC-based remote fertilization irrigation control method, first, a soil surface state monitoring video of a predetermined period of time acquired by a camera is acquired; then, extracting a plurality of soil surface state monitoring key frames from the soil surface state monitoring video; then, the plurality of soil surface state monitoring key frames are respectively passed through a convolutional neural network model comprising a depth feature fusion module to obtain a plurality of soil state feature matrixes; then, calculating a transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices to obtain a state transfer time sequence feature diagram composed of a plurality of transfer matrices; then, the state transition time sequence feature map passes through a time sequence feature extractor based on a three-dimensional convolution neural network model to obtain a classification feature map; then, carrying out feature distribution optimization on the classification feature map to obtain an optimized classification feature map; then, the optimized classification characteristic diagram is passed through a classifier to obtain a classification result, wherein the classification result is used for indicating whether irrigation is needed; and finally, generating a PLC control instruction to a PLC controller based on the classification result, wherein the PLC controller is used for executing the PLC control instruction to control the opening or closing of the water outlet valve.
In a specific example, in the PLC-based remote fertilization irrigation control method, the step of passing the plurality of soil surface state monitoring key frames through a convolutional neural network model including a depth feature fusion module to obtain a plurality of soil state feature matrices includes: respectively inputting the plurality of soil surface state monitoring key frames into the convolutional neural network model to extract a plurality of shallow feature maps from the shallow layer of the convolutional neural network model and a plurality of deep feature maps from the deep layer of the convolutional neural network model; the depth feature fusion module of the convolutional neural network model is used for respectively cascading the shallow feature images and the deep feature images to obtain a plurality of soil state feature images; and respectively carrying out pooling treatment on the soil state feature maps along the channel dimension to obtain the soil state feature matrixes.
In a specific example, in the PLC-based remote fertilization and irrigation control method, calculating a transition matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices to obtain a state transition timing feature map composed of a plurality of transition matrices, includes: calculating a transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices according to the following transfer matrix calculation formula to obtain a plurality of transfer matrices; the calculation formula of the transfer matrix is as follows:
Wherein M is 1 、M 2 Representing each adjacent two of the plurality of soil state feature matrices, M representing a transition matrix between each adjacent two of the plurality of soil state feature matrices,representing matrix multiplication; and arranging the plurality of transfer matrixes to obtain the state transfer timing characteristic diagram.
In a specific example, in the PLC-based remote fertilization irrigation control method, the step of passing the state transition timing characteristic map through a timing characteristic extractor based on a three-dimensional convolutional neural network model to obtain a classification characteristic map includes: input data are separately processed in forward pass of the layer using the timing feature extractor: performing three-dimensional convolution processing on the input data based on the three-dimensional convolution check to obtain a convolution characteristic diagram; carrying out mean pooling treatment on the convolution feature map to obtain a pooled feature map; performing nonlinear activation on the pooled feature map to obtain an activated feature map; the output of the last layer of the time sequence feature extractor is the classification feature diagram, and the input of the first layer of the time sequence feature extractor is the state transition time sequence feature diagram.
In a specific example, in the PLC-based remote fertilization and irrigation control method, the optimizing the feature distribution of the classification feature map to obtain an optimized classification feature map includes: calculating a position information schema attention response factor of each position feature value in the classification feature map to obtain a plurality of position information schema attention response factors; and weighting and optimizing each position characteristic value of the classification characteristic map by taking the plurality of position information schema attention response factors as weighting coefficients to obtain the optimized classification characteristic map.
In a specific example, in the PLC-based remote fertilization irrigation control method, calculating the location information schema attention response factors of the respective location feature values in the classification feature map to obtain a plurality of location information schema attention response factors includes: calculating location information schema attention response factors for each location feature value in the classification feature map with the following optimization formula to obtain the plurality of location information schema attention response factors; wherein, the optimization formula is:
wherein f i Is the value of each position characteristic in the classification characteristic diagram, (x) i ,y i ,z i ) Position coordinates for respective position feature values of the classification feature map, andis the global mean of all feature values of the classification feature map,/for>And->Represent the functions of mapping three-dimensional real numbers and two-dimensional real numbers into one-dimensional real numbers, W, H and C are the width, the height and the channel number of the classification characteristic diagram respectively, log represents a logarithmic function based on 2, w i Representing individual ones of the plurality of location information schema attention response factors.
In a specific example, in the PLC-based remote fertilization and irrigation control method, the optimizing classification feature map is passed through a classifier to obtain a classification result, where the classification result is used to indicate whether irrigation is needed, and the method includes: expanding the optimized classification feature map into an optimized classification feature vector according to a row vector or a column vector; performing full-connection coding on the optimized classification feature vector by using a plurality of full-connection layers of the classifier to obtain a coding classification feature vector; and passing the coding classification feature vector through a Softmax classification function of the classifier to obtain the classification result.
Here, it will be understood by those skilled in the art that the specific operations of the respective steps in the PLC-based remote fertigation irrigation control method described above have been described in detail in the above description of the PLC-based remote fertigation irrigation control system 100 with reference to fig. 1 to 5, and thus, repetitive descriptions thereof will be omitted.
According to another aspect of the present application there is also provided a non-volatile computer readable storage medium having stored thereon computer readable instructions which when executed by a computer can perform a method as described above.
Program portions of the technology may be considered to be "products" or "articles of manufacture" in the form of executable code and/or associated data, embodied or carried out by a computer readable medium. A tangible, persistent storage medium may include any memory or storage used by a computer, processor, or similar device or related module. Such as various semiconductor memories, tape drives, disk drives, or the like, capable of providing storage functionality for software.
The application uses specific words to describe embodiments of the application. Reference to "a first/second embodiment," "an embodiment," and/or "some embodiments" means that a particular feature, structure, or characteristic is associated with at least one embodiment of the application. Thus, it should be emphasized and should be appreciated that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various positions in this specification are not necessarily referring to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of the application may be combined as suitable.
Furthermore, those skilled in the art will appreciate that the various aspects of the application are illustrated and described in the context of a number of patentable categories or circumstances, including any novel and useful procedures, machines, products, or materials, or any novel and useful modifications thereof. Accordingly, aspects of the application may be performed entirely by hardware, entirely by software (including firmware, resident software, micro-code, etc.) or by a combination of hardware and software. The above hardware or software may be referred to as a "data block," module, "" engine, "" unit, "" component, "or" system. Furthermore, aspects of the application may take the form of a computer product, comprising computer-readable program code, embodied in one or more computer-readable media.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The foregoing is illustrative of the present invention and is not to be construed as limiting thereof. Although a few exemplary embodiments of this invention have been described, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of this invention. Accordingly, all such modifications are intended to be included within the scope of this invention as defined in the following claims. It is to be understood that the foregoing is illustrative of the present invention and is not to be construed as limited to the specific embodiments disclosed, and that modifications to the disclosed embodiments, as well as other embodiments, are intended to be included within the scope of the appended claims. The invention is defined by the claims and their equivalents.
Claims (10)
1. A PLC-based remote fertigation irrigation control system, comprising:
the video acquisition module is used for acquiring a soil surface state monitoring video of a preset time period acquired by the camera;
the key frame extraction module is used for extracting a plurality of soil surface state monitoring key frames from the soil surface state monitoring video;
the soil surface state feature extraction module is used for respectively enabling the plurality of soil surface state monitoring key frames to pass through a convolutional neural network model comprising a depth feature fusion module so as to obtain a plurality of soil state feature matrixes;
The soil surface state relative change module is used for calculating a transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices to obtain a state transfer time sequence feature diagram composed of a plurality of transfer matrices;
the state time sequence change feature extraction module is used for enabling the state transition time sequence feature graph to pass through a time sequence feature extractor based on a three-dimensional convolutional neural network model to obtain a classification feature graph;
the feature optimization module is used for carrying out feature distribution optimization on the classification feature map so as to obtain an optimized classification feature map;
the irrigation evaluation module is used for enabling the optimized classification feature map to pass through a classifier to obtain classification results, wherein the classification results are used for indicating whether irrigation is needed or not; and
and the control module is used for generating a PLC control instruction to a PLC controller based on the classification result, wherein the PLC controller is used for executing the PLC control instruction to control the opening or closing of the water outlet valve.
2. The PLC-based remote fertigation irrigation control system of claim 1, wherein the soil surface state feature extraction module comprises:
a depth feature extraction unit, configured to input the plurality of soil surface state monitoring key frames into the convolutional neural network model respectively to extract a plurality of shallow feature maps from a shallow layer of the convolutional neural network model and a plurality of deep feature maps from a deep layer of the convolutional neural network model;
The feature fusion unit is used for respectively cascading the shallow feature images and the deep feature images by using the depth feature fusion module of the convolutional neural network model so as to obtain a plurality of soil state feature images; and
and the pooling unit is used for pooling the soil state feature maps along the channel dimension respectively to obtain the soil state feature matrixes.
3. The PLC-based remote fertigation irrigation control system of claim 2, wherein the soil surface state relative change module is configured to:
calculating a transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices according to the following transfer matrix calculation formula to obtain a plurality of transfer matrices;
the calculation formula of the transfer matrix is as follows:
wherein M is 1 、M 2 Representing each adjacent two of the plurality of soil state feature matrices, M representing a transition matrix between each adjacent two of the plurality of soil state feature matrices,representing matrix multiplication; and
and arranging the plurality of transfer matrixes to obtain the state transfer time sequence characteristic diagram.
4. The PLC-based remote fertigation irrigation control system of claim 3, wherein the state timing change feature extraction module is configured to:
input data are separately processed in forward pass of the layer using the timing feature extractor:
performing three-dimensional convolution processing on the input data based on the three-dimensional convolution check to obtain a convolution characteristic diagram;
carrying out mean pooling treatment on the convolution feature map to obtain a pooled feature map; and
non-linear activation is carried out on the pooled feature map so as to obtain an activated feature map;
the output of the last layer of the time sequence feature extractor is the classification feature diagram, and the input of the first layer of the time sequence feature extractor is the state transition time sequence feature diagram.
5. The PLC-based remote fertigation irrigation control system of claim 4, wherein the feature optimization module comprises:
an optimization factor calculation unit for calculating a position information schema attention response factor of each position feature value in the classification feature map to obtain a plurality of position information schema attention response factors; and
and the weighted optimization unit is used for weighted optimization of the position characteristic values of the classification characteristic map by taking the plurality of position information schema attention response factors as weighting coefficients so as to obtain the optimized classification characteristic map.
6. The PLC-based remote fertigation irrigation control system of claim 5, wherein the optimization factor calculation unit is configured to:
calculating location information schema attention response factors for each location feature value in the classification feature map with the following optimization formula to obtain the plurality of location information schema attention response factors;
wherein, the optimization formula is:
wherein f i Is the value of each position characteristic in the classification characteristic diagram, (x) i ,y i ,z i ) Position coordinates for respective position feature values of the classification feature map, andis the global mean of all feature values of the classification feature map,/for>And->Represent the functions of mapping three-dimensional real numbers and two-dimensional real numbers into one-dimensional real numbers, W, H and C are the width, the height and the channel number of the classification characteristic diagram respectively, log represents a logarithmic function based on 2, w i Representing individual ones of the plurality of location information schema attention response factors.
7. The PLC-based remote fertigation irrigation control system of claim 6, wherein the irrigation assessment module comprises:
the feature map unfolding unit is used for unfolding the optimized classification feature map into an optimized classification feature vector according to a row vector or a column vector;
The full-connection coding unit is used for carrying out full-connection coding on the optimized classification feature vector by using a plurality of full-connection layers of the classifier so as to obtain a coding classification feature vector; and
and the classification unit is used for passing the coding classification feature vector through a Softmax classification function of the classifier to obtain the classification result.
8. The remote fertilization and irrigation control method based on the PLC is characterized by comprising the following steps of:
acquiring a soil surface state monitoring video of a preset time period acquired by a camera;
extracting a plurality of soil surface state monitoring key frames from the soil surface state monitoring video;
the soil surface state monitoring key frames are respectively processed through a convolutional neural network model comprising a depth feature fusion module to obtain a plurality of soil state feature matrixes;
calculating a transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices to obtain a state transfer time sequence feature diagram composed of a plurality of transfer matrices;
the state transition time sequence feature diagram is passed through a time sequence feature extractor based on a three-dimensional convolutional neural network model to obtain a classification feature diagram;
performing feature distribution optimization on the classification feature map to obtain an optimized classification feature map;
The optimized classification characteristic diagram is passed through a classifier to obtain a classification result, and the classification result is used for indicating whether irrigation is needed; and
and generating a PLC control instruction to a PLC controller based on the classification result, wherein the PLC controller is used for executing the PLC control instruction to control the opening or closing of the water outlet valve.
9. The PLC-based remote fertilization and irrigation control method of claim 8, wherein the step of passing the plurality of soil surface state monitoring key frames through a convolutional neural network model including a depth feature fusion module to obtain a plurality of soil state feature matrices, respectively, comprises:
respectively inputting the plurality of soil surface state monitoring key frames into the convolutional neural network model to extract a plurality of shallow feature maps from the shallow layer of the convolutional neural network model and a plurality of deep feature maps from the deep layer of the convolutional neural network model;
the depth feature fusion module of the convolutional neural network model is used for respectively cascading the shallow feature images and the deep feature images to obtain a plurality of soil state feature images; and
and respectively carrying out pooling treatment on the soil state feature maps along the channel dimension to obtain the soil state feature matrixes.
10. The PLC based remote fertigation irrigation control method of claim 9, wherein calculating a transition matrix between each adjacent two of the plurality of soil state feature matrices to obtain a state transition timing feature map comprised of a plurality of transition matrices comprises:
calculating a transfer matrix between every two adjacent soil state feature matrices in the plurality of soil state feature matrices according to the following transfer matrix calculation formula to obtain a plurality of transfer matrices;
the calculation formula of the transfer matrix is as follows:
wherein M is 1 、M 2 Representing each adjacent two of the plurality of soil state feature matrices, M representing a transition matrix between each adjacent two of the plurality of soil state feature matrices,representing matrix multiplication; and
and arranging the plurality of transfer matrixes to obtain the state transfer time sequence characteristic diagram.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310704018.9A CN116649191A (en) | 2023-06-14 | 2023-06-14 | Remote fertilization and irrigation control system and method based on PLC |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310704018.9A CN116649191A (en) | 2023-06-14 | 2023-06-14 | Remote fertilization and irrigation control system and method based on PLC |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116649191A true CN116649191A (en) | 2023-08-29 |
Family
ID=87709504
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310704018.9A Pending CN116649191A (en) | 2023-06-14 | 2023-06-14 | Remote fertilization and irrigation control system and method based on PLC |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116649191A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117021435A (en) * | 2023-05-12 | 2023-11-10 | 浙江闽立电动工具有限公司 | Trimming control system and method of trimmer |
| CN117743975A (en) * | 2024-02-21 | 2024-03-22 | 君研生物科技(山西)有限公司 | Hillside cultivated land soil environment improvement method |
-
2023
- 2023-06-14 CN CN202310704018.9A patent/CN116649191A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117021435A (en) * | 2023-05-12 | 2023-11-10 | 浙江闽立电动工具有限公司 | Trimming control system and method of trimmer |
| CN117021435B (en) * | 2023-05-12 | 2024-03-26 | 浙江闽立电动工具有限公司 | Trimming control system and method of trimmer |
| CN117743975A (en) * | 2024-02-21 | 2024-03-22 | 君研生物科技(山西)有限公司 | Hillside cultivated land soil environment improvement method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Kong et al. | Multi-stream hybrid architecture based on cross-level fusion strategy for fine-grained crop species recognition in precision agriculture | |
| CN110135319A (en) | A kind of anomaly detection method and its system | |
| CN108416266B (en) | Method for rapidly identifying video behaviors by extracting moving object through optical flow | |
| CN116649191A (en) | Remote fertilization and irrigation control system and method based on PLC | |
| CN108021947B (en) | A kind of layering extreme learning machine target identification method of view-based access control model | |
| CN107330357A (en) | Vision SLAM closed loop detection methods based on deep neural network | |
| Lee et al. | Plant Identification System based on a Convolutional Neural Network for the LifeClef 2016 Plant Classification Task. | |
| CN111008618B (en) | Self-attention deep learning end-to-end pedestrian re-identification method | |
| CN103440510A (en) | Method for positioning characteristic points in facial image | |
| CN109871892A (en) | A kind of robot vision cognitive system based on small sample metric learning | |
| CN110222760A (en) | A kind of fast image processing method based on winograd algorithm | |
| Ilyas et al. | Multi-scale context aggregation for strawberry fruit recognition and disease phenotyping | |
| CN110163069A (en) | Method for detecting lane lines for assisting driving | |
| CN113610144A (en) | Vehicle classification method based on multi-branch local attention network | |
| CN114092487A (en) | Target fruit instance segmentation method and system | |
| CN111738074B (en) | Pedestrian attribute identification method, system and device based on weak supervision learning | |
| CN116796269A (en) | Management method and system for Internet of things equipment | |
| CN111291695B (en) | Training method and recognition method for recognition model of personnel illegal behaviors and computer equipment | |
| CN113269182A (en) | Target fruit detection method and system based on small-area sensitivity of variant transform | |
| CN117136765A (en) | Greenhouse control system and method based on intelligent agriculture | |
| CN117037028A (en) | Intelligent wig preparation method and system | |
| Aversano et al. | Water stress classification using Convolutional Deep Neural Networks. | |
| CN116883364A (en) | Apple leaf disease identification method based on CNN and Transformer | |
| CN116246184A (en) | Papaver intelligent identification method and system applied to unmanned aerial vehicle aerial image | |
| CN113537240B (en) | Deformation zone intelligent extraction method and system based on radar sequence image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |