CN116383768A - Crop yield prediction method and device based on multi-mode information fusion - Google Patents
Crop yield prediction method and device based on multi-mode information fusion Download PDFInfo
- Publication number
- CN116383768A CN116383768A CN202310407023.3A CN202310407023A CN116383768A CN 116383768 A CN116383768 A CN 116383768A CN 202310407023 A CN202310407023 A CN 202310407023A CN 116383768 A CN116383768 A CN 116383768A
- Authority
- CN
- China
- Prior art keywords
- module
- crop yield
- data
- crop
- representation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 75
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000003860 storage Methods 0.000 claims abstract description 25
- 230000015654 memory Effects 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 8
- 238000012417 linear regression Methods 0.000 claims description 8
- 238000011176 pooling Methods 0.000 claims description 8
- 238000001514 detection method Methods 0.000 claims description 6
- 235000013399 edible fruits Nutrition 0.000 claims description 6
- 238000013139 quantization Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 abstract description 4
- 239000002689 soil Substances 0.000 description 11
- 230000008569 process Effects 0.000 description 9
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000007637 random forest analysis Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000007499 fusion processing Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 238000012271 agricultural production Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000004927 clay Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 235000019580 granularity Nutrition 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- YHXISWVBGDMDLQ-UHFFFAOYSA-N moclobemide Chemical compound C1=CC(Cl)=CC=C1C(=O)NCCN1CCOCC1 YHXISWVBGDMDLQ-UHFFFAOYSA-N 0.000 description 1
- 239000005416 organic matter Substances 0.000 description 1
- 238000001556 precipitation Methods 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 239000004576 sand Substances 0.000 description 1
- 229920006395 saturated elastomer Polymers 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/02—Agriculture; Fishing; Forestry; Mining
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- Business, Economics & Management (AREA)
- Agronomy & Crop Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Animal Husbandry (AREA)
- Marine Sciences & Fisheries (AREA)
- Mining & Mineral Resources (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Economics (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a crop yield prediction method and device based on multi-mode information fusion and a computer storage medium, which are used for acquiring crop multi-mode data comprising daily growth data and historical annual yield data of crops; establishing a crop yield prediction model and predicting the crop yield by utilizing the crop yield prediction model, wherein the model comprises a representation module, a first modeling module, a first fusion module, a second modeling module and a crop yield prediction module; on the other hand, modeling is carried out on daily growth data and historical annual output of crops in a layered time sequence modeling mode, and potential association relations among the historical annual output of different crops in the area are considered, so that the crop output prediction effect is further improved. The method has the characteristics of easy realization and good effect.
Description
Technical Field
The invention relates to the technical field of agricultural information processing, in particular to a crop yield prediction method and device based on multi-mode information fusion and a computer storage medium.
Background
With the rapid development of artificial intelligence and agricultural informatization technologies, the artificial intelligence technology is gradually applied to various aspects of agricultural production and management, and achieves good effects. The purpose of crop yield prediction is to measure and calculate the yield in advance in the growth period of crops, and has important value and significance for national planning and development of national economy, in particular for preparing agricultural policies (including acquisition, storage, processing, trade and the like).
There are several methods for predicting yield of different crops, and these methods can be divided into three main categories: (1) Statistical analysis-based methods such as differential integration moving average autoregressive models (Autoregressive Integrated Moving Average Model, ARIMA), gray predictive models, stepwise regression models, and the like. The model has a relatively simple structure, few consideration factors and high model precision; (2) Traditional machine learning methods, such as multiple linear regression (Multiple Linear Regression, MLR), BP neural network (Back Propagation Neural Network, BPNN), random Forest (RF), support vector machine (Support Vector Machine, SVM), multi-layer neural network (Artificial Neural Network, ANN), K-Nearest Neighbor (KNN), and the like. The model can establish nonlinear relations between soil, weather, space and other influencing factors and yield, achieves good precision, and still has an improved space; (3) Deep learning methods, such as convolutional neural networks (Convolutional Neural Network, CNN), long Short-Term Memory networks (LSTM), etc., are used for encoding data such as images, LSTM is used for modeling time series data, and on crop yield prediction problems, the performance of the conventional machine learning methods is achieved. However, the current crop yield prediction method based on the deep learning method is mainly based on structural data such as soil, weather and the like, and a small part of the crop yield prediction method utilizes remote sensing image data, so that modeling capability is slightly insufficient in tracking the growth trend of crops.
Disclosure of Invention
Aiming at the problems, the invention provides a crop yield prediction method, a device and a computer storage medium based on multi-mode information fusion, which are characterized in that on one hand, daily crop growth picture time sequence data is introduced to carry out fine-grained modeling on crop growth trend; on the other hand, modeling is carried out on daily growth data and historical annual output of crops in a layered time sequence modeling mode, and potential association relations among the historical annual output of different crops in the region are considered; the method of the invention has the characteristics of easy realization and good effect.
In a first aspect of the invention, a crop yield prediction method based on multi-modal information fusion, the method comprising the steps of:
acquiring crop multi-modal data including daily growth data and historical annual yield data of the crop;
establishing a crop yield prediction model and utilizing the crop yield prediction model to predict the crop yield, wherein the model comprises a representation module, a first modeling module, a first fusion module, a second modeling module and a crop yield prediction module, and the model comprises the following components:
the representation module is used for representing the input crop multi-modal data to obtain multi-modal daily growth data representation and historical annual output data representation;
the first modeling module is configured to model the multimodal daily growth data representation for a plurality of consecutive days;
the first fusion module is used for fusing the output modeled by the first modeling module to obtain the integral representation of the daily growth time sequence data of the multi-mode crops;
the second fusion module is used for fusing the integral representation of the daily growth time sequence data of the multi-mode crops with the output data representation of the historical years to obtain the integral representation of the fused annual data;
the second modeling module is used for modeling the whole annual data representation;
and the crop yield prediction module is used for predicting the crop yield according to the output modeled by the second modeling module to obtain a target crop yield prediction result.
The invention further adopts the technical scheme that: the first modeling module/the second modeling module is any one of LSTM, transformer.
The invention further adopts the technical scheme that: the second fusion module is a self-attention network.
The invention further adopts the technical scheme that: the crop yield prediction module adopts a prediction method of linear regression, and the utilized loss function is L1 loss or L2 loss.
The invention further adopts the technical scheme that: the representation module represents the input crop multi-mode data, and specifically comprises the following steps:
coding the obtained daily growth data of crops through ResNet or DenseNet and carrying out average pooling on coding results;
crop fruit detection is carried out through a YOLO series model, and the fruit detection result is quantified numerically;
and splicing the output result after the average pooling and the output result after the numerical quantization to obtain the multi-modal daily growth data representation of the crops.
In a second aspect of the present invention, there is provided a crop yield prediction apparatus based on multimodal information fusion, comprising:
a crop multi-modal data acquisition unit configured to acquire crop multi-modal data including crop daily growth data and historical annual output data;
the model establishment and prediction unit is used for establishing a crop yield prediction model and predicting the crop yield by utilizing the crop yield prediction model, wherein the model comprises a representation module, a first modeling module, a first fusion module, a second modeling module and a crop yield prediction module, and the model comprises the following components:
the representation module is used for representing the input crop multi-modal data to obtain multi-modal daily growth data representation and historical annual output data representation;
the first modeling module is configured to model the multimodal daily growth data representation for a plurality of consecutive days;
the first fusion module is used for fusing the output modeled by the first modeling module to obtain the integral representation of the daily growth time sequence data of the multi-mode crops;
the second fusion module is used for fusing the integral representation of the daily growth time sequence data of the multi-mode crops with the output data representation of the historical years to obtain the integral representation of the fused annual data;
the second modeling module is used for modeling the whole annual data representation;
and the crop yield prediction module is used for predicting the crop yield according to the output modeled by the second modeling module to obtain a target crop yield prediction result.
The invention further adopts the technical scheme that: the first modeling module/the second modeling module is any one of LSTM, transformer.
The invention further adopts the technical scheme that: the crop yield prediction module adopts a prediction method of linear regression, and the utilized loss function is L1 loss or L2 loss.
In a third aspect of the present invention, a crop yield prediction apparatus based on multi-modal information fusion, includes: a processor; and a memory, wherein the memory stores a computer executable program which, when executed by the processor, performs the crop yield prediction method based on multimodal information fusion described above.
In a fourth aspect of the present invention, there is provided a computer readable storage medium having instructions stored thereon, which when executed by a processor, cause the processor to perform the above-described crop yield prediction method based on multimodal information fusion.
The invention provides a crop yield prediction method and device based on multi-mode information fusion and a computer storage medium, which are used for acquiring crop multi-mode data comprising daily growth data and historical annual yield data of crops; the crop yield prediction method comprises the steps of establishing a crop yield prediction model and utilizing the crop yield prediction model to predict the crop yield, wherein the model comprises a representation module, a first modeling module, a first fusion module, a second modeling module and a crop yield prediction module, and the crop yield prediction method has the following specific advantages that:
(1) The conventional crop yield prediction method does not consider that daily growth image data of crops and structural data such as soil, weather and the like are fused, and the data adopted by the conventional crop yield prediction method is supplemented by the work based on multi-mode data fusion, so that the crop yield prediction method and device based on multi-mode data fusion are provided;
(2) The first modeling module is used for modeling the multi-mode daily growth data representation for a plurality of continuous days, and compared with the past method which only considers historical annual data, the modeling of the multi-mode daily growth data representation for a plurality of continuous days can more finely describe the growth trend of crops, and the accuracy of the model is better improved;
(3) The second fusion module is used for fusing the integral representation of the daily growth time sequence data of the multi-mode crops and the historical annual output data representation, and fusing the crop growth trend representation, the historical output representations of other crops in the area and the historical output representations of target crops in the specific implementation process, and in the annual data fusion process, the attention mechanism can be used for modeling the association relationship between the crop growth trend and the historical output of different crops, so that the model progress is further improved.
Drawings
FIG. 1 is a schematic flow chart of a crop yield prediction method based on multi-modal information fusion in an embodiment of the invention;
FIG. 2 is a schematic diagram of a crop yield prediction model established in an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a crop yield prediction device based on multi-modal information fusion in an embodiment of the invention;
fig. 4 is an architecture of a computer device in an embodiment of the invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart depicts steps as a sequential process, many of the steps may be implemented in parallel, concurrently, or with other steps. Furthermore, the order of the steps may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, and the like.
The embodiment of the invention provides a crop yield prediction method and device based on multi-mode information fusion and a computer storage medium, and provides the following embodiments:
example 1 according to the invention
Referring to fig. 1, a flow chart of a crop yield prediction method based on multi-modal information fusion is shown, and the method comprises the following steps:
s110, acquiring crop multi-mode data comprising daily growth data and historical annual output data of crops;
in the specific implementation process, the crop multi-mode data comprise growth images, soil data, weather data, crop areas, crop historical yield and the like of crops; crop growth images can be obtained through a certain number of cameras at fixed positions; the soil data comprises the percentage of clay/silt/sand, the effective water content, the pH value, the organic matter content, the saturated water conductivity of the soil, the temperature and the like, and can be obtained by a soil moisture monitor; weather data including daily length, precipitation, solar radiation, maximum temperature, minimum temperature, average temperature, etc. can be obtained by a meteorological data acquisition instrument; the agricultural crop area and the crop historical yield can belong to historical record data and can be directly obtained. Examples crop growth images, soil data and weather data were collected in days, and farmland crop areas and crop historical yields were collected in years. The influence of daily time sequence data such as crop growth images, soil data, weather data and the like on crop yield is considered, and the historical yields of different crops in the area are referred; crop yield can be effectively predicted by modeling crop yield influencing factors from different granularities and layers.
S120, referring to FIG. 2, a crop yield prediction model is established and is utilized to predict the crop yield, wherein the model comprises a representation module, a first modeling module, a first fusion module, a second modeling module and a crop yield prediction module, and the crop yield prediction module comprises the following components:
the representation module is used for representing the input crop multi-modal data to obtain multi-modal daily growth data representation and historical annual output data representation;
e＝f(x)
the first modeling module is configured to model the multimodal daily growth data representation for a plurality of consecutive days;
h＝Encoder1(e 1 ,e 2 ,…,en)
preferably, the first modeling module may be any one of LSTM, transformer.
The first fusion module is used for fusing the output modeled by the first modeling module to obtain the integral representation of the daily growth time sequence data of the multi-mode crops;
in the specific implementation process, networks such as Pooling (Pooling), attention (Attention) and the like are used for fusion, so that the integral representation of daily growth time sequence data of the multi-mode crops is obtained;
h 1 ＝Fusion1(h)
the second fusion module is used for fusing the integral representation of the daily growth time sequence data of the multi-mode crops with the d representation of the yield data of the historical years to obtain the integral representation of the annual data after fusion;
h 2 ＝Fusion2([h 1 ,d])
preferably, the second fusion module is a self-attention network.
The second modeling module is used for modeling the whole annual data representation;
h 3 ＝Encoder2(h 2 )
preferably, the second modeling module is any one of LSTM, transformer.
And the crop yield prediction module is used for predicting the crop yield according to the output modeled by the second modeling module to obtain a target crop yield prediction result.
m＝g θ (h 3 )
Preferably, the crop yield prediction module employs a prediction method that is linear regression, and the loss function utilized is either L1 loss or L2 loss.
In the implementation process, the representation module carries out numerical quantization on the obtained daily soil data, weather data and historical annual output data to obtain corresponding representation, and the representation module represents the input crop multi-mode data, which specifically comprises the following steps:
for the obtained daily growth data X of crops 1 Coding by ResNet or DenseNet and carrying out average pooling on coding results;
crop fruit X through YOLO series model 2 Real detection and numerical quantification of fruit detection results are carried out;
and splicing the output result after the average pooling and the output result after the numerical quantization to obtain the multi-modal daily growth data representation of the crops.
Example 2 according to the invention
The crop yield prediction device 300 based on multi-mode information fusion provided by the second embodiment of the present invention may perform the crop yield prediction method based on multi-mode information fusion provided by the embodiment 1 of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. Fig. 3 is a schematic structural diagram of a crop yield prediction apparatus 300 based on multi-modal information fusion in embodiment 2 of the present invention. Referring to fig. 3, a crop yield prediction apparatus 300 based on multi-modal information fusion according to an embodiment of the present invention may specifically include:
a crop multi-modal data acquisition unit 310 for acquiring crop multi-modal data including crop daily growth data and historical annual output data;
the model building and predicting unit 320 is configured to build a crop yield prediction model and predict a crop yield using the crop yield prediction model, where the model includes a representation module, a first modeling module, a first fusion module, a second modeling module, and a crop yield prediction module, and wherein:
the representation module is used for representing the input crop multi-modal data to obtain multi-modal daily growth data representation and historical annual output data representation;
the first modeling module is configured to model the multimodal daily growth data representation for a plurality of consecutive days;
the first fusion module is used for fusing the output modeled by the first modeling module to obtain the integral representation of the daily growth time sequence data of the multi-mode crops;
the second fusion module is used for fusing the integral representation of the daily growth time sequence data of the multi-mode crops with the output data representation of the historical years to obtain the integral representation of the fused annual data;
the second modeling module is used for modeling the whole annual data representation;
and the crop yield prediction module is used for predicting the crop yield according to the output modeled by the second modeling module to obtain a target crop yield prediction result.
Further, the first modeling module/the second modeling module is any one of LSTM, transformer.
Further, the prediction method adopted by the crop yield prediction module is linear regression, and the loss function used is L1 loss or L2 loss.
In addition to the above-described units, the crop yield prediction apparatus 300 based on multi-modal information fusion may further include other components, however, since these components are not related to the contents of the embodiments of the present disclosure, illustration and description thereof are omitted herein.
The specific operation of the crop yield prediction apparatus 300 based on the multi-mode information fusion is described with reference to the above description of the crop yield prediction method embodiment 1 based on the multi-mode information fusion, and will not be repeated.
Example 3 according to the invention
A system according to an embodiment of the invention may also be implemented by means of the architecture of the computing device shown in fig. 4. Fig. 4 illustrates an architecture of the computing device. As shown in fig. 4, a computer system 410, a system bus 430, one or more CPUs 440, input/output 420, memory 450, and the like. The memory 450 may store various data or files used for computer processing and/or communication and program instructions executed by the CPU, including the method of embodiment 1. The architecture shown in fig. 4 is merely exemplary, and one or more of the components in fig. 4 may be adapted as needed to implement different devices. The memory 450 is used as a computer readable storage medium for storing software programs, computer executable programs and modules, such as program instructions/modules corresponding to the crop yield prediction method based on multimodal information fusion in the embodiment of the invention (for example, the crop multi-modality data acquisition unit 310 and the model building and prediction unit 320 in the crop yield prediction apparatus 300 based on multimodal information fusion). The one or more CPUs 440 execute various functional applications and data processing of the system of the present invention by running software programs, instructions and modules stored in the memory 450, i.e., implement the above-described crop yield prediction method based on multi-modal information fusion, which includes:
acquiring crop multi-modal data including daily growth data and historical annual yield data of the crop;
establishing a crop yield prediction model and utilizing the crop yield prediction model to predict the crop yield, wherein the model comprises a representation module, a first modeling module, a first fusion module, a second modeling module and a crop yield prediction module, and the model comprises the following components:
the representation module is used for representing the input crop multi-modal data to obtain multi-modal daily growth data representation and historical annual output data representation;
the first modeling module is configured to model the multimodal daily growth data representation for a plurality of consecutive days;
the first fusion module is used for fusing the output modeled by the first modeling module to obtain the integral representation of the daily growth time sequence data of the multi-mode crops;
the second fusion module is used for fusing the integral representation of the daily growth time sequence data of the multi-mode crops with the output data representation of the historical years to obtain the integral representation of the fused annual data;
the second modeling module is used for modeling the whole annual data representation;
and the crop yield prediction module is used for predicting the crop yield according to the output modeled by the second modeling module to obtain a target crop yield prediction result.
Of course, the processor of the server provided by the embodiment of the present invention is not limited to executing the method operations described above, and may also execute the related operations in the crop yield prediction method based on multimodal information fusion provided by any embodiment of the present invention.
Input/output 420 may be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the device. Input/output 420 may also include a display device such as a display screen.
Example 4 according to the invention
Embodiments of the present invention may also be implemented as a computer-readable storage medium. The computer-readable storage medium according to embodiment 4 has a computer program stored thereon. The crop yield prediction method based on multimodal information fusion according to embodiment 1 of the present invention described with reference to the above drawings may be performed when the computer program is executed by a processor.
Of course, the storage medium containing the computer executable instructions provided in the embodiments of the present invention is not limited to the above-described method operations, and may also perform the related operations in the crop yield prediction method based on multimodal information fusion provided in any embodiment of the present invention.
The computer-readable storage media of embodiments of the present invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or terminal. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
In summary, the method, the device and the computer storage medium for predicting crop yield based on multi-modal information fusion provided by the above embodiments acquire crop multi-modal data including daily growth data and historical annual yield data of crops; the crop yield prediction method comprises the steps of establishing a crop yield prediction model and utilizing the crop yield prediction model to predict the crop yield, wherein the model comprises a representation module, a first modeling module, a first fusion module, a second modeling module and a crop yield prediction module, and the crop yield prediction method has the following specific advantages that: the conventional crop yield prediction method does not consider that daily growth image data of crops and structural data such as soil, weather and the like are fused, and the data adopted by the conventional crop yield prediction method is supplemented by the work based on multi-mode data fusion, so that the crop yield prediction method and device based on multi-mode data fusion are provided; the first modeling module is used for modeling the multi-mode daily growth data representation for a plurality of continuous days, and compared with the past method which only considers historical annual data, the modeling of the multi-mode daily growth data representation for a plurality of continuous days can more finely describe the growth trend of crops, and the accuracy of the model is better improved; the second fusion module is used for fusing the integral representation of the daily growth time sequence data of the multi-mode crops and the historical annual output data representation, and fusing the crop growth trend representation, the historical output representations of other crops in the area and the historical output representations of target crops in the specific implementation process, and in the annual data fusion process, the attention mechanism can be used for modeling the association relationship between the crop growth trend and the historical output of different crops, so that the model progress is further improved.
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.
Claims (10)
1. A crop yield prediction method based on multi-modal information fusion, the method comprising the steps of:
acquiring crop multi-modal data including daily growth data and historical annual yield data of the crop;
establishing a crop yield prediction model and utilizing the crop yield prediction model to predict the crop yield, wherein the model comprises a representation module, a first modeling module, a first fusion module, a second modeling module and a crop yield prediction module, and the model comprises the following components:
the representation module is used for representing the input crop multi-modal data to obtain multi-modal daily growth data representation and historical annual output data representation;
the first modeling module is configured to model the multimodal daily growth data representation for a plurality of consecutive days;
the first fusion module is used for fusing the output modeled by the first modeling module to obtain the integral representation of the daily growth time sequence data of the multi-mode crops;
the second fusion module is used for fusing the integral representation of the daily growth time sequence data of the multi-mode crops with the output data representation of the historical years to obtain the integral representation of the fused annual data;
the second modeling module is used for modeling the whole annual data representation;
and the crop yield prediction module is used for predicting the crop yield according to the output modeled by the second modeling module to obtain a target crop yield prediction result.
2. The crop yield prediction method based on multimodal information fusion as claimed in claim 1, wherein the first modeling module/the second modeling module is any one of LSTM, transformer.
3. The method for predicting crop yield based on multimodal information fusion as recited in claim 1, wherein the second fusion module is a self-attention network.
4. The method for predicting crop yield based on multimodal information fusion as claimed in claim 1, wherein the prediction method adopted by the crop yield prediction module is linear regression, and the loss function used is L1 loss or L2 loss.
5. The crop yield prediction method based on multi-modal information fusion according to claim 1, wherein the representation module represents the input crop multi-modal data, and specifically comprises:
coding the obtained daily growth data of crops through ResNet or DenseNet and carrying out average pooling on coding results;
crop fruit detection is carried out through a YOLO series model, and the fruit detection result is quantified numerically;
and splicing the output result after the average pooling and the output result after the numerical quantization to obtain the multi-modal daily growth data representation of the crops.
6. Crop yield prediction device based on multi-mode information fusion, which is characterized by comprising:
a crop multi-modal data acquisition unit configured to acquire crop multi-modal data including crop daily growth data and historical annual output data;
the model establishment and prediction unit is used for establishing a crop yield prediction model and predicting the crop yield by utilizing the crop yield prediction model, wherein the model comprises a representation module, a first modeling module, a first fusion module, a second modeling module and a crop yield prediction module, and the model comprises the following components:
the representation module is used for representing the input crop multi-modal data to obtain multi-modal daily growth data representation and historical annual output data representation;
the first modeling module is configured to model the multimodal daily growth data representation for a plurality of consecutive days;
the first fusion module is used for fusing the output modeled by the first modeling module to obtain the integral representation of the daily growth time sequence data of the multi-mode crops;
the second fusion module is used for fusing the integral representation of the daily growth time sequence data of the multi-mode crops with the output data representation of the historical years to obtain the integral representation of the fused annual data;
the second modeling module is used for modeling the whole annual data representation;
and the crop yield prediction module is used for predicting the crop yield according to the output modeled by the second modeling module to obtain a target crop yield prediction result.
7. The crop yield prediction device based on multimodal information fusion as claimed in claim 6, wherein the first modeling module/the second modeling module is any one of LSTM, transformer.
8. The crop yield prediction apparatus based on multimodal information fusion according to claim 6, wherein the prediction method adopted by the crop yield prediction module is linear regression, and the loss function used is L1 loss or L2 loss.
9. Crop yield prediction device based on multi-mode information fusion, which is characterized by comprising:
a processor; and a memory, wherein the memory has stored therein a computer executable program that, when executed by the processor, performs the crop yield prediction method based on multimodal information fusion of any one of claims 1-5.
10. A computer readable storage medium having instructions stored thereon, which when executed by a processor, cause the processor to perform the crop yield prediction method based on multimodal information fusion as claimed in any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310407023.3A CN116383768A (en) | 2023-04-17 | 2023-04-17 | Crop yield prediction method and device based on multi-mode information fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310407023.3A CN116383768A (en) | 2023-04-17 | 2023-04-17 | Crop yield prediction method and device based on multi-mode information fusion |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116383768A true CN116383768A (en) | 2023-07-04 |
Family
ID=86976802
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310407023.3A Pending CN116383768A (en) | 2023-04-17 | 2023-04-17 | Crop yield prediction method and device based on multi-mode information fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116383768A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117575111A (en) * | 2024-01-16 | 2024-02-20 | 安徽农业大学 | Agricultural remote sensing image space-time sequence prediction method based on transfer learning |
-
2023
- 2023-04-17 CN CN202310407023.3A patent/CN116383768A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117575111A (en) * | 2024-01-16 | 2024-02-20 | 安徽农业大学 | Agricultural remote sensing image space-time sequence prediction method based on transfer learning |
| CN117575111B (en) * | 2024-01-16 | 2024-04-12 | 安徽农业大学 | Agricultural remote sensing image space-time sequence prediction method based on transfer learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | A novel Encoder-Decoder model based on read-first LSTM for air pollutant prediction | |
| Li et al. | A ship motion forecasting approach based on empirical mode decomposition method hybrid deep learning network and quantum butterfly optimization algorithm | |
| CN115879535B (en) | Training method, device, equipment and medium for automatic driving perception model | |
| CN110705684A (en) | Environment self-adaptive learning method and system based on end cloud cooperation | |
| CN112766596B (en) | Construction method of building energy consumption prediction model, energy consumption prediction method and device | |
| CN116383768A (en) | Crop yield prediction method and device based on multi-mode information fusion | |
| CN115512005A (en) | Data processing method and device | |
| CN116502692A (en) | Model training method, photovoltaic power generation power prediction method and device | |
| Zhang et al. | A novel encoder-decoder model for multivariate time series forecasting | |
| Liang et al. | Foundation models for time series analysis: A tutorial and survey | |
| Zhang et al. | Comparative study of rainfall prediction based on different decomposition methods of VMD | |
| CN111062477B (en) | Data processing method, device and storage medium | |
| CN111046321B (en) | Photovoltaic power station operation and maintenance strategy optimization method and device | |
| Ding et al. | JMDC: A joint model and data compression system for deep neural networks collaborative computing in edge-cloud networks | |
| CN113723712A (en) | Wind power prediction method, system, device and medium | |
| Gupta et al. | Global Horizontal Irradiance Estimation Using Bi-LSTM Algorithm | |
| Wang et al. | Dynamic graphs attention for ocean variable forecasting | |
| Zhang et al. | Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models | |
| Dai et al. | Multimodal deep learning water level forecasting model for multiscale drought alert in Feiyun River basin | |
| KR102613227B1 (en) | Electronic device for evaluating water supply reliability in agricultural reservoir based on big data and machine learning and controlling method thereof | |
| Morales-García et al. | Evaluation of low-power devices for smart greenhouse development | |
| CN117407754B (en) | Multi-mode large model training strategy determination method, electronic equipment and medium | |
| Vignesh et al. | Deep Reinforcement Learning Based Weather Monitoring Systemusing Arduino for Smart Environment | |
| CN113155136B (en) | Training method and device of GNSS satellite orbit prediction deviation analysis model | |
| Wei et al. | Embedded Design and Implementation of a Wireless Multimedia Platform for Digital Estimation of Agricultural Output |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |