CN110349044A - A kind of agriculture feelings monitoring method, system and electronic equipment - Google Patents
A kind of agriculture feelings monitoring method, system and electronic equipment Download PDFInfo
- Publication number
- CN110349044A CN110349044A CN201910637226.5A CN201910637226A CN110349044A CN 110349044 A CN110349044 A CN 110349044A CN 201910637226 A CN201910637226 A CN 201910637226A CN 110349044 A CN110349044 A CN 110349044A
- Authority
- CN
- China
- Prior art keywords
- agricultural condition
- remote sensing
- sensing data
- agricultural
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 122
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000004364 calculation method Methods 0.000 claims description 41
- 238000012545 processing Methods 0.000 claims description 28
- 230000015654 memory Effects 0.000 claims description 18
- 238000007726 management method Methods 0.000 claims description 9
- 238000013499 data model Methods 0.000 claims description 8
- 238000013500 data storage Methods 0.000 claims description 7
- 239000002689 soil Substances 0.000 claims description 7
- 238000011835 investigation Methods 0.000 claims description 6
- 230000002265 prevention Effects 0.000 abstract description 3
- 238000013507 mapping Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000013468 resource allocation Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000000547 structure data Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000003973 irrigation Methods 0.000 description 1
- 230000002262 irrigation Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/02—Agriculture; Fishing; Forestry; Mining
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Marine Sciences & Fisheries (AREA)
- Economics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Agronomy & Crop Science (AREA)
- Animal Husbandry (AREA)
- Databases & Information Systems (AREA)
- Mining & Mineral Resources (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Alarm Systems (AREA)
Abstract
This application involves a kind of agriculture feelings monitoring method, system and electronic equipments.The described method includes: step a: obtaining original agriculture feelings remotely-sensed data；Step b: the original agriculture feelings remotely-sensed data is imported into cloud platform, the cloud platform establishes nuclear sparse expression model to agriculture feelings remotely-sensed data, the sparse agriculture feelings remotely-sensed data of corresponding core and its metadata are generated, and the sparse agriculture feelings remotely-sensed data of the core and its metadata are stored in file system or database；Step c: the agriculture feelings that receiving terminal apparatus is initiated monitor request, and agriculture feelings monitoring request is put into Kafka message queue, corresponding agriculture feelings remotely-sensed data is searched in the file system or database using Kafka, and index calculating is carried out to the agriculture feelings remotely-sensed data, obtain agriculture feelings monitoring result.The application reduces agricultural drought disaster harm by the monitoring of timely and effectively agriculture feelings and early warning, General Promotion disaster alarm and prevention and control capability.
Description
Technical Field
The application belongs to the technical field of agricultural condition monitoring, and particularly relates to an agricultural condition monitoring method, an agricultural condition monitoring system and electronic equipment.
Background
Agricultural condition remote sensing data is special monitoring data generated by satellite agricultural condition remote sensing equipment, and characteristics of objects such as positions, time, distribution and the like are recorded. The mining of agricultural condition remote sensing data provides great support for wide application including farmland monitoring, irrigation planning, disaster early warning, environment protection and the like. On one hand, since the satellite equipment mainly generates agricultural condition remote sensing image data in a periodic sampling mode, the data scale generated by daily and monthly accumulation is extremely large. On the other hand, the data have different formats, different qualities and different contents, and really form large data with multiple sources and different structures. How to rapidly and effectively process agricultural remote sensing data and realize real-time monitoring and early warning of drought is an important challenge.
With the rapid development of remote sensing technology, agricultural condition remote sensing data is also growing explosively, which brings great challenges to the calculation of agricultural condition remote sensing data. A space big data processing system based on a Hadoop architecture is a current research hotspot, and a plurality of research works in the aspect are recently carried out. Such as GeoMesa [ Fox A. GeoMesa: assisted distributed architecture for distributed-temporal fusion [ C ]// SPIE Defence + Security.2015:94730F ] based on the Accumulo system, PAIRS [ Leven J.Klein, Fernando J.Marianno, Conrad M.Albrecht, Marcus Freetag, Siyuan Lu, Nigel Hinds, Xioayan Shao, Sergio Bermeudadreigutz, and HendrrikF. Hamanis Pairs: A scalable gel-mapping Data, and distributed-type API S.S.H. for distributed-mapping Data, I.S. Zymustric Data, and E.S. distributed mapping Data for distributed-mapping Data, I.S. H.S. for distributed-mapping Data, I.S. for distributed-mapping, I.S. H.S. for distributed mapping [ C ]/(S. GeoMesa: S. for mapping [ C ]// S. Spie. Zhendong Bei, Huiling Zhang, Wen Xiong, LievenEeckhout, Chengzhong Xu, and Shenzhong Feng. Rfhoc A random-for ap pro auto-tuning hdoop's configuration. IEEE Transactions on Parallel and distributed Systems,1 (2015). The space big data processing systems are built based on Hadoop or spark with an open source, and each system has advantages and disadvantages, for example, a GeoMesa system does not allow a single point of failure, and the expansibility of the system is poor; the pair system adopts single batch processing, and cannot realize real-time streaming data processing work. The adoption of memory-based computing Spark [ Zaharia M, Chowdury M, Franklin M J, et al. Spark: computing with working sections [ C ]// Usenix Conference on Hot tips in Cloud computing. Using Association,2010:10-10 ] distributed computing framework and the selection of YARN as resource scheduling system and the adoption of HDFS as distributed storage system is usually an effective means to overcome the problem. Spark is an open-source distributed computing framework, based on the concept of elastic distributed data set (RDD), an advanced directed acyclic graph execution mechanism is adopted to support cyclic data flow operation, and multiple iterative operations can be completed by importing data into a memory once. Therefore, the method is particularly suitable for a big data calculation analysis method based on multiple iterations, and has greater advantages compared with MapReduce which needs to import data into a memory in each iteration.
In summary, the existing spatial big data processing system has respective characteristics and advantages in the fields of spatial big data modeling, index design, system architecture and processing application, but most of the systems lack the support of the drought monitoring big data storage for nuclear sparse modeling, so that the use efficiency of the drought monitoring big data is low, and the necessary technical support cannot be provided for agricultural drought early warning.
Disclosure of Invention
The application provides an agricultural condition monitoring method, an agricultural condition monitoring system and electronic equipment, and aims to solve at least one of the technical problems in the prior art to a certain extent.
In order to solve the above problems, the present application provides the following technical solutions:
an agricultural condition monitoring method comprises the following steps:
step a: acquiring original agricultural condition remote sensing data;
step b: importing the original agricultural condition remote sensing data into a cloud platform, establishing a nuclear sparse representation model for the agricultural condition remote sensing data by the cloud platform, generating corresponding nuclear sparse agricultural condition remote sensing data and metadata thereof, and storing the nuclear sparse agricultural condition remote sensing data and the metadata thereof in a file system or a database;
step c: receiving an agricultural condition monitoring request initiated by terminal equipment, putting the agricultural condition monitoring request into a Kafka message queue, searching corresponding agricultural condition remote sensing data in the file system or the database by utilizing Kafka, and performing index calculation on the agricultural condition remote sensing data to obtain an agricultural condition monitoring result.
The technical scheme adopted by the embodiment of the application further comprises the following steps: in the step a, the obtaining of the original agricultural condition remote sensing data specifically includes: acquiring original agricultural condition remote sensing data through a satellite remote sensing image, an unmanned aerial vehicle aerial image, a mobile phone shot image or agricultural soil moisture observation and agricultural condition ground investigation data, and storing the original agricultural condition remote sensing data in a local database; the obtained original agricultural condition remote sensing Data is uniformly managed by a Management Engine, and a uniform user interface is provided through a Data Model and a Language.
The technical scheme adopted by the embodiment of the application further comprises the following steps: in the step b, the cloud platform establishes a kernel sparse representation model for the agricultural condition remote sensing data, generates corresponding kernel sparse agricultural condition remote sensing data and metadata thereof, and stores the kernel sparse agricultural condition remote sensing data and the metadata thereof in a file system or a database specifically includes: the method comprises the steps of adopting HDFS + Database to store agricultural condition remote sensing big data, firstly establishing kernel sparse representation on the agricultural condition remote sensing data, generating corresponding kernel sparse agricultural condition remote sensing data and metadata thereof, wherein the metadata comprises a kernel sparse data file name, a file type, a storage path, an index path and creation time, then adopting a tile structure to encode the kernel sparse agricultural condition remote sensing data and the metadata thereof, then storing the encoded kernel sparse agricultural condition remote sensing data and the metadata thereof in a file system or a Database, and establishing an index on the kernel sparse agricultural condition remote sensing data.
The technical scheme adopted by the embodiment of the application further comprises the following steps: in the step c, the receiving of the agricultural condition monitoring request initiated by the terminal device, the placing of the agricultural condition monitoring request into a Kafka message queue, the searching of the corresponding agricultural condition remote sensing data in the file system or the database by using Kafka, and the performing of the index calculation on the agricultural condition remote sensing data to obtain the agricultural condition monitoring result specifically include:
1) the APP sends a request to the Tomcat through the Servlet;
2) WebServer sends a request instruction to Hadoop,
WebServer places the request instruction in Kafka Producer,
kafka Consumer passes the received instruction to Hadoop;
3) the Hadoop/Spark executes the received instruction and sends a required data request to PostgreSQL;
4) the PostgreSQL searches a required data position and initiates a data acquisition request to the HDFS;
5) the HDFS returns data required by Hadoop/Spark processing;
6) performing exponential calculation on Hadoop/Spark;
7) the Hadoop/Spark returns an index calculation result to the HDFS, and the index is built by the PostgreSQL;
8) the Hadoop/Spark returns the index calculation result to WebServer,
Hadoop/Spark returns the result to Kafka Producer,
kafka Consumer gets the output to WebServer.
The technical scheme adopted by the embodiment of the application further comprises the following steps: the step c further comprises: and releasing the agricultural condition monitoring result to an agricultural condition monitoring and early warning display platform in real time through micro-service, and carrying out agricultural condition drought message early warning according to the agricultural condition monitoring result.
Another technical solution adopted in the embodiment of the present application further includes: an agricultural condition monitoring and early warning system, comprising:
a data acquisition module: the system is used for acquiring original agricultural condition remote sensing data;
a data storage module: the system comprises a cloud platform, a kernel sparse representation model, a file system or a database, wherein the cloud platform is used for importing the original agricultural condition remote sensing data into the cloud platform, establishing the kernel sparse representation model for the agricultural condition remote sensing data by the cloud platform, generating corresponding kernel sparse agricultural condition remote sensing data and metadata thereof, and storing the kernel sparse agricultural condition remote sensing data and the metadata thereof in the file system or the database;
a data calculation module: the agricultural condition monitoring system is used for receiving an agricultural condition monitoring request initiated by terminal equipment, putting the agricultural condition monitoring request into a Kafka message queue, searching corresponding agricultural condition remote sensing data in the file system or the database by utilizing Kafka, and performing index calculation on the agricultural condition remote sensing data to obtain an agricultural condition monitoring result.
The technical scheme adopted by the embodiment of the application further comprises the following steps: the data acquisition module is used for acquiring original agricultural condition remote sensing data and specifically comprises the following steps: acquiring original agricultural condition remote sensing data through a satellite remote sensing image, an unmanned aerial vehicle aerial image, a mobile phone shot image or agricultural soil moisture observation and agricultural condition ground investigation data, and storing the original agricultural condition remote sensing data in a local database; the obtained original agricultural condition remote sensing Data is uniformly managed by a Management Engine, and a uniform user interface is provided through a Data Model and a Language.
The technical scheme adopted by the embodiment of the application further comprises the following steps: the data storage module establishes a nuclear sparse representation model for the agricultural condition remote sensing data, generates corresponding nuclear sparse agricultural condition remote sensing data and metadata thereof, and stores the nuclear sparse agricultural condition remote sensing data and the metadata thereof in a file system or a database, wherein the nuclear sparse representation model specifically comprises the following steps: the method comprises the steps of adopting HDFS + Database to store agricultural condition remote sensing big data, firstly establishing kernel sparse representation on the agricultural condition remote sensing data, generating corresponding kernel sparse agricultural condition remote sensing data and metadata thereof, wherein the metadata comprises a kernel sparse data file name, a file type, a storage path, an index path and creation time, then adopting a tile structure to encode the kernel sparse agricultural condition remote sensing data and the metadata thereof, then storing the encoded kernel sparse agricultural condition remote sensing data and the metadata thereof in a file system or a Database, and establishing an index on the kernel sparse agricultural condition remote sensing data.
The technical scheme adopted by the embodiment of the application further comprises the following steps: the data calculation module receives an agricultural condition monitoring request initiated by terminal equipment, puts the agricultural condition monitoring request into a Kafka message queue, searches corresponding agricultural condition remote sensing data in the file system or the database by utilizing Kafka, and performs index calculation on the agricultural condition remote sensing data to obtain an agricultural condition monitoring result, wherein the agricultural condition monitoring result specifically comprises the following steps:
1) the APP sends a request to the Tomcat through the Servlet;
2) WebServer sends a request instruction to Hadoop,
WebServer places the request instruction in Kafka Producer,
kafka Consumer passes the received instruction to Hadoop;
3) the Hadoop/Spark executes the received instruction and sends a required data request to PostgreSQL;
4) the PostgreSQL searches a required data position and initiates a data acquisition request to the HDFS;
5) the HDFS returns data required by Hadoop/Spark processing;
6) performing exponential calculation on Hadoop/Spark;
7) the Hadoop/Spark returns an index calculation result to the HDFS, and the index is built by the PostgreSQL;
8) the Hadoop/Spark returns the index calculation result to WebServer,
Hadoop/Spark returns the result to Kafka Producer,
kafka Consumer gets the output to WebServer.
The technical scheme adopted by the embodiment of the application further comprises a result publishing module, wherein the result publishing module is used for publishing the agricultural condition monitoring result to the agricultural condition monitoring and early warning display platform in real time through micro-service and carrying out agricultural condition drought message early warning according to the agricultural condition monitoring result.
The embodiment of the application adopts another technical scheme that: an electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the one processor to cause the at least one processor to perform the following operations of the agricultural condition monitoring method described above:
step a: acquiring original agricultural condition remote sensing data;
step b: importing the original agricultural condition remote sensing data into a cloud platform, establishing a nuclear sparse representation model for the agricultural condition remote sensing data by the cloud platform, generating corresponding nuclear sparse agricultural condition remote sensing data and metadata thereof, and storing the nuclear sparse agricultural condition remote sensing data and the metadata thereof in a file system or a database;
step c: receiving an agricultural condition monitoring request initiated by terminal equipment, putting the agricultural condition monitoring request into a Kafka message queue, searching corresponding agricultural condition remote sensing data in the file system or the database by utilizing Kafka, and performing index calculation on the agricultural condition remote sensing data to obtain an agricultural condition monitoring result.
Compared with the prior art, the embodiment of the application has the advantages that: the agricultural condition monitoring method, the agricultural condition monitoring system and the electronic equipment realize storage of agricultural condition remote sensing data in a background part, and realize distributed storage of agricultural condition remote sensing big data with nuclear sparse modeling, cross-platform data parallel processing of agricultural condition remote sensing data, cross-platform multi-user concurrent access and agricultural condition release by processing the data through a big data platform. Meanwhile, the interaction between the background and the foreground is realized, the processed result can be checked on the terminal equipment, the communication of the whole route is realized, the agricultural drought hazard is reduced through timely and effective agricultural condition monitoring and early warning, and the disaster early warning and prevention and control capacity is comprehensively improved.
Drawings
FIG. 1 is a flow chart of a method of agricultural condition monitoring according to an embodiment of the present application;
FIG. 2 is a flow chart of agricultural condition monitoring operation request processing;
FIG. 3 is a schematic structural diagram of an agricultural condition monitoring and early warning system according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of hardware equipment of the agricultural condition monitoring method provided in the embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Please refer to fig. 1, which is a flowchart of an agricultural condition monitoring method according to an embodiment of the present application. The agricultural condition monitoring method of the embodiment of the application comprises the following steps:
step 100: acquiring original agricultural condition remote sensing data through a satellite remote sensing image, an unmanned aerial vehicle aerial image, a mobile phone shot image or other agricultural soil moisture observation and agricultural condition ground investigation data, and storing the original agricultural condition remote sensing data in a local database;
in step 100, the obtained original agricultural condition remote sensing Data is managed uniformly by a Management Engine, and a uniform user interface is provided through a Data Model and a Language.
Step 110: importing original agricultural condition remote sensing data into a cloud platform, establishing a nuclear sparse representation model for the non-structural agricultural condition remote sensing data by the cloud platform, generating corresponding nuclear sparse agricultural condition remote sensing data and metadata thereof, and storing the nuclear sparse agricultural condition remote sensing data and the metadata thereof in a file system or a database after coding by adopting a tile structure;
in the step 110, the method and the device adopt HDFS + Database to realize the storage of agricultural remote sensing big data, so that big data frames such as MapReduce or Spark and the like can quickly acquire and process data. In order to express unstructured agricultural condition remote sensing data, a unified data model is required, and a nuclear sparse expression model of unstructured data is adopted. The model firstly establishes kernel sparse representation for unstructured agricultural condition remote sensing data and generates corresponding kernel sparse agricultural condition remote sensing data and metadata thereof, wherein the metadata comprises information such as a kernel sparse data file name, a file type, a storage path, an index path, creation time and the like. According to the method and the device, the tile structure is adopted to encode the data and then store the data in a file system or a database, and indexes are established for the agricultural condition remote sensing data expressed by the kernel sparsity, so that meta search of the agricultural condition remote sensing data can be realized, and the agricultural condition remote sensing data structure is optimized. The method and the device improve the data retrieval capability by designing the efficient row key for the agricultural condition remote sensing data; meanwhile, a secondary index is established for the agricultural condition remote sensing data, and frequently-indexed information is put into the secondary index, so that the retrieval efficiency of the non-row key index information is improved. After the model is abstracted, any agricultural condition remote sensing data can be stored and reconstructed in the system after nuclear sparse representation, and operations such as addition, deletion, query or/and modification based on metadata and content can be realized.
Step 120: adopting YARN to distribute and manage resources for the cloud platform;
in step 120, the present application realizes the fusion of the computing capability of the Linux big data framework and the ArcGIS geographic information data processing capability in the Windows System by using a Shared File System (Distributed Shared File System). And YARN is adopted to distribute and manage resources of the cloud platform, so that the multi-purpose calculation with different requirements is realized, and different agricultural condition predictions are published.
Step 130: driving ArcGIS to process geographic data under a large data programming model such as Spark and the like, storing the result in a shared file system, and accessing the shared file system in real time by a computing frame such as Spark and the like to meet the subsequent data requirement of a program;
in step 130, the Zookeeper cluster is used to maintain the data consistency in the whole process.
Step 140: receiving an agricultural condition monitoring request initiated by terminal equipment, putting the agricultural condition monitoring request into a Kafka message queue, calling computing resources of a computing center by using Kafka, and then responding the agricultural condition monitoring request initiated by the terminal equipment by a big data processing frame to obtain an agricultural condition monitoring result;
in step 140, please refer to fig. 2 together, which is a flow chart of agricultural condition monitoring operation request processing. The specific treatment process comprises the following steps:
data request (first) (solid line part of FIG. 2)
1) The APP sends a request to the Tomcat through the Servlet;
2) WebServer sends a request instruction to Hadoop,
WebServer places the request instruction in Kafka Producer,
kafka Consumer passes the received instruction to Hadoop;
3) the Hadoop/Spark executes the received instruction and sends a required data request to PostgreSQL;
4) PostgreSQL looks up the required data location and initiates a data acquisition request to the HDFS.
(II) result return (dotted line part of FIG. 2)
5) The HDFS returns data required by Hadoop/Spark processing;
6) performing exponential calculation on Hadoop/Spark;
7) the Hadoop/Spark returns a calculation result (index) to the HDFS, and the PostgreSQL establishes an index;
8) the Hadoop/Spark returns the calculation result (index) to WebServer,
Hadoop/Spark returns the result to Kafka Producer,
kafka Consumer obtains an output result to WebServer;
9) and the WebServer returns the result to the APP through the Servlet and displays the result to the APP.
Step 150: releasing the agricultural condition monitoring result to an agricultural condition monitoring and early warning display platform in real time through micro-service, and early warning the agricultural condition drought information according to the agricultural condition monitoring result;
in step 150, the agricultural condition monitoring result is temporarily stored in the Kafka message queue, and meanwhile, the agricultural condition monitoring result can also be directly returned to the terminal device, so that the agricultural condition monitoring result is convenient for the user to check. In the embodiment of the application, the agricultural drought and drought information early warning mode includes but is not limited to short message or voice.
In the embodiment of the application, in the aspects of data resource integration and integration, the architecture of the multi-source agricultural condition remote sensing data management system is realized on the basis of a large data processing platform. The database is primarily designed to collect data, while the synchronized data is primarily used to process data. Due to different purposes, the storage of large data on both sides is not necessarily performed in a homogeneous manner. Therefore, the application mainly aims to support multi-source data processing, and the synchronous structure data and the non-structure data are respectively stored in the database and the distributed file system for unified management, for example, a FUSE file system is adopted. And realizing efficient storage and rapid retrieval of the agricultural condition remote sensing data between the database and the file system, and finishing the logic management of the agricultural condition remote sensing data on the basis.
Please refer to fig. 3, which is a schematic structural diagram of an agricultural condition monitoring system according to an embodiment of the present application. The agricultural condition monitoring system of the embodiment of the application comprises a data acquisition module, a data storage module, a resource allocation module, a data calculation module and a result publishing module.
A data acquisition module: the system is used for acquiring original agricultural condition remote sensing data through a satellite remote sensing image, an unmanned aerial vehicle aerial image, a mobile phone shot image or other agricultural soil moisture observation and agricultural soil moisture ground investigation data, and storing the original agricultural condition remote sensing data in a local database; the obtained original agricultural condition remote sensing Data is uniformly managed by a Management Engine, and a uniform user interface is provided through a Data Model and a Language.
A data storage module: the system comprises a cloud platform, a file system or a database, a kernel sparse representation model, a tile structure and a kernel sparse representation model, wherein the cloud platform is used for importing original agricultural condition remote sensing data into the cloud platform, the cloud platform establishes the kernel sparse representation model for unstructured agricultural condition remote sensing data and generates corresponding kernel sparse agricultural condition remote sensing data and metadata thereof; the method and the device have the advantages that the HDFS + Database is adopted to realize the storage of the agricultural condition remote sensing big data, so that the MapReduce or Spark and other big data frames can rapidly acquire and process the data. In order to express unstructured agricultural condition remote sensing data, a unified data model is required, and a nuclear sparse expression model of unstructured data is adopted. The model firstly establishes kernel sparse representation for unstructured agricultural condition remote sensing data and generates corresponding kernel sparse agricultural condition remote sensing data and metadata thereof, wherein the metadata comprises information such as a kernel sparse data file name, a file type, a storage path, an index path, creation time and the like. According to the method and the device, the tile structure is adopted to encode the data and then store the data in a file system or a database, and indexes are established for the agricultural condition remote sensing data expressed by the kernel sparsity, so that meta search of the agricultural condition remote sensing data can be realized, and the agricultural condition remote sensing data structure is optimized. The method and the device improve the data retrieval capability by designing the efficient row key for the agricultural condition remote sensing data; meanwhile, a secondary index is established for the agricultural condition remote sensing data, and frequently-indexed information is put into the secondary index, so that the retrieval efficiency of the non-row key index information is improved. After the model is abstracted, any agricultural condition remote sensing data can be stored and reconstructed in the system after nuclear sparse representation, and operations such as addition, deletion, query or/and modification based on metadata and content can be realized.
A resource allocation module: the method is used for adopting the YARN to carry out resource allocation and management on the cloud platform, driving ArcGIS to process geographic data under large data programming models such as Spark and the like, storing results in a shared file system, and accessing the shared file system in real time by computing frames such as Spark and the like to meet the subsequent data requirements of a program; the method and the device adopt a Shared File System (Distributed Shared File System) mode to realize the fusion of the computing capacity of a Linux big data frame and the ArcGIS geographic information data processing capacity under a Windows System. And YARN is adopted to distribute and manage resources of the cloud platform, so that the multi-purpose calculation with different requirements is realized, and different agricultural condition predictions are published.
A data calculation module: the system comprises a big data processing frame, a Kafka message queue and a Kafka message queue, wherein the big data processing frame is used for receiving an agricultural condition monitoring request initiated by terminal equipment, putting the agricultural condition monitoring request into the Kafka message queue, calling computing resources of a computing center by using Kafka, and then responding the agricultural condition monitoring request initiated by the terminal equipment by the big data processing frame to obtain an agricultural condition monitoring result; the agricultural condition monitoring operation request processing method specifically comprises the following steps:
data request
1) The APP sends a request to the Tomcat through the Servlet;
2) WebServer sends a request instruction to Hadoop,
WebServer places the request instruction in Kafka Producer,
kafka Consumer passes the received instruction to Hadoop;
3) the Hadoop/Spark executes the received instruction and sends a required data request to PostgreSQL;
4) PostgreSQL looks up the required data location and initiates a data acquisition request to the HDFS.
(II) result return
5) The HDFS returns data required by Hadoop/Spark processing;
6) performing exponential calculation on Hadoop/Spark;
7) the Hadoop/Spark returns a calculation result (index) to the HDFS, and the PostgreSQL establishes an index;
8) the Hadoop/Spark returns the calculation result (index) to WebServer,
Hadoop/Spark returns the result to Kafka Producer,
kafka Consumer obtains an output result to WebServer;
9) and the WebServer returns the result to the APP through the Servlet and displays the result to the APP.
And a result issuing module: the agricultural condition monitoring and early warning system is used for publishing the agricultural condition monitoring result to an agricultural condition monitoring and early warning display platform in real time through the micro-service and carrying out agricultural condition drought message early warning according to the agricultural condition monitoring result; and the result issuing module temporarily stores the agricultural condition monitoring result in a Kafka message queue, and meanwhile, the agricultural condition monitoring result can also be directly returned to the terminal equipment, so that the agricultural condition monitoring result can be conveniently checked by a user. In the embodiment of the application, the agricultural drought and drought information early warning mode includes but is not limited to short message or voice.
Fig. 4 is a schematic structural diagram of hardware equipment of the agricultural condition monitoring method provided in the embodiment of the present application. As shown in fig. 4, the device includes one or more processors and memory. Taking a processor as an example, the apparatus may further include: an input system and an output system.
The processor, memory, input system, and output system may be connected by a bus or other means, as exemplified by the bus connection in fig. 4.
The memory, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules. The processor executes various functional applications and data processing of the electronic device, i.e., implements the processing method of the above-described method embodiment, by executing the non-transitory software program, instructions and modules stored in the memory.
The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data and the like. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory located remotely from the processor, and these remote memories may be connected to the processing system over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input system may receive input numeric or character information and generate a signal input. The output system may include a display device such as a display screen.
The one or more modules are stored in the memory and, when executed by the one or more processors, perform the following for any of the above method embodiments:
step a: acquiring original agricultural condition remote sensing data;
step b: importing the original agricultural condition remote sensing data into a cloud platform, establishing a nuclear sparse representation model for the agricultural condition remote sensing data by the cloud platform, generating corresponding nuclear sparse agricultural condition remote sensing data and metadata thereof, and storing the nuclear sparse agricultural condition remote sensing data and the metadata thereof in a file system or a database;
step c: receiving an agricultural condition monitoring request initiated by terminal equipment, putting the agricultural condition monitoring request into a Kafka message queue, searching corresponding agricultural condition remote sensing data in the file system or the database by utilizing Kafka, and performing index calculation on the agricultural condition remote sensing data to obtain an agricultural condition monitoring result.
The product can execute the method provided by the embodiment of the application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the methods provided in the embodiments of the present application.
Embodiments of the present application provide a non-transitory (non-volatile) computer storage medium having stored thereon computer-executable instructions that may perform the following operations:
step a: acquiring original agricultural condition remote sensing data;
step b: importing the original agricultural condition remote sensing data into a cloud platform, establishing a nuclear sparse representation model for the agricultural condition remote sensing data by the cloud platform, generating corresponding nuclear sparse agricultural condition remote sensing data and metadata thereof, and storing the nuclear sparse agricultural condition remote sensing data and the metadata thereof in a file system or a database;
step c: receiving an agricultural condition monitoring request initiated by terminal equipment, putting the agricultural condition monitoring request into a Kafka message queue, searching corresponding agricultural condition remote sensing data in the file system or the database by utilizing Kafka, and performing index calculation on the agricultural condition remote sensing data to obtain an agricultural condition monitoring result.
Embodiments of the present application provide a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to perform the following:
step a: acquiring original agricultural condition remote sensing data;
step b: importing the original agricultural condition remote sensing data into a cloud platform, establishing a nuclear sparse representation model for the agricultural condition remote sensing data by the cloud platform, generating corresponding nuclear sparse agricultural condition remote sensing data and metadata thereof, and storing the nuclear sparse agricultural condition remote sensing data and the metadata thereof in a file system or a database;
step c: receiving an agricultural condition monitoring request initiated by terminal equipment, putting the agricultural condition monitoring request into a Kafka message queue, searching corresponding agricultural condition remote sensing data in the file system or the database by utilizing Kafka, and performing index calculation on the agricultural condition remote sensing data to obtain an agricultural condition monitoring result.
The agricultural condition monitoring method, the agricultural condition monitoring system and the electronic equipment realize storage of agricultural condition remote sensing data in a background part, and realize distributed storage of agricultural condition remote sensing big data with nuclear sparse modeling, cross-platform data parallel processing of agricultural condition remote sensing data, cross-platform multi-user concurrent access and agricultural condition release by processing the data through a big data platform. Meanwhile, the interaction between the background and the foreground is realized, the processed result can be checked on the terminal equipment, the communication of the whole route is realized, the agricultural drought hazard is reduced through timely and effective agricultural condition monitoring and early warning, and the disaster early warning and prevention and control capacity is comprehensively improved.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (11)
1. An agricultural condition monitoring method is characterized by comprising the following steps:
step a: acquiring original agricultural condition remote sensing data;
step b: importing the original agricultural condition remote sensing data into a cloud platform, establishing a nuclear sparse representation model for the agricultural condition remote sensing data by the cloud platform, generating corresponding nuclear sparse agricultural condition remote sensing data and metadata thereof, and storing the nuclear sparse agricultural condition remote sensing data and the metadata thereof in a file system or a database;
step c: receiving an agricultural condition monitoring request initiated by terminal equipment, putting the agricultural condition monitoring request into a Kafka message queue, searching corresponding agricultural condition remote sensing data in the file system or the database by utilizing Kafka, and performing index calculation on the agricultural condition remote sensing data to obtain an agricultural condition monitoring result.
2. The agricultural condition monitoring method according to claim 1, wherein in the step a, the obtaining of the original agricultural condition remote sensing data specifically comprises: acquiring original agricultural condition remote sensing data through a satellite remote sensing image, an unmanned aerial vehicle aerial image, a mobile phone shot image or agricultural soil moisture observation and agricultural condition ground investigation data, and storing the original agricultural condition remote sensing data in a local database; the obtained original agricultural condition remote sensing data is uniformly managed by a Management Engine, and a uniform user interface is provided through a DataModel and a Language.
3. The agricultural condition monitoring method according to claim 2, wherein in the step b, the establishing, by the cloud platform, a nuclear sparse representation model for the agricultural condition remote sensing data, generating corresponding nuclear sparse agricultural condition remote sensing data and metadata thereof, and storing the nuclear sparse agricultural condition remote sensing data and the metadata thereof in a file system or a database specifically comprises: the method comprises the steps of adopting HDFS + Database to store agricultural condition remote sensing big data, firstly establishing kernel sparse representation on the agricultural condition remote sensing data, generating corresponding kernel sparse agricultural condition remote sensing data and metadata thereof, wherein the metadata comprises a kernel sparse data file name, a file type, a storage path, an index path and creation time, then adopting a tile structure to encode the kernel sparse agricultural condition remote sensing data and the metadata thereof, then storing the encoded kernel sparse agricultural condition remote sensing data and the metadata thereof in a file system or a Database, and establishing an index on the kernel sparse agricultural condition remote sensing data.
4. The agricultural condition monitoring method according to any one of claims 1 to 3, wherein in the step c, the receiving terminal device initiates an agricultural condition monitoring request, the agricultural condition monitoring request is placed in a Kafka message queue, Kafka is used for searching corresponding agricultural condition remote sensing data in the file system or the database, and index calculation is performed on the agricultural condition remote sensing data to obtain an agricultural condition monitoring result specifically includes:
1) the APP sends a request to the Tomcat through the Servlet;
2) WebServer sends a request instruction to Hadoop,
WebServer places the request instruction in Kafka Producer,
kafka Consumer passes the received instruction to Hadoop;
3) the Hadoop/Spark executes the received instruction and sends a required data request to PostgreSQL;
4) the PostgreSQL searches a required data position and initiates a data acquisition request to the HDFS;
5) the HDFS returns data required by Hadoop/Spark processing;
6) performing exponential calculation on Hadoop/Spark;
7) the Hadoop/Spark returns an index calculation result to the HDFS, and the index is built by the PostgreSQL;
8) the Hadoop/Spark returns the index calculation result to WebServer,
Hadoop/Spark returns the result to Kafka Producer,
kafka Consumer gets the output to WebServer.
5. The agricultural condition monitoring method according to claim 4, wherein the step c further comprises: and releasing the agricultural condition monitoring result to an agricultural condition monitoring and early warning display platform in real time through micro-service, and carrying out agricultural condition drought message early warning according to the agricultural condition monitoring result.
6. The utility model provides an agricultural condition monitoring and early warning system which characterized in that includes:
a data acquisition module: the system is used for acquiring original agricultural condition remote sensing data;
a data storage module: the system comprises a cloud platform, a kernel sparse representation model, a file system or a database, wherein the cloud platform is used for importing the original agricultural condition remote sensing data into the cloud platform, establishing the kernel sparse representation model for the agricultural condition remote sensing data by the cloud platform, generating corresponding kernel sparse agricultural condition remote sensing data and metadata thereof, and storing the kernel sparse agricultural condition remote sensing data and the metadata thereof in the file system or the database;
a data calculation module: the agricultural condition monitoring system is used for receiving an agricultural condition monitoring request initiated by terminal equipment, putting the agricultural condition monitoring request into a Kafka message queue, searching corresponding agricultural condition remote sensing data in the file system or the database by utilizing Kafka, and performing index calculation on the agricultural condition remote sensing data to obtain an agricultural condition monitoring result.
7. The agricultural condition monitoring system of claim 6, wherein the data acquisition module acquiring the original agricultural condition remote sensing data specifically comprises: acquiring original agricultural condition remote sensing data through a satellite remote sensing image, an unmanned aerial vehicle aerial image, a mobile phone shot image or agricultural soil moisture observation and agricultural condition ground investigation data, and storing the original agricultural condition remote sensing data in a local database; the obtained original agricultural condition remote sensing data is uniformly managed by a Management Engine, and a uniform user interface is provided through a DataModel and a Language.
8. The agricultural condition monitoring system of claim 7, wherein the data storage module establishes a nuclear sparse representation model for agricultural condition remote sensing data, generates corresponding nuclear sparse agricultural condition remote sensing data and metadata thereof, and stores the nuclear sparse agricultural condition remote sensing data and the metadata thereof in a file system or a database specifically comprises: the method comprises the steps of adopting HDFS + Database to store agricultural condition remote sensing big data, firstly establishing kernel sparse representation on the agricultural condition remote sensing data, generating corresponding kernel sparse agricultural condition remote sensing data and metadata thereof, wherein the metadata comprises a kernel sparse data file name, a file type, a storage path, an index path and creation time, then adopting a tile structure to encode the kernel sparse agricultural condition remote sensing data and the metadata thereof, then storing the encoded kernel sparse agricultural condition remote sensing data and the metadata thereof in a file system or a Database, and establishing an index on the kernel sparse agricultural condition remote sensing data.
9. The agricultural condition monitoring system according to any one of claims 6 to 8, wherein the data calculation module receives an agricultural condition monitoring request initiated by a terminal device, puts the agricultural condition monitoring request into a Kafka message queue, searches corresponding agricultural condition remote sensing data in the file system or the database by using Kafka, and performs index calculation on the agricultural condition remote sensing data to obtain an agricultural condition monitoring result, specifically:
1) the APP sends a request to the Tomcat through the Servlet;
2) WebServer sends a request instruction to Hadoop,
WebServer places the request instruction in Kafka Producer,
kafka Consumer passes the received instruction to Hadoop;
3) the Hadoop/Spark executes the received instruction and sends a required data request to PostgreSQL;
4) the PostgreSQL searches a required data position and initiates a data acquisition request to the HDFS;
5) the HDFS returns data required by Hadoop/Spark processing;
6) performing exponential calculation on Hadoop/Spark;
7) the Hadoop/Spark returns an index calculation result to the HDFS, and the index is built by the PostgreSQL;
8) the Hadoop/Spark returns the index calculation result to WebServer,
Hadoop/Spark returns the result to Kafka Producer,
kafka Consumer gets the output to WebServer.
10. The agricultural condition monitoring system of claim 9, further comprising a result publishing module, wherein the result publishing module is configured to publish the agricultural condition monitoring result to an agricultural condition monitoring and early warning display platform in real time through a micro service, and perform early warning of an agricultural condition drought message according to the agricultural condition monitoring result.
11. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the following operations of the agricultural condition monitoring method of any one of 1 to 5 above:
step a: acquiring original agricultural condition remote sensing data;
step b: importing the original agricultural condition remote sensing data into a cloud platform, establishing a nuclear sparse representation model for the agricultural condition remote sensing data by the cloud platform, generating corresponding nuclear sparse agricultural condition remote sensing data and metadata thereof, and storing the nuclear sparse agricultural condition remote sensing data and the metadata thereof in a file system or a database;
step c: receiving an agricultural condition monitoring request initiated by terminal equipment, putting the agricultural condition monitoring request into a Kafka message queue, searching corresponding agricultural condition remote sensing data in the file system or the database by utilizing Kafka, and performing index calculation on the agricultural condition remote sensing data to obtain an agricultural condition monitoring result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910637226.5A CN110349044A (en) | 2019-07-15 | 2019-07-15 | A kind of agriculture feelings monitoring method, system and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910637226.5A CN110349044A (en) | 2019-07-15 | 2019-07-15 | A kind of agriculture feelings monitoring method, system and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110349044A true CN110349044A (en) | 2019-10-18 |
Family
ID=68176359
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910637226.5A Pending CN110349044A (en) | 2019-07-15 | 2019-07-15 | A kind of agriculture feelings monitoring method, system and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110349044A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110955731A (en) * | 2019-12-13 | 2020-04-03 | 深圳先进技术研究院 | Multi-source remote sensing big data processing method and device based on Chord ring |
| CN113761012A (en) * | 2020-06-05 | 2021-12-07 | 义乌市时空人工智能有限公司 | Analysis visualization method of remote sensing data, server and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160178590A1 (en) * | 2014-12-23 | 2016-06-23 | Electronics And Telecommunications Research Institute | System and method for predicting harmful materials |
| CN108287926A (en) * | 2018-03-02 | 2018-07-17 | 宿州学院 | A kind of multi-source heterogeneous big data acquisition of Agro-ecology, processing and analysis framework |
| CN109635068A (en) * | 2018-12-20 | 2019-04-16 | 中国地质大学（武汉） | Mass remote sensing data high-efficiency tissue and method for quickly retrieving under cloud computing environment |
-
2019
- 2019-07-15 CN CN201910637226.5A patent/CN110349044A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160178590A1 (en) * | 2014-12-23 | 2016-06-23 | Electronics And Telecommunications Research Institute | System and method for predicting harmful materials |
| CN108287926A (en) * | 2018-03-02 | 2018-07-17 | 宿州学院 | A kind of multi-source heterogeneous big data acquisition of Agro-ecology, processing and analysis framework |
| CN109635068A (en) * | 2018-12-20 | 2019-04-16 | 中国地质大学（武汉） | Mass remote sensing data high-efficiency tissue and method for quickly retrieving under cloud computing environment |
Non-Patent Citations (5)
| Title |
|---|
| 吴上: "核稀疏表示相关问题研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 方堃: "基于稀疏字典学习和核稀疏表示的图像超分辨率重建", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 曹恒瑞等: "一种基于Hadoop平台的分布式数据检索系统", 《软件导刊》 * |
| 谢杨: "基于云计算的现代农业物联网监控系统", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 高万林: "《新农村信息化探索》", 31 March 2011 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110955731A (en) * | 2019-12-13 | 2020-04-03 | 深圳先进技术研究院 | Multi-source remote sensing big data processing method and device based on Chord ring |
| CN113761012A (en) * | 2020-06-05 | 2021-12-07 | 义乌市时空人工智能有限公司 | Analysis visualization method of remote sensing data, server and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106649670B (en) | Data monitoring method and device based on stream computing | |
| Yang et al. | Design and implementation of cloud platform for intelligent logistics in the trend of intellectualization | |
| CN103399887A (en) | Query and statistical analysis system for mass logs | |
| US11366809B2 (en) | Dynamic creation and configuration of partitioned index through analytics based on existing data population | |
| CN110019267A (en) | A kind of metadata updates method, apparatus, system, electronic equipment and storage medium | |
| CN105824974B (en) | The method and system of Data Analysis Services | |
| CN104331421A (en) | High-efficiency processing method and system for big data | |
| CN103646073A (en) | Condition query optimizing method based on HBase table | |
| CN109194762B (en) | SSH-based soil information network publishing system and method | |
| CN105183834A (en) | Ontology library based transportation big data semantic application service method | |
| US11188443B2 (en) | Method, apparatus and system for processing log data | |
| CN111258978A (en) | Data storage method | |
| US10657099B1 (en) | Systems and methods for transformation and analysis of logfile data | |
| CN103514205A (en) | Mass data processing method and system | |
| CN107798059B (en) | NCO meteorological data structured storage method and device | |
| EP4170498A2 (en) | Federated learning method and apparatus, device and medium | |
| CN110349044A (en) | A kind of agriculture feelings monitoring method, system and electronic equipment | |
| CN111753034A (en) | One-stop type geographical big data platform | |
| CN103034650B (en) | A kind of data handling system and method | |
| CN116594836A (en) | Cloud protogenesis-based unified monitoring method and device for multi-cloud heterogeneous resources | |
| CN112100179A (en) | HBASE-based data fusion method, HBASE-based data fusion device, HBASE-based data fusion equipment and computer readable medium | |
| Hajji et al. | Optimizations of Distributed Computing Processes on Apache Spark Platform. | |
| CN117591025B (en) | Multi-source heterogeneous data processing system | |
| CN108319604B (en) | Optimization method for association of large and small tables in hive | |
| CN110083598A (en) | A kind of remotely-sensed data indexing means, system and electronic equipment towards Spark |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191018 |
|
| RJ01 | Rejection of invention patent application after publication |