CN111986042A - Agricultural big data service system constructed based on internet technology - Google Patents
Agricultural big data service system constructed based on internet technology Download PDFInfo
- Publication number
- CN111986042A CN111986042A CN202010859823.5A CN202010859823A CN111986042A CN 111986042 A CN111986042 A CN 111986042A CN 202010859823 A CN202010859823 A CN 202010859823A CN 111986042 A CN111986042 A CN 111986042A
- Authority
- CN
- China
- Prior art keywords
- data
- information
- agricultural
- platform
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000005516 engineering process Methods 0.000 title claims abstract description 30
- 238000007726 management method Methods 0.000 claims abstract description 55
- 238000004458 analytical method Methods 0.000 claims abstract description 32
- 238000012545 processing Methods 0.000 claims abstract description 32
- 101001072091 Homo sapiens ProSAAS Proteins 0.000 claims abstract description 22
- 101150096185 PAAS gene Proteins 0.000 claims abstract description 22
- 102100036366 ProSAAS Human genes 0.000 claims abstract description 22
- 238000000034 method Methods 0.000 claims abstract description 19
- 230000008569 process Effects 0.000 claims abstract description 14
- 238000003860 storage Methods 0.000 claims description 17
- 238000004364 calculation method Methods 0.000 claims description 13
- 238000006243 chemical reaction Methods 0.000 claims description 13
- 238000011161 development Methods 0.000 claims description 13
- 244000144972 livestock Species 0.000 claims description 12
- 238000012544 monitoring process Methods 0.000 claims description 12
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 10
- 230000008520 organization Effects 0.000 claims description 9
- 239000002689 soil Substances 0.000 claims description 9
- 230000007613 environmental effect Effects 0.000 claims description 7
- 238000012986 modification Methods 0.000 claims description 7
- 230000004048 modification Effects 0.000 claims description 7
- 238000009313 farming Methods 0.000 claims description 6
- 238000004519 manufacturing process Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000009395 breeding Methods 0.000 claims description 5
- 230000001488 breeding effect Effects 0.000 claims description 5
- 238000005065 mining Methods 0.000 claims description 5
- 235000013305 food Nutrition 0.000 claims description 4
- 239000012298 atmosphere Substances 0.000 claims description 3
- 238000011068 loading method Methods 0.000 claims description 3
- 230000001932 seasonal effect Effects 0.000 claims description 3
- 239000010865 sewage Substances 0.000 claims description 3
- 238000003307 slaughter Methods 0.000 claims description 3
- 238000012937 correction Methods 0.000 claims description 2
- 230000010354 integration Effects 0.000 claims description 2
- 238000013500 data storage Methods 0.000 abstract description 6
- 238000012549 training Methods 0.000 description 13
- 230000018109 developmental process Effects 0.000 description 10
- 230000007246 mechanism Effects 0.000 description 4
- 238000012271 agricultural production Methods 0.000 description 3
- 238000004140 cleaning Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 241000607479 Yersinia pestis Species 0.000 description 2
- 239000012773 agricultural material Substances 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000000306 component Substances 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000033772 system development Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/02—Agriculture; Fishing; Forestry; Mining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2216/00—Indexing scheme relating to additional aspects of information retrieval not explicitly covered by G06F16/00 and subgroups
- G06F2216/03—Data mining
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Marine Sciences & Fisheries (AREA)
- General Health & Medical Sciences (AREA)
- Agronomy & Crop Science (AREA)
- Mining & Mineral Resources (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- Animal Husbandry (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses an agricultural big data service system constructed based on the internet technology, which comprises an SAAS layer, a DAAS layer, a PAAS layer, a data processing and analyzing platform and a background management platform which are established; the SAAS layer uploads the acquired data and collects and summarizes the information, and the summarized information enters an agricultural management platform for collecting information of the SAAS resume; the DAAS layer carries out large database analysis sharing on the information collected by the SAAS layer to form platform resources; the PAAS layer performs geographic position distributed data storage on large database analysis sharing platform resources of the DAAS layer, then integrates the data, calculates and processes the integrated data, and reads initialization data into a background management platform for background reading and external supervision. The method and the system uniformly collect information and establish and analyze the internal and external big data through the internet big data, realize the prejudgment of the internet big data service system on agriculture, and have great significance for mastering the internal relation and rule of agricultural information.
Description
Technical Field
The invention belongs to the technical field of big data, and particularly relates to an agricultural big data service system constructed based on an internet technology.
Background
After cloud computing and the internet of things, big data technology is undoubtedly one of leading-edge technologies which again raise important changes of information technology, and the advantages of big data technology on the aspects of data mining and intelligent application are shown, so that the world is advanced to the big data era. The technology has great influence on social management, development prediction, enterprise and department decision and various social fields. China is a big agricultural country, and the solution of problems of agriculture, rural areas and farmers is related to the vital interests of people, the stability of society and the development of the whole national economy. The agricultural data coverage area is wide, the data source is complex, and agricultural production is suitable for the problems of acquisition, mining, storage, processing and application of a large amount of continuously generated structured and unstructured data.
The agricultural data comprises various links such as planting, breeding, agricultural science and technology, agricultural equipment, meteorology, land, water conservancy, agricultural materials, pest control, ecological environment, food safety, agricultural product processing, agricultural product transaction and the like. Therefore, the agricultural data has the characteristics of large capacity, strong relevance, complexity, variability and the like, and the agricultural large data system is wide in source, various in type, complex in structure and potential in value after self characteristics such as agricultural regionality, seasonality, diversity and periodicity are fused, and a data set processed and analyzed by a common method is difficult to apply.
The existing agricultural data collection mode is backward, the data collection is mainly manual labor, the data quantity is small, the variability is poor, the labor intensity is high, the data general investigation period is long, particularly, the fragmentation of the data is realized, the data is analyzed, and the data sharing and utilization value is not high. Through internet big data, collect in good time through a large amount of terminals on line, can carry out more abundant matching with the plain resource, can convert data into powerful productivity.
The continuous deep development and application of precision agriculture, intelligent agriculture, internet of things and cloud computing enable the data volume in the aspect of agriculture to be increased day by day and contain value, so that how to construct an agricultural big data service system by using the internet technology becomes the direction of efforts of technicians in the field.
Disclosure of Invention
The invention aims to provide an agricultural big data service system constructed based on the internet technology, valuable data and knowledge are searched and mined from a huge data set, the practice and application of the big data technology in the agricultural field are promoted, and the method has great significance for mastering the internal connection and rule of agricultural information.
The purpose of the invention is realized by the following technical scheme: an agricultural big data service system is constructed based on the Internet technology, and comprises an SAAS layer, a DAAS layer, a PAAS layer, a data processing and analyzing platform and a background management platform which are established;
the SAAS layer self-defines keywords, acquires data through various terminal devices, uploads the acquired data and collects and summarizes information, and the summarized information enters an agricultural management platform of SAAS resume collected information;
the DAAS layer carries out large database analysis sharing on the information collected by the SAAS layer to form platform resources;
the PAAS layer performs geographic position distributed data storage on large database analysis sharing platform resources of the DAAS layer, then integrates the data, calculates and processes the integrated data, performs data conversion and unified management and control on the processed data, and respectively displays the data on an internal data platform and an external data platform after the data are managed and controlled;
comparing the data displayed by the PAAS layer with the initial resource information, entering the unsuccessfully compared data into the distributed storage data again for storing again, and initializing the container resource by the successfully compared data through the data processing and analyzing platform;
and reading the initialization data into a background management platform for background reading and external supervision.
As a preferred mode, the data types collected by the various terminal devices include: planting industry information, animal husbandry information, animal slaughter house information, farming organization information, agricultural investment information, scientific and technical education information, energy environmental protection information, poverty-relief development information, agricultural product market information and agricultural element market information.
As one of the preferable modes, among them: the planting industry information relates to information collection of agricultural ecological environment, including soil, atmosphere, water quality, weather, pollution and disaster information collection, the influence factors of the agricultural environment are comprehensively monitored, and the analysis and the collection of the composition information of soil in the soil, the analysis and the collection of seasonal rainfall information, the analysis and the collection of the information of soil planting types, the analysis and the collection of yield information, the analysis and the collection of natural disaster information and the analysis and the collection of the production cost of agricultural products are established;
the animal husbandry information relates to animal husbandry sites, animal husbandry quantity, environment, air, temperature, pollution and water quality information, the site size, the intelligent degree, the environment and pollution conditions, local air, temperature conditions and water quality conditions in the animal husbandry breeding process are collected, and influence factors influencing animal husbandry development are comprehensively monitored;
the livestock killing farm information relates to the size and scale of the killing farm, food safety problems in the killing process, and sewage environment treatment problems in the killing process and after killing, and a camera is arranged in the livestock killing farm for unified monitoring and standardized management;
in the service and management department, data is transmitted by issuing agricultural and computerized information, agricultural investment information, scientific and technological education information, energy and environmental protection information and poverty relief development information.
As a preferred mode, the procedure of PAAS layer calculation and processing includes the following steps: preprocessing data source data information, then carrying out format check and conversion, checking out error data, putting correct data into a temporary storage area for temporary storage, then extracting a clean conversion loading module for correction and conversion into a detailed data layer, then extracting, converting and summarizing into a plurality of layers again, and putting the summarized data layer into a data warehouse.
As a preferred mode, the data processing and analyzing platform: preprocessing the collected information, checking the resource information, and registering and uploading data errors; establishing a structured database storage, an MPP and a complete relational database; calculating and summarizing to perform secondary calculation, calculation analysis, error report and real-time data processing on all data; processing unstructured big data resources and establishing a related database; and collecting, summarizing, analyzing and mining the total data.
As a preferable mode, the display of the internal data platform includes real-time monitoring information, early warning notification information, working logs registered and uploaded in the platform, and the like of the system, the registration, uploading, modification and acquisition of management authority of data can be set by the internal data platform, and different registration, uploading, modification and management authorities are set by different people in different categories.
As a preferable mode, the external data platform display comprises the supervision of the official certification and the collection of suggestions made by the peasant household and the only channel and the official certification; data are presented in the form of lists, curves, bars, pie charts, gantt charts.
As an optimal mode, data of the external data platform is uploaded and updated in real time, and a working log of the external service PAAS is established for timely checking by management personnel. External supervisors enter through the channel of official website authentication to supervise the whole service system, can provide suggestions according to external observation, collect the provided suggestions, generate working logs after collection, upload the collected working logs in real time and upload the working logs to unified management.
As a preferred mode, the technical framework of the data processing and analyzing platform adopts a Hadoop technology.
The users of the internal data are mainly providers of agricultural public services, including farmers, farmer cooperation organizations, agricultural material supply organizations, agricultural product processing enterprises and agricultural socialized service organizations. The internal data platform provides data services in the aspects of basic data, technical popularization data, information service data, agricultural market data, policy and law and regulation of infrastructure, and integrates the services before, during and after agricultural production and sale. The advanced technology of big data is applied to perform a large amount of data analysis processing and support on each main production process of agriculture, so that accurate and intelligent management decisions are provided, and the purposes of increasing the yield of agriculture and increasing the income of farmers are achieved.
Compared with the prior art, the invention has the beneficial effects that: the agricultural big data service system based on the Internet technical component uses an intelligent frame, and starts from resource-service-management-application, multi-directional data in the district are uniformly collected through the Internet to establish and analyze internal and external big data, so that the agricultural prejudgment of the Internet big data service system is realized, and meanwhile, the agricultural big data service system can be supervised by the outside and can be used for collecting suggestions from the outside, and the agricultural big data service system has great significance for mastering the internal contact and rule of agricultural information.
Drawings
FIG. 1 is a schematic flow chart of the agricultural big data service system of the invention.
FIG. 2 is a flow chart illustrating the data calculation and processing of FIG. 1 according to the present invention.
Fig. 3 is a system diagram of the PAAS layer of fig. 1 according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1-3, an agricultural big data service system is constructed based on the internet technology, and includes an established SAAS layer, a DAAS layer, a PAAS layer, a data processing and analyzing platform, and a background management platform.
Wherein: the SAAS layer defines keywords by self and defines corresponding keyword information according to specific user requirements. The SAAS layer collects data through various terminal devices, uploads the collected data and collects and summarizes information, and the summarized information enters an agricultural management platform of SAAS resume collected information.
The data collected by various terminal devices include: animal husbandry information, crop farming information, farming organization information, agricultural investment information, scientific and technological education information, energy and environmental protection information, poverty alleviation development information, agricultural product market, and other information, such as agricultural product market information and agricultural element market information.
For example, in the management jurisdiction, there are basic agricultural information such as planting industry and livestock farms, basic processing information such as livestock slaughter farms, service institutions such as agricultural machinery service institutions, agricultural machinery training institutions, agricultural investment institutions and poverty relief institutions, service institutions such as environment management institutions and market supervision management institutions. In order to acquire comprehensive information, sensors and cameras are arranged in production units of basic agriculture, such as fruit tree sites, fields and livestock breeding sites, cameras are arranged in livestock killing sites, and information acquired by the sensors and the cameras is collected uniformly by taking each farmer and each killing site as a unit; terminals, such as computers or smart phones, are provided in service, management and supervision departments for uploading and publishing information.
Wherein, the information of planting industry involves the information acquisition of agricultural ecological environment, specifically includes soil, atmosphere, quality of water, meteorological phenomena, pollution, calamity etc. need realize comprehensive monitoring to these agricultural environment's influence factor, just can realize accurate management. The planting industry information comprises the steps of establishing and collecting the analysis and the collection of the composition information of soil, the analysis and the collection of seasonal rainfall information, the analysis and the collection of soil planting type information, the analysis and the collection of yield information, the analysis and the collection of natural disaster information and the analysis and the collection of agricultural product production cost.
The information of livestock farm relates to the animal husbandry place, the livestock quantity, the environment, the air, the temperature, the pollution, the quality of water etc. need carry out comprehensive monitoring to influence factors such as the place size, intelligent degree, environment and the pollution condition, local air, temperature condition and the quality of water condition among the animal husbandry farming breeding process, just can realize the management of accuracy.
The information of the livestock killing field relates to the size and scale of the killing field, the food safety problem in the killing process, the environmental treatment problems of sewage in the killing process and after killing and the like, and a camera is arranged in the livestock killing field for unified monitoring and standardized management.
In service, management and supervision departments such as agricultural machinery service organizations, agricultural machinery training organizations, agricultural investment organizations, poverty relief organizations, environment management organizations and market supervision and management organizations, data are transmitted by means of terminal information distribution.
Wherein: the information of the agricultural machinery service mechanism relates to the collection of farmland mechanization degree information, the collection of information such as mechanical and intelligent machinery contained in each farmer and each agricultural enterprise, the release of new agricultural machinery in the existing market and the jurisdiction and the like is carried out, the agricultural machinery service mechanism can be an agricultural management department in the jurisdiction or an agricultural organization association, new intelligent information of the existing agricultural machinery can be known in a network mode, and the new information can be released to a platform through the service system for the first time.
The information of the agricultural machinery training mechanism relates to training of agricultural machinery operation of farmers, training of field management of the farmers and training of planting and pest management of the farmers, corresponding training information can be issued according to local real-time conditions, for example, a certain crop produced in a certain place can improve yield under scientific field management, in order to popularize the technology, corresponding concentrated training consultation can be issued through the agricultural machinery training mechanism, so that compared with a traditional mode, the training cost can be saved, and the training effect can be improved through concentrated training and concentrated guidance.
The information of agricultural investment institutions, poverty-relief institutions and the like is mainly information in aspects of providing funds for farmers and the like, helps the farmers to perform upgrading services such as production expansion and the like, and seeks more benefits for agricultural development in jurisdictions.
Management departments such as an environment management organization, a market supervision management organization and the like mainly provide policy file information and information such as management supervision, publication and the like, and because the data acquisition information is disclosed and transparent, the environment management organization and the market supervision management organization can supervise and obtain evidence more conveniently.
The service and management department can carry out unified management by issuing information, for example, when the agricultural machinery is busy in the farming and the emergency situation that the agricultural machinery is insufficient occurs, the agricultural machinery can be intensively allocated and called in time, the intelligent degree of the agricultural machinery in the district can be known, and the intelligent level of the agricultural machinery and the skill of farmers can be improved by issuing the intelligent information of the agricultural machinery and training.
Besides, agricultural investment information, poverty-relief development information, scientific and technological education information, energy environmental protection information, agricultural product market information and agricultural element market information are mainly used for providing service information of consultation classes, the near conditions of the market can be timely fed back to agriculture and animal husbandry, overall management can be intensively provided for agricultural scientific and technological education, meanwhile, poverty-relief policies, development information, market information and the like can be timely embodied in the big data service system in the first time, and a user can conveniently master one-hand resource information.
In addition, the hand-held agricultural land information of each farmer is analyzed and summarized, and the hand-held agricultural land annual expenditure and income registration information of each farmer is summarized; the hand-held agricultural land information analysis of each farmer needs to accurately register the number of the farmer population, the sex, the age, the relationship with the farmer population and the health report of the farmer population according to the household account book. The method is used for managing and controlling basic information of farmers in the district.
After the SAAS layer collects basic data, the DAAS layer carries out large database analysis sharing on the information collected by the SAAS layer to form platform resources. DAAS is a digital audio analysis system, is a test system based on PC, and supports DOS, android and WINDOWS operating systems. The data information formed by the SAAS layer can be intuitively known on the terminal through the DAAS layer, and the data information is shared in a digital audio mode.
The Platform developed by software is used as a service, and is submitted to a user in a SAAS mode. The PAAS mainly has the following three characteristics:
1. platform as a service: PAAS provides a basic platform, a special platform service provider builds and operates the basic platform, and provides the platform to an application system operator in a service mode; the services required by the PAAS operator include not only a simple basic platform but also technical support services for the platform and services such as application system development and optimization for the platform. The platform-level service can ensure that various application systems of the SAAS service provider can be supported to run stably for a long time.
The PAAS layer performs geographic position distributed data storage on large database analysis sharing platform resources of the DAAS layer, then performs administrative region integration on data transmitted by the DAAS layer, performs layered calculation and processing on the integrated data, and performs data conversion and unified management and control on the processed data. And the data are respectively displayed on an internal data platform and an external data platform after being controlled.
Wherein: the distributed storage data comprises structured data storage and unstructured/big data storage, event processing is acquired in real time, data processing is performed in real time, and ETL is adopted for data acquisition, cleaning, summarizing and label mining.
The PAAS layer calculation and processing process comprises the following steps: preprocessing data source data information, wherein the data source needs to be processed into a uniform format according to different information acquired by the SAAS layer; and carrying out format check and conversion, converting the format into a uniform system readable format, checking error data, and independently placing the error data in a specific data area for viewing. The correct data are placed in a temporary storage area for temporary storage, then the extraction, cleaning, conversion and loading module is used for correcting and converting the correct data into a detailed data layer, then extraction, conversion and summarization are carried out again to form a plurality of layers, the summarized data layer is placed in a data warehouse, and the process of extraction, cleaning and conversion is carried out on the received various information.
In PAAS, the data processing analysis platform: preprocessing the collected information, checking the resource information, and registering and uploading data errors; establishing a structured database storage, an MPP and a complete relational database; calculating and summarizing to perform secondary calculation, calculation analysis, error report and real-time data processing on all data; processing unstructured big data resources and establishing a related database; and collecting, summarizing, analyzing and mining the total data to form database resources of big data.
Comparing the data displayed by the PAAS layer with the initial resource information, entering the unsuccessfully compared data into the distributed storage data again for storing again, and initializing the container resource by the successfully compared data through the data processing and analyzing platform; and reading the initialization data into a background management platform for background reading and external supervision.
In the SAAP, set information resources are all information such as various languages, audios and videos, images, charts, numbers, sensor acquisition information and the like generated in the agricultural production activities of people, and the information is subjected to data management, so that not only is massive information stored, but also the information is applied and processed as a whole through a big data technology.
The internal data service platform is provided with a unified interface, a permission management unit, a task scheduling unit, a service monitoring unit, a data export unit and a data import unit. The internal data service platform is externally connected with other visual terminal machines through a network open interface, wherein the visual terminal machines comprise an integrated terminal machine, a computer, a mobile phone, an ipad and the like. The data registration, uploading, modification and management authority can be acquired by setting an internal data platform and setting different registration, uploading, modification and management authorities according to different people classes. The external data platform display comprises a unique channel of official authentication, an official and a farmer supervision of service and a collection of suggestions for the outside world. The content of the display is set according to the administrative requirement of the district in the display of the internal data platform, and the content can comprise real-time monitoring information, early warning notification information, working logs registered and uploaded in the platform and the like of the system. Different display contents are set according to different login authorities so as to realize classification management and facilitate the separation of the management of big data and users.
The external data service platform is provided with an authentication unit, a service monitoring unit, a data pushing unit, a metering unit, a task scheduling unit and a data encryption unit. The authentication unit is used for distinguishing the authority, and the service monitoring unit is used for monitoring the service content and is mainly used for a third-party supervision department; the data pushing unit, the metering unit and the task scheduling unit are used for processing data; the data encryption unit is used for setting authority such as data transfer and transmission, and preventing confidential data from leaking. And uploading and updating the data of the external data platform in real time, and establishing a working log of the external service PAAS for managers with authority to check in time. External supervisors enter through official website authentication channels to supervise the whole service system, can provide suggestions according to external observation, collect the provided suggestions, generate working logs after collection, upload the collected working logs in real time to form feedback data, and transmit the feedback data back to the management platform through an external data service platform.
The whole technical architecture of the big data service system adopts Hadoop technology. Hadoop is a distributed system infrastructure of Apache open source organization, wherein core components comprise a distributed file system (HDFS) (HadoopDistributedFileSystems) and a distributed computing MapReduce, and the Hadoop is one of the defects and the defects. The Hadoop distributed file system HDFS is deployed on a universal and low-cost storage server, a hardware architecture design of a master-slave structure is adopted, a master node is called NameNode, a working node is composed of a plurality of DataNodes, and the Hadoop distributed file system HDFS provides a mass data storage solution with high fault tolerance and high throughput. The HDFS subverts the traditional storage architecture, a plurality of data blocks are stored in the bottom hardware device in a distributed mode, and data access and calculation are carried out through a MapReduce distributed algorithm. The HDFS had the following characteristics:
1. the expansibility is strong, and the system online capacity expansion can be carried out according to the service requirement: the NameNode main node is responsible for storing data block distribution management information, and the DataNode working node stores the data block information, so that the capacity expansion requirement of the system can be completed only by adding 175 DataNode nodes, and the main node automatically brings in newly added DataNode nodes. Through the system, the HDFS can realize real-time online dynamic capacity expansion without manpower.
2. High reliability, automatic data detection and replication: the HDFS file system is repeated when data is written in, and is repeated immediately after errors are found in verification in reading and writing, so that the system continuously and automatically verifies the consistency and the data of the data to guarantee the system.
3. Access bottleneck is eliminated, and high throughput is achieved: the user accesses the system, and the HDFS stores the data blocks in a hardware device in a distributed manner, so that the HDFS selects the system with the minimum access amount and the maximum network routing path. When accessing large files, the system can increase data access, and each copy of the data block can be provided for users to access, so the access performance of the HDFS to a single data block is the traditional access performance, and the single device can access the throughput through a distributed algorithm.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (9)
1. An agricultural big data service system is constructed based on the internet technology, which is characterized in that: the system comprises an SAAS layer, a DAAS layer, a PAAS layer, a data processing and analyzing platform and a background management platform which are established;
the SAAS layer self-defines keywords, acquires data through various terminal devices, uploads the acquired data and collects and summarizes information, and the summarized information enters an agricultural management platform of SAAS resume collected information;
the DAAS layer carries out large database analysis sharing on the information collected by the SAAS layer to form platform resources;
the PAAS layer performs geographic position distributed storage data on large database analysis sharing platform resources of the DAAS layer, then performs administrative region integration on data transmitted by the DAAS layer, performs layered calculation and processing on the integrated data, performs data conversion and unified management and control on the processed data, and respectively displays the data on an internal data platform and an external data platform after the data are managed and controlled;
comparing the data displayed by the PAAS layer with the initial resource information, entering the unsuccessfully compared data into the distributed storage data again for storing again, and initializing the container resource by the successfully compared data through the data processing and analyzing platform;
and reading the initialization data into a background management platform for background reading and external supervision.
2. The internet technology based agricultural big data service system built according to claim 1, wherein: the types of data collected by various terminal devices include: planting industry information, animal husbandry information, animal slaughter house information, farming organization information, agricultural investment information, scientific and technical education information, energy environmental protection information, poverty-relief development information, agricultural product market information and agricultural element market information.
3. The internet technology based agricultural big data service architecture as claimed in claim 2, wherein: the planting industry information relates to information collection of agricultural ecological environment, including soil, atmosphere, water quality, weather, pollution and disaster information collection, the influence factors of the agricultural environment are comprehensively monitored, and analysis and collection of composition information of soil in collected land, analysis and collection of seasonal rainfall information, analysis and collection of land planting type information, analysis and collection of yield information, analysis and collection of natural disaster information and analysis and collection of agricultural product production cost are established;
the animal husbandry information relates to animal husbandry sites, animal husbandry quantity, environment, air, temperature, pollution and water quality information, the site size, the intelligent degree, the environment and pollution conditions, local air, temperature conditions and water quality conditions in the animal husbandry breeding process are collected, and influence factors influencing animal husbandry development are comprehensively monitored;
the livestock killing farm information relates to the size and scale of the killing farm, food safety problems in the killing process, and sewage environment treatment problems in the killing process and after killing, and a camera is arranged in the livestock killing farm for unified monitoring and standardized management;
in the service and management department, data is transmitted by issuing agricultural and computerized information, agricultural investment information, scientific and technological education information, energy and environmental protection information and poverty relief development information.
4. The internet technology based agricultural big data service system built according to claim 1, wherein: the PAAS layer calculation and processing process comprises the following steps: preprocessing data source data information, then carrying out format check and conversion, checking out error data, putting correct data into a temporary storage area for temporary storage, then extracting a clean conversion loading module for correction and conversion into a detailed data layer, then extracting, converting and summarizing into a plurality of layers again, and putting the summarized data layer into a data warehouse.
5. The internet technology based agricultural big data service system built according to claim 1, wherein: the data processing and analyzing platform comprises: preprocessing the collected information, checking the resource information, and registering and uploading data errors; establishing a structured database storage, an MPP and a complete relational database; calculating and summarizing to perform secondary calculation, calculation analysis, error report and real-time data processing on all data; processing unstructured big data resources and establishing a related database; and collecting, summarizing, analyzing and mining the total data.
6. The internet technology based agricultural big data service system built according to claim 1, wherein: the internal data platform display comprises real-time monitoring information, early warning notification information and working logs which are registered and uploaded in the platform, the data registration, uploading, modification and management authority can be set through the internal data platform, and different people are classified to set different registration, uploading, modification and management authorities.
7. The internet technology based agricultural big data service system built according to claim 1, wherein: the external data platform display comprises a unique channel of official authentication, an official and the supervision of farmers on the service and the collection of suggestions proposed to the outside; data are presented in the form of lists, curves, bars, pie charts, gantt charts.
8. The internet technology based agricultural big data service architecture as claimed in claim 7, wherein: uploading and updating the data of the external data platform in real time, and establishing a working log of the external service PAAS for managers to check in time; external supervisors enter through the channel of official website authentication to supervise the whole service system, can provide suggestions according to external observation, collect the provided suggestions, generate working logs after collection, upload the collected working logs in real time and upload the working logs to unified management.
9. The internet technology based agricultural big data service system built according to claim 1, wherein: the technical architecture of the data processing and analyzing platform adopts a Hadoop technology.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010859823.5A CN111986042A (en) | 2020-08-24 | 2020-08-24 | Agricultural big data service system constructed based on internet technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010859823.5A CN111986042A (en) | 2020-08-24 | 2020-08-24 | Agricultural big data service system constructed based on internet technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111986042A true CN111986042A (en) | 2020-11-24 |
Family
ID=73443113
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010859823.5A Pending CN111986042A (en) | 2020-08-24 | 2020-08-24 | Agricultural big data service system constructed based on internet technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111986042A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112732670A (en) * | 2020-12-31 | 2021-04-30 | 石河子大学 | Agricultural resource integration method and system based on network big data |
| CN112861905A (en) * | 2020-12-31 | 2021-05-28 | 杭州金时科技有限公司 | Tree species classification platform based on internet |
| WO2023004881A1 (en) * | 2021-07-28 | 2023-02-02 | 深圳市爱云信息科技有限公司 | Smart agriculture aiot distributed big data storage platform |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106780239A (en) * | 2016-11-18 | 2017-05-31 | 常州市农畜水产品质量监督检验测试中心 | Agricultural product quality and safety supervises public notification of information system |
| CN109726848A (en) * | 2018-11-20 | 2019-05-07 | 江苏智途科技股份有限公司 | A kind of wisdom agricultural big data service platform |
| CN110019141A (en) * | 2017-12-30 | 2019-07-16 | 广州明领基因科技有限公司 | Big data analysis system towards university educational management decision |
| CN110032556A (en) * | 2019-03-14 | 2019-07-19 | 云南昆钢电子信息科技有限公司 | A kind of method and its system that can fast implement data cleansing, verification of correctness |
| CN110457333A (en) * | 2019-07-29 | 2019-11-15 | 招商局金融科技有限公司 | Data real time updating method, device and computer readable storage medium |
| CN110555067A (en) * | 2018-03-30 | 2019-12-10 | 南京理工大学 | method for collecting, processing and integrally managing assembly quality data |
-
2020
- 2020-08-24 CN CN202010859823.5A patent/CN111986042A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106780239A (en) * | 2016-11-18 | 2017-05-31 | 常州市农畜水产品质量监督检验测试中心 | Agricultural product quality and safety supervises public notification of information system |
| CN110019141A (en) * | 2017-12-30 | 2019-07-16 | 广州明领基因科技有限公司 | Big data analysis system towards university educational management decision |
| CN110555067A (en) * | 2018-03-30 | 2019-12-10 | 南京理工大学 | method for collecting, processing and integrally managing assembly quality data |
| CN109726848A (en) * | 2018-11-20 | 2019-05-07 | 江苏智途科技股份有限公司 | A kind of wisdom agricultural big data service platform |
| CN110032556A (en) * | 2019-03-14 | 2019-07-19 | 云南昆钢电子信息科技有限公司 | A kind of method and its system that can fast implement data cleansing, verification of correctness |
| CN110457333A (en) * | 2019-07-29 | 2019-11-15 | 招商局金融科技有限公司 | Data real time updating method, device and computer readable storage medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112732670A (en) * | 2020-12-31 | 2021-04-30 | 石河子大学 | Agricultural resource integration method and system based on network big data |
| CN112861905A (en) * | 2020-12-31 | 2021-05-28 | 杭州金时科技有限公司 | Tree species classification platform based on internet |
| CN112861905B (en) * | 2020-12-31 | 2024-03-01 | 杭州普睿益思信息科技有限公司 | Tree species classification platform based on internet |
| WO2023004881A1 (en) * | 2021-07-28 | 2023-02-02 | 深圳市爱云信息科技有限公司 | Smart agriculture aiot distributed big data storage platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Sarker et al. | Big data driven smart agriculture: Pathway for sustainable development | |
| Iaksch et al. | Digitalization and Big data in smart farming–a review | |
| Zheng et al. | Construct food safety traceability system for people’s health under the internet of things and big data | |
| Bandara et al. | Smart farm and monitoring system for measuring the Environmental condition using wireless sensor network-IOT Technology in farming | |
| CN111986042A (en) | Agricultural big data service system constructed based on internet technology | |
| CN105512949A (en) | Culturing farm informationized management system based on social platform and method thereof | |
| Jinbo et al. | Research on monitoring platform of agricultural product circulation efficiency supported by cloud computing | |
| Zhang et al. | A traceability public service cloud platform incorporating IDcode system and colorful QR code technology for important product | |
| CN114387119A (en) | Agricultural big data platform based on high in clouds | |
| CN112749952A (en) | County digital country construction management service platform | |
| CN106022659A (en) | Grid type monitoring system for agricultural product quality security | |
| Jaswal et al. | Atmospheric data collection, processing and database management in India Meteorological Department | |
| Jie | RETRACTED ARTICLE: Precision and intelligent agricultural decision support system based on big data analysis | |
| CN117541077A (en) | Digital rural sustainable development management cloud platform | |
| Zhou et al. | A Data‐Secured Intelligent IoT System for Agricultural Environment Monitoring | |
| CN104392383A (en) | Control system for agricultural physical energy circulating synthesis database and efficient circulating technology | |
| CN206283540U (en) | Internet big data managing device | |
| CN116341923A (en) | Digital rural business data analysis method, device, equipment and medium | |
| CN114897498A (en) | Information service management system for ecological agriculture informatization | |
| Zhang et al. | Production management of professional farmers under the new rural construction based on big data technology | |
| CN202385134U (en) | Agricultural and forest diseases and insect pests intelligent monitoring service system based on PDA | |
| Tian et al. | Big Data based Agricultural Intelligent Comprehensive Service Platform | |
| Zheng | [Retracted] Study and Application on Big Data Information Fusion System Based on IoT | |
| Yang et al. | Research status and future expectation of agricultural big data technology | |
| Zhang et al. | Research on Chinese Herbal Medicine Planting Information Service Platform System Based on Cloud Computing Network Technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201124 |
|
| RJ01 | Rejection of invention patent application after publication |