CN105718508A - Aquaculture information collecting and processing system - Google Patents
Aquaculture information collecting and processing system Download PDFInfo
- Publication number
- CN105718508A CN105718508A CN201610009741.5A CN201610009741A CN105718508A CN 105718508 A CN105718508 A CN 105718508A CN 201610009741 A CN201610009741 A CN 201610009741A CN 105718508 A CN105718508 A CN 105718508A
- Authority
- CN
- China
- Prior art keywords
- information
- aquaculture
- data
- module
- url
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000009360 aquaculture Methods 0.000 title claims abstract description 36
- 244000144974 aquaculture Species 0.000 title claims abstract description 36
- 238000012545 processing Methods 0.000 title claims abstract description 18
- 238000000605 extraction Methods 0.000 claims abstract description 22
- 238000001914 filtration Methods 0.000 claims abstract description 8
- 238000004821 distillation Methods 0.000 claims description 18
- 238000000034 method Methods 0.000 claims description 13
- 238000004458 analytical method Methods 0.000 claims description 6
- 238000007619 statistical method Methods 0.000 claims description 6
- 238000013459 approach Methods 0.000 claims description 3
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 238000004891 communication Methods 0.000 claims description 3
- 238000007405 data analysis Methods 0.000 claims description 3
- 238000003066 decision tree Methods 0.000 claims description 3
- 230000007613 environmental effect Effects 0.000 claims description 3
- 230000002068 genetic effect Effects 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 7
- 238000013473 artificial intelligence Methods 0.000 abstract 1
- 238000009348 integrated aquaculture Methods 0.000 abstract 1
- 238000009395 breeding Methods 0.000 description 2
- 230000001488 breeding effect Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000010030 laminating Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
- 239000003643 water by type Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/02—Agriculture; Fishing; Mining
Abstract
The invention discloses an aquaculture information collecting and processing system, which comprises an Nutch web page grabber, a topic filtering module, an information extraction module, an aquaculture information database, a data processing module, a Web server and a browser. The system achieves the functions of grabbing, classifying, filtering, extracting, warehousing, updating and the like of network resources on the basis of topic filtering and information extraction technologies of aquaculture service information and achieves reasonable and accurate classification of the extracted information through building an aquaculture dictionary. A networked service and WEB service-based integrated aquaculture information service system is developed by a cloud computing technology and an artificial intelligence algorithm; and high-quality aquaculture information service is achieved.
Description
Technical field
The invention belongs to technical field of aquaculture, relate to a kind of aquaculture data collecting and processing system.
Background technology
Aquaculture is the production activity breeding, cultivate and gather in the crops aquatic animals and plants under manual control.Along with developing rapidly of culture fishery, increasing modernization breeding way, such as cage culture, flowing water culture etc., three-dimension use waters, land and water laminating production ecological fishery, keep the technology of sustainable use of fishery resources to be used widely.The culture fishery that develops into of technology of Internet of things creates condition, but, to there is search and webpage quantity in current vertical search engine few in aquaculture data, information extraction comprehensively and topic distillation classify inaccurate problem, the present situation of current existing aquaculture mass data storage data message method for digging specific aim difference difficult, big, face the big crucial problem of the big data of Aquatic product four: data preparation, data storage, data platform, data process, and can not meet the aquaculture practitioner great demand to visualization resource.
Summary of the invention
It is an object of the invention to the defect overcoming above-mentioned technology to exist, a kind of aquaculture data collecting and processing system is provided, this system is based on the topic distillation of aquaculture information on services and information extraction technique, realize the functions such as the crawl of Internet resources, classification, filtration, extraction, warehouse-in renewal, and by building Aquatic product dictionary, it is achieved the reasonable Accurate classification to institute's Extracting Information.Adopt cloud computing technology and intelligent algorithm, carry out the aquaculture information comprehensive service system of Network Basedization service, WEB service, it is achieved high-quality aquatic product information service.
Its concrete technical scheme is:
A kind of aquaculture data collecting and processing system, including Nutch webpage capture device, topic distillation module, information extraction module, aquatic product information data base, data processing module, Web server and browser；
Described Nutch webpage capture device URL from Present Domestic aquaculture information service platform captures, capture strategy according to link and access all webpages, parse web page contents and new URL, and the info web parsed and new URL information feeding topic distillation module are filtered；
Described topic distillation module filters out the webpage unrelated with aquaculture theme and URL through topic distillation algorithm, then URL enters URL storehouse, and webpage is sent to information extraction module；
Information extraction module utilizes information extraction technique file connector by Ultraseek of the file of extended formatting and corresponding information extraction technique to be extracted for structurized information data for the Aquatic product web data obtained after topic distillation modular filtration, and the data extracted is stored in aquatic product information data base；
Described data processing module builds rationally comprehensively Aquatic product dictionary, the characteristic information of Internet resources will be obtained according to dictionary, carry out statistical analysis and classification, utilize current existing intelligent algorithm, by algorithm, data are excavated, relative analysis, set up Data Analysis Model, by model analysis, obtain intelligence, deep, valuable information；
Web server and browser pass through Internet and Web server communication, the aquaculture practitioner demand according to self, it is thus achieved that relevant information for aquaculture practitioner.
Further, the characteristic information of described Internet resources includes word frequency, lexeme, word length, webpage.
Further, described current existing intelligent algorithm includes neural network algorithm, genetic algorithm, traditional decision-tree, Rough Set, statistical analysis technique, FUZZY SET APPROACH TO ENVIRONMENTAL.
Compared with prior art, the invention have the benefit that
The present invention is based on the topic distillation of aquaculture information on services and information extraction technique, it is achieved the functions such as the crawl of Internet resources, classification, filtration, extraction, warehouse-in renewal, and by building Aquatic product dictionary, it is achieved the reasonable Accurate classification to institute's Extracting Information.Adopt cloud computing technology and intelligent algorithm, carry out the aquaculture information comprehensive service system of Network Basedization service, WEB service, it is achieved high-quality aquatic product information service.
Accompanying drawing explanation
Fig. 1 is the schematic diagram of aquaculture data collecting and processing system of the present invention.
Detailed description of the invention
Below in conjunction with the drawings and specific embodiments, technical scheme is described in more detail.
With reference to Fig. 1, a kind of aquaculture data collecting and processing system, including Nutch webpage capture device, topic distillation module, information extraction module, aquatic product information data base, data processing module, Web server and browser；
Described Nutch webpage capture device URL from Present Domestic aquaculture information service platform captures, capture strategy according to link and access all webpages, parse web page contents and new URL, and the info web parsed and new URL information feeding topic distillation module are filtered；
Described topic distillation module filters out the webpage unrelated with aquaculture theme and URL through topic distillation algorithm, then URL enters URL storehouse, and webpage is sent to information extraction module；
Information extraction module utilizes information extraction technique file connector by Ultraseek of the file of extended formatting and corresponding information extraction technique to be extracted for structurized information data for the Aquatic product web data obtained after topic distillation modular filtration, and the data extracted is stored in aquatic product information data base；
Described data processing module builds rationally comprehensively Aquatic product dictionary, the characteristic information of Internet resources will be obtained according to dictionary, statistical analysis and classification is carried out such as word frequency, lexeme, word length, webpage grade etc., utilize current existing intelligent algorithm (neural network algorithm, genetic algorithm, traditional decision-tree, Rough Set, statistical analysis technique, FUZZY SET APPROACH TO ENVIRONMENTAL), by algorithm, data are excavated, relative analysis, set up Data Analysis Model, by model analysis, obtain intelligence, deep, valuable information；
Web server and browser pass through Internet and Web server communication, the aquaculture practitioner demand according to self, it is thus achieved that relevant information for aquaculture practitioner.
The above; it is only the present invention preferably detailed description of the invention; protection scope of the present invention is not limited to this; any those familiar with the art is in the technical scope of present disclosure, and the simple change of the technical scheme that can become apparent to or equivalence are replaced and each fallen within protection scope of the present invention.
Claims (3)
1. an aquaculture data collecting and processing system, it is characterised in that include Nutch webpage capture device, topic distillation module, information extraction module, aquatic product information data base, data processing module, Web server and browser；
Described Nutch webpage capture device URL from Present Domestic aquaculture information service platform captures, capture strategy according to link and access all webpages, parse web page contents and new URL, and the info web parsed and new URL information feeding topic distillation module are filtered；
Described topic distillation module filters out the webpage unrelated with aquaculture theme and URL through topic distillation algorithm, then URL enters URL storehouse, and webpage is sent to information extraction module；
Information extraction module utilizes information extraction technique file connector by Ultraseek of the file of extended formatting and corresponding information extraction technique to be extracted for structurized information data for the Aquatic product web data obtained after topic distillation modular filtration, and the data extracted is stored in aquatic product information data base；
Described data processing module builds rationally comprehensively Aquatic product dictionary, the characteristic information of Internet resources will be obtained according to dictionary, carry out statistical analysis and classification, utilize current existing intelligent algorithm, by algorithm, data are excavated, relative analysis, set up Data Analysis Model, by model analysis, obtain intelligence, deep, valuable information；
Web server and browser pass through Internet and Web server communication, the aquaculture practitioner demand according to self, it is thus achieved that relevant information for aquaculture practitioner.
2. aquaculture data collecting and processing system according to claim 1, it is characterised in that the characteristic information of described Internet resources includes word frequency, lexeme, word length, webpage.
3. aquaculture data collecting and processing system according to claim 1, it is characterised in that described current existing intelligent algorithm includes neural network algorithm, genetic algorithm, traditional decision-tree, Rough Set, statistical analysis technique, FUZZY SET APPROACH TO ENVIRONMENTAL.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610009741.5A CN105718508A (en) | 2016-01-08 | 2016-01-08 | Aquaculture information collecting and processing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610009741.5A CN105718508A (en) | 2016-01-08 | 2016-01-08 | Aquaculture information collecting and processing system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105718508A true CN105718508A (en) | 2016-06-29 |
Family
ID=56147581
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610009741.5A Pending CN105718508A (en) | 2016-01-08 | 2016-01-08 | Aquaculture information collecting and processing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105718508A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111062511A (en) * | 2019-11-14 | 2020-04-24 | 佛山科学技术学院 | Aquaculture disease prediction method and system based on decision tree and neural network |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104516982A (en) * | 2015-01-06 | 2015-04-15 | 南通大学 | Method and system for extracting Web information based on Nutch |
-
2016
- 2016-01-08 CN CN201610009741.5A patent/CN105718508A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104516982A (en) * | 2015-01-06 | 2015-04-15 | 南通大学 | Method and system for extracting Web information based on Nutch |
Non-Patent Citations (2)
| Title |
|---|
| 周鹏: "农业搜索引擎系统的关键技术研究" * |
| 高亮亮 等: "基于Nutch框架的农业信息垂直搜索引擎研究与设计" * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111062511A (en) * | 2019-11-14 | 2020-04-24 | 佛山科学技术学院 | Aquaculture disease prediction method and system based on decision tree and neural network |
| CN111062511B (en) * | 2019-11-14 | 2023-04-25 | 佛山科学技术学院 | Aquaculture disease prediction method and system based on decision tree and neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109189901A (en) | Automatically a kind of method of the new classification of discovery and corresponding corpus in intelligent customer service system | |
| CN106650725A (en) | Full convolutional neural network-based candidate text box generation and text detection method | |
| CN107705066A (en) | Information input method and electronic equipment during a kind of commodity storage | |
| CN101650715A (en) | Method and device for screening links on web pages | |
| CN108470032A (en) | Overseas garden trade and investment promotion service system based on digital earth frame | |
| CN107808375B (en) | Merge the rice disease image detecting method of a variety of context deep learning models | |
| CN110134849A (en) | A kind of network public-opinion monitoring method and system | |
| CN105160038A (en) | Data analysis method and system based on audit database | |
| CN107437038A (en) | A kind of detection method and device of webpage tamper | |
| CN103823824A (en) | Method and system for automatically constructing text classification corpus by aid of internet | |
| CN101894351A (en) | Multi-agent based tour multimedia information personalized service system | |
| CN107885793A (en) | A kind of hot microblog topic analyzing and predicting method and system | |
| CN106529564A (en) | Food image automatic classification method based on convolutional neural networks | |
| CN103577581B (en) | Agricultural product price trend forecasting method | |
| CN106251234A (en) | A kind of agricultural product production and marketing integrated service platform based on the Internet and big data | |
| CN103927400A (en) | Web site product detailed information classification crawling and product information base establishing method | |
| CN108628994A (en) | A kind of public sentiment data processing system | |
| CN104077295A (en) | Data label mining method and data label mining system | |
| CN104615701B (en) | The embedded big data visualization engine cluster in smart city based on video cloud platform | |
| CN109784408A (en) | A kind of embedded time series Decision-Tree Method and system of marginal end | |
| CN106202467A (en) | A kind of definable towards peer-to-peer network searches for the web crawlers method of emphasis | |
| CN111582219B (en) | Intelligent pet management system | |
| CN103976468A (en) | Tobacco leaf grading method | |
| CN103714120B (en) | A kind of system that user interest topic is extracted in the access record from user url | |
| CN102306182A (en) | Method for excavating user interest based on conceptual semantic background image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160629 |
|
| RJ01 | Rejection of invention patent application after publication |