CN104462456A - Life data processing based big data system - Google Patents
Life data processing based big data system Download PDFInfo
- Publication number
- CN104462456A CN104462456A CN201410783349.7A CN201410783349A CN104462456A CN 104462456 A CN104462456 A CN 104462456A CN 201410783349 A CN201410783349 A CN 201410783349A CN 104462456 A CN104462456 A CN 104462456A
- Authority
- CN
- China
- Prior art keywords
- data
- unit
- mining
- life
- data processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of network information, in particular to a life data processing based big data system which is reasonable in structure and high in processing efficiency. The life data processing based big data system is characterized by comprising at least one data terminal unit, a data uploading unit connected with the data terminal units, a data storage unit used for storing use habits of users, a preprocessing unit used for extracting key information from the data storage unit, a data reduction unit used for screening preprocessed data, a data mining unit used for analyzing and mining user data and an output unit used for displaying and outputting mining results. Compared with the prior art, the life data processing based big data system has the advantages of reasonable structure, easiness in operation, moderate computation burden and capabilities of effectively utilizing an existing big database and providing better usage experience for users.
Description
Technical field:
The present invention relates to technical field of network information, the large data system based on life data processing that specifically a kind of rational in infrastructure, treatment effeciency is high.
Background technology:
Large data technique or claim flood tide data, refers to involved data quantity huge to cannot by current main software instrument, reaches to draw, manage, process and arranges and become the more positive object information of help enterprise management decision-making within the rational time.The strategic importance of large data counts does not lie in grasps huge data message, and be to carry out specialized process to these containing significant data, in other words, if large data are compared to a kind of industry, the key that so this industry realizes profit is to improve " working ability " to data, realizes increment by process data.
How from a large amount of, incomplete, noisy, fuzzy, random extracting data lie in wherein, ignorant in advance but process that the is information of potentially useful sometimes is called as data mining, obviously, the key of large data technique during data mining.
The serial mode on single machine node is still adopted to realize during existing large data system process data, the degree of load of its data processing amount and algorithm depends on the performance of single XM, and often will mass data be processed due to large data handling system, obviously there is the problem that efficiency is low, operand is low in existing single machine node serial mechanism.
Along with the maturation of wireless communication technology and the development of network technology, the family product of the wireless communication technologys such as wifi or zigbee is used to enter in everybody life, the a series of Smart Home products being representative with intelligent switch and smart jack are for consumer brings increasing convenient experience, but because current wifi equipment and zigbee equipment cannot fully merge, and wifi equipment and zigbee equipment are the independently closed-loop information circulation system, cause user's usage data utilization factor low, constrain the lifting of industry development and Consumer's Experience.
Summary of the invention:
The present invention is directed to the shortcoming and defect existed in prior art, propose the large data system based on life data processing that a kind of rational in infrastructure, treatment effeciency is high.
The present invention can be reached by following measures:
Based on a large data system for life data processing, it is characterized in that being provided with at least one data terminal unit, the data upload unit be connected with data terminal unit, for store user's use habit data storage cell, for extract from data storage cell key message pretreatment unit, for pretreated data are screened Data Reduction unit, for user data is carried out analysis mining data mining unit, for the output unit to Result display translation.
Pretreatment unit of the present invention is multidiameter delay pretreatment unit, for realizing the pre-service to multichannel user data, being provided with sliding-model control unit, ID attribute assignment unit, attribute increases/delete unit, attribute replaces processing unit in the pretreatment unit of every road.
Data terminal unit of the present invention can for gathering the intelligent terminal of user's usage data, and the data upload unit be connected with data terminal unit can be WIFI telecommunication circuit or Zigbee communication circuit.
Data storage cell of the present invention adopts cloud platform storage mode to realize, and completes the long-range of data and gathers and share.
Data Reduction unit of the present invention is used for carrying out yojan process to the data extracted in data storage cell, rejects, the data of redundancy, deficiency of data to improve data analysis efficiency and accuracy.
Data mining unit of the present invention is provided with the data fusion module that multidiameter delay data-mining module and the result for being obtained by multidiameter delay data-mining module carry out fusion treatment, the output of data fusion module is given to merge and is excavated module, merge excavation module again to excavate the data after fusion, obtain final analysis result.
When the present invention uses, can by the usage data of data terminal unit Real-time Collection user, and by telecommunication circuits such as wifi or zigbee by gathered data upload to the data storage cell of cloud platform, pretreatment unit needs to extract sample data from data storage cell according to use, and carry out additions and deletions attribute, conversion attribute, after distributing the process such as ID, the different data of the attribute that multiple data terminal unit is uploaded have consistance, then pretreated data are sent into Data Reduction unit, after eliminate redundancy data, send into the data mining unit being used for user data being carried out to analysis mining, analysis data, export after obtaining key message.
The present invention compared with prior art, have rational in infrastructure, easy and simple to handle, operand is moderate, can effectively utilize existing large database concept to provide better experience for user.
Accompanying drawing illustrates:
Accompanying drawing 1 is structural representation of the present invention.
Reference numeral: data terminal unit 1, data upload unit 2, data storage cell 3, pretreatment unit 4, Data Reduction unit 5, data mining unit 6, output unit 7.
Embodiment:
Below in conjunction with accompanying drawing, the present invention is further illustrated.
As shown in Figure 1, the present invention proposes a kind of large data system based on life data processing, it is characterized in that being provided with at least one data terminal unit 1, the data upload unit 2 be connected with data terminal unit 1, for store user's use habit data storage cell 3, for extract from data storage cell 3 key message pretreatment unit 4, for pretreated data are screened Data Reduction unit 5, for user data is carried out analysis mining data mining unit 6, for the output unit 7 to Result display translation.
Pretreatment unit 4 of the present invention is multidiameter delay pretreatment units, for realizing the pre-service to multichannel user data, being provided with sliding-model control unit, ID attribute assignment unit, attribute increases/delete unit, attribute replaces processing unit in the pretreatment unit of every road.
Data terminal unit 1 of the present invention can for gathering the intelligent terminal of user's usage data, and the data upload unit be connected with data terminal unit can be WIFI telecommunication circuit or Zigbee communication circuit.
Data storage cell 3 of the present invention adopts cloud platform storage mode to realize, and completes the long-range of data and gathers and share.
The data of redundancy, deficiency of data, for carrying out yojan process to the data extracted in data storage cell, are rejected, to improve data analysis efficiency and accuracy by Data Reduction unit 5 of the present invention.
Data mining unit 6 of the present invention is provided with the data fusion module that multidiameter delay data-mining module and the result for being obtained by multidiameter delay data-mining module carry out fusion treatment, the output of data fusion module is given to merge and is excavated module, merge excavation module again to excavate the data after fusion, obtain final analysis result.
When the present invention uses, can by the usage data of data terminal unit 1 Real-time Collection user, and by telecommunication circuits such as wifi or zigbee by gathered data upload to the data storage cell 3 of cloud platform, pretreatment unit 4 needs to extract sample data from data storage cell 3 according to use, and carry out additions and deletions attribute, conversion attribute, after distributing the process such as ID, the different data of the attribute that multiple data terminal unit 1 is uploaded have consistance, then pretreated data are sent into Data Reduction unit 5, after eliminate redundancy data, send into the data mining unit 6 being used for user data being carried out to analysis mining, analysis data, export after obtaining key message.
The present invention compared with prior art, have rational in infrastructure, easy and simple to handle, operand is moderate, can effectively utilize existing large database concept to provide better experience for user.
Claims (5)
1. based on a large data system for life data processing, it is characterized in that being provided with at least one data terminal unit, the data upload unit be connected with data terminal unit, for store user's use habit data storage cell, for extract from data storage cell key message pretreatment unit, for pretreated data are screened Data Reduction unit, for user data is carried out analysis mining data mining unit, for the output unit to Result display translation.
2. a kind of large data system based on life data processing according to claim 1, it is characterized in that described pretreatment unit is multidiameter delay pretreatment unit, for realizing the pre-service to multichannel user data, being provided with sliding-model control unit, ID attribute assignment unit in the pretreatment unit of every road, attribute increases/delete unit, attribute replaces processing unit.
3. a kind of large data system based on life data processing according to claim 1, it is characterized in that described data terminal unit can for gathering the intelligent terminal of user's usage data, the data upload unit be connected with data terminal unit can be WIFI telecommunication circuit or Zigbee communication circuit.
4. a kind of large data system based on life data processing according to claim 1, is characterized in that described data storage cell adopts cloud platform storage mode to realize, completes the long-range of data and gather and share.
5. a kind of large data system based on life data processing according to claim 1, it is characterized in that described data mining unit is provided with the data fusion module that multidiameter delay data-mining module and the result for being obtained by multidiameter delay data-mining module carry out fusion treatment, the output of data fusion module is given to merge and is excavated module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410783349.7A CN104462456A (en) | 2014-12-16 | 2014-12-16 | Life data processing based big data system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410783349.7A CN104462456A (en) | 2014-12-16 | 2014-12-16 | Life data processing based big data system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104462456A true CN104462456A (en) | 2015-03-25 |
Family
ID=52908491
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410783349.7A Pending CN104462456A (en) | 2014-12-16 | 2014-12-16 | Life data processing based big data system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104462456A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105550905A (en) * | 2015-12-30 | 2016-05-04 | 芜湖乐锐思信息咨询有限公司 | Product selling analysis system based on network |
| CN105608539A (en) * | 2015-12-30 | 2016-05-25 | 芜湖乐锐思信息咨询有限公司 | Rapid analysis system based on complex product production planning |
| CN105678571A (en) * | 2015-12-30 | 2016-06-15 | 芜湖乐锐思信息咨询有限公司 | Networked product planning analysis system based on Internet |
| CN106295961A (en) * | 2016-07-30 | 2017-01-04 | 江苏商达水务有限公司 | A kind of big Data Analysis Services system and method for water environment supervision |
| CN106528870A (en) * | 2016-12-05 | 2017-03-22 | 深圳万智联合科技有限公司 | Intelligent analysis system of big data |
| CN107093019A (en) * | 2017-04-21 | 2017-08-25 | 北京恒冠网络数据处理有限公司 | A kind of big data analysis system for macro adjustments and controls |
| CN111787082A (en) * | 2020-06-22 | 2020-10-16 | 珠海格力电器股份有限公司 | Method, equipment and system for reporting multi-stage service data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101408754A (en) * | 2008-10-30 | 2009-04-15 | 中山大学 | Intelligent house optimizing system based on data excavation |
| CN101799809A (en) * | 2009-02-10 | 2010-08-11 | 中国移动通信集团公司 | Data mining method and system |
| CN102495875A (en) * | 2011-12-02 | 2012-06-13 | 上海海洋大学 | Marine disaster early warning expert system based on data mining |
-
2014
- 2014-12-16 CN CN201410783349.7A patent/CN104462456A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101408754A (en) * | 2008-10-30 | 2009-04-15 | 中山大学 | Intelligent house optimizing system based on data excavation |
| CN101799809A (en) * | 2009-02-10 | 2010-08-11 | 中国移动通信集团公司 | Data mining method and system |
| CN102495875A (en) * | 2011-12-02 | 2012-06-13 | 上海海洋大学 | Marine disaster early warning expert system based on data mining |
Non-Patent Citations (2)
| Title |
|---|
| 赵伟: "《大数据在中国》", 30 June 2014, 江苏文艺出版社 * |
| 阎毅 等: "《信息科学技术导论》", 31 August 2014, 西安电子科技大学出版社 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105550905A (en) * | 2015-12-30 | 2016-05-04 | 芜湖乐锐思信息咨询有限公司 | Product selling analysis system based on network |
| CN105608539A (en) * | 2015-12-30 | 2016-05-25 | 芜湖乐锐思信息咨询有限公司 | Rapid analysis system based on complex product production planning |
| CN105678571A (en) * | 2015-12-30 | 2016-06-15 | 芜湖乐锐思信息咨询有限公司 | Networked product planning analysis system based on Internet |
| CN106295961A (en) * | 2016-07-30 | 2017-01-04 | 江苏商达水务有限公司 | A kind of big Data Analysis Services system and method for water environment supervision |
| CN106528870A (en) * | 2016-12-05 | 2017-03-22 | 深圳万智联合科技有限公司 | Intelligent analysis system of big data |
| CN106528870B (en) * | 2016-12-05 | 2018-06-08 | 贵州国信通电子商务有限公司 | A kind of big data intelligent analysis system |
| CN107093019A (en) * | 2017-04-21 | 2017-08-25 | 北京恒冠网络数据处理有限公司 | A kind of big data analysis system for macro adjustments and controls |
| CN111787082A (en) * | 2020-06-22 | 2020-10-16 | 珠海格力电器股份有限公司 | Method, equipment and system for reporting multi-stage service data |
| CN111787082B (en) * | 2020-06-22 | 2021-11-23 | 珠海格力电器股份有限公司 | Method, equipment and system for reporting multi-stage service data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104462456A (en) | Life data processing based big data system | |
| CN102955977B (en) | Energy efficiency service method and energy efficiency service platform adopting same on basis of cloud technology | |
| CN107402976B (en) | Power grid multi-source data fusion method and system based on multi-element heterogeneous model | |
| CN107945053A (en) | A kind of multiple source power distribution network data convergence analysis platform and its control method | |
| CN104461551A (en) | Parallel data processing based big data processing system | |
| CN107730394A (en) | The polynary isomeric data fusion method of panorama power network based on big data | |
| CN104899325A (en) | APP (application) acquired data collection method, terminal, server and system | |
| CN103379136A (en) | Compression method and decompression method of log acquisition data, compression apparatus and decompression apparatus of log acquisition data | |
| CN107611962B (en) | Power grid system branch searching method and system and electronic equipment | |
| CN103325071A (en) | User typical load curve construction method based on density clustering | |
| CN107239232B (en) | Data storage method for electric energy meter | |
| CN104731852A (en) | Big data system | |
| CN103825936B (en) | Distributed real-time energy consumption data monitoring system and method | |
| CN105183557A (en) | Configurable data compression system based on hardware | |
| CN109345628A (en) | A kind of model light-weight technologg method, apparatus and electronic equipment | |
| Kraus et al. | Optimal data compression techniques for smart grid and power quality trend data | |
| CN112365366A (en) | Micro-grid management method and system based on intelligent 5G slice | |
| CN102222112A (en) | Resource management device and resource management method | |
| CN204331727U (en) | A kind of data acquisition system (DAS) based on cloud computing | |
| CN102622447B (en) | Hadoop-based frequent closed itemset mining method | |
| CN104822175B (en) | A kind of code migration method and system suitable for cellular network | |
| CN103607731A (en) | Method and device for processing measurement reports | |
| CN108710347A (en) | A kind of monitoring cloud platform | |
| CN103944271B (en) | Based on the control method of the Intelligent Substation System of technology of Internet of things | |
| CN115765153A (en) | Method and system for fusion monitoring of Internet of things and online monitoring data of primary electric power equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150325 |
|
| RJ01 | Rejection of invention patent application after publication |