CN113196257A - 数据处理设备和方法、计算机可读存储介质 - Google Patents
数据处理设备和方法、计算机可读存储介质 Download PDFInfo
- Publication number
- CN113196257A CN113196257A CN201980002701.XA CN201980002701A CN113196257A CN 113196257 A CN113196257 A CN 113196257A CN 201980002701 A CN201980002701 A CN 201980002701A CN 113196257 A CN113196257 A CN 113196257A
- Authority
- CN
- China
- Prior art keywords
- data
- sub
- subdata
- period
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 135
- 238000003860 storage Methods 0.000 title claims abstract description 44
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000000605 extraction Methods 0.000 claims abstract description 52
- 238000004458 analytical method Methods 0.000 claims abstract description 33
- 230000010354 integration Effects 0.000 claims abstract description 32
- 238000007405 data analysis Methods 0.000 claims abstract description 16
- 238000003672 processing method Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 14
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 6
- 238000012986 modification Methods 0.000 claims description 5
- 230000004048 modification Effects 0.000 claims description 5
- 230000006835 compression Effects 0.000 claims 1
- 238000007906 compression Methods 0.000 claims 1
- 238000004519 manufacturing process Methods 0.000 description 20
- 238000010586 diagram Methods 0.000 description 18
- 238000013075 data extraction Methods 0.000 description 9
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 206010036086 Polymenorrhoea Diseases 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 2
- 239000003638 chemical reducing agent Substances 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 238000013501 data transformation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000011068 loading method Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000001550 time effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本公开提供一种数据处理设备和方法、计算机可读存储介质。数据处理设备包括存储器和处理器，处理器基于存储器存储的指令执行以下操作：以第一抽取周期抽取工厂数据库中的第一数据表中的第一数据，第一数据包括工厂在第一抽取周期内更新的数据，工厂数据库为关系型数据库，第一抽取周期大于1分钟；将第一数据存入分布式存储系统的第二数据表中形成第二数据；将第二数据进行数据整合后存入分布式存储系统的第三数据表中形成第三数据；以第一分析周期调用第三数据表中的数据进行数据分析处理，第一分析周期不小于第一抽取周期。本公开能够将数据从关系型数据库提取到分布式存储系统中，并满足用户对数据的分钟级别同步的需求。
Description
PCT国内申请，说明书已公开。
Claims (15)
- PCT国内申请，权利要求书已公开。
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2019/121896 WO2021102888A1 (zh) | 2019-11-29 | 2019-11-29 | 数据处理设备和方法、计算机可读存储介质 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113196257A true CN113196257A (zh) | 2021-07-30 |
Family
ID=76129782
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201980002701.XA Pending CN113196257A (zh) | 2019-11-29 | 2019-11-29 | 数据处理设备和方法、计算机可读存储介质 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113196257A (zh) |
| WO (1) | WO2021102888A1 (zh) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106407360A (zh) * | 2016-09-07 | 2017-02-15 | 广州视源电子科技股份有限公司 | 一种数据的处理方法及装置 |
| CN106649355A (zh) * | 2015-10-30 | 2017-05-10 | 北京国双科技有限公司 | 一种数据处理方法及装置 |
| WO2017092444A1 (zh) * | 2015-12-02 | 2017-06-08 | 中兴通讯股份有限公司 | 基于Hadoop的日志数据挖掘方法及系统 |
| CN107657049A (zh) * | 2017-09-30 | 2018-02-02 | 深圳市华傲数据技术有限公司 | 一种基于数据仓库的数据处理方法 |
| CN107729448A (zh) * | 2017-09-30 | 2018-02-23 | 深圳市华傲数据技术有限公司 | 一种基于数据仓库的数据处理系统 |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108038181A (zh) * | 2017-12-08 | 2018-05-15 | 山东浪潮商用系统有限公司 | 一种数据处理系统及数据处理方法 |
| CN109669965A (zh) * | 2018-11-13 | 2019-04-23 | 广州欧赛斯信息科技有限公司 | 一种支持非结构化数据的采集分析系统和方法 |
-
2019
- 2019-11-29 WO PCT/CN2019/121896 patent/WO2021102888A1/zh active Application Filing
- 2019-11-29 CN CN201980002701.XA patent/CN113196257A/zh active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106649355A (zh) * | 2015-10-30 | 2017-05-10 | 北京国双科技有限公司 | 一种数据处理方法及装置 |
| WO2017092444A1 (zh) * | 2015-12-02 | 2017-06-08 | 中兴通讯股份有限公司 | 基于Hadoop的日志数据挖掘方法及系统 |
| CN106407360A (zh) * | 2016-09-07 | 2017-02-15 | 广州视源电子科技股份有限公司 | 一种数据的处理方法及装置 |
| CN107657049A (zh) * | 2017-09-30 | 2018-02-02 | 深圳市华傲数据技术有限公司 | 一种基于数据仓库的数据处理方法 |
| CN107729448A (zh) * | 2017-09-30 | 2018-02-23 | 深圳市华傲数据技术有限公司 | 一种基于数据仓库的数据处理系统 |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021102888A1 (zh) | 2021-06-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230067182A1 (en) | Data Processing Device and Method, and Computer Readable Storage Medium | |
| US10176208B2 (en) | Processing time series data from multiple sensors | |
| EP2695086B1 (en) | Methods and systems for loading data into a temporal data warehouse | |
| WO2018051097A1 (en) | System for analysing data relationships to support query execution | |
| WO2017183065A1 (ja) | リレーショナルデータベースのチューニング装置及び方法 | |
| EP3365812A1 (en) | Create table for exchange | |
| EP2973046B1 (en) | System and method for compressing data in database | |
| US9779121B2 (en) | Transparent access to multi-temperature data | |
| EP1256896A2 (en) | "A plant information management system" | |
| US9489413B2 (en) | Asynchronous global index maintenance during partition maintenance | |
| WO2015168988A1 (zh) | 一种数据索引创建方法、装置及计算机存储介质 | |
| CN117033424A (zh) | 慢sql语句的查询优化方法、装置和计算机设备 | |
| US20220229821A1 (en) | Data restoration using dynamic data structure altering | |
| CN111708895B (zh) | 一种知识图谱系统的构建方法及装置 | |
| CN113196257A (zh) | 数据处理设备和方法、计算机可读存储介质 | |
| CN107085603A (zh) | 一种数据处理方法及装置 | |
| US9830323B2 (en) | Method and system for archiving data from a source database to a target database | |
| US11762831B2 (en) | Adaptive sparse indexing in cloud-based data warehouses | |
| CN113032495B (zh) | 基于数据仓库的多层数据存储系统、处理方法、及服务器 | |
| EP2960809B1 (en) | Transparent access to multi-temperature data | |
| CN113661510A (zh) | 基于非线性规划模型的投产规划系统，投产规划方法和计算机可读存储介质 | |
| CN118093441B (zh) | 一种cfd软件云测试算例同步方法及自动化测试平台 | |
| CN118260330A (zh) | 业务数据存储方法、装置、计算机设备以及存储介质 | |
| CN117520266A (zh) | 文件的处理方法、系统和电子设备及存储介质 | |
| CN116362212A (zh) | 报表生成方法、装置、设备及存储介质 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |