CN104239362A - High frequency data retrieval and storage method for use in cold rolling process - Google Patents
High frequency data retrieval and storage method for use in cold rolling process Download PDFInfo
- Publication number
- CN104239362A CN104239362A CN201310252685.4A CN201310252685A CN104239362A CN 104239362 A CN104239362 A CN 104239362A CN 201310252685 A CN201310252685 A CN 201310252685A CN 104239362 A CN104239362 A CN 104239362A
- Authority
- CN
- China
- Prior art keywords
- data
- partition
- subregion
- retrieval
- rolling process
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24554—Unary operations; Data partitioning operations
- G06F16/24557—Efficient disk access during query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
Abstract
The invention provides a high frequency data retrieval and storage method for use in a cold rolling process and aims to solve the problems that mass data and mass recordings are stored in the traditional cold rolling process control system, reading the data in retrieval applications takes a very long time, and usage of the system is severely affected. The method includes the steps of 1, according to a time field in a data table, dividing the data table into multiple data partitions, and storing the data partitions in different table spaces or a same table space, corresponding to different storage objects; 2, establishing a partition index on the basis of the data partitions; 3, scanning the corresponding partitions to retrieval the data stored in a partition-based storage manner. The method is applicable to retrieval and storage of the high frequency data of the cold rolling process.
Description
technical field:
The present invention relates to a kind of retrieval and storage means of cold-rolling process High-frequency Data.
background technology:
Process control of cold rolling system is attached most importance to product quality system, collects, key time, the Key output value data of integration process machine and energy management system, sets up the data platform that user uses the procedure quality of more convenient support product to analyze.Because process control of cold rolling system will extract a large amount of process machine high-frequency datas, and need the preservation time of 24 months even longer.Do these high-frequency datas have much? for each frame roll-force total length curve of sour rolling mill, the total length data highest frequency gathered is every meter, continuous seepage, the record of 24 months roughly has 144,000,000 (the 24(month) * 30(days) * 200 (average every day produces coil of strip) * 1000(, according to sample frequency and gantry speed, is converted into the record number of every bar coil of strip) bar.In process control of cold rolling system in the past, what data stored employing is common storage mode, the data of correlated variables are put into a table, process control of cold rolling system acquires altogether hundreds of bar total length curve, if according to the method, the capacity of these total length curve tables is all very large, record is many, the problem brought thus is that in application query, read time of corresponding data very long, even reach more than 5 minutes sometimes, have a strong impact on system use, in process control of cold rolling system in the past, the record of storage mode with steel in tables of data of mass data is not unique, the data of corresponding about 1000 of band grade of steel, and the tables of data storage data volume of 24 months itself is very large, grade of steel is being with to obtain the process of these band grade of steel data from tables of data by specifying, the time expended is very long, substantially to expend 5 minutes even longer, for the table that data volume is larger, time is longer, user cannot stand this time substantially.How effectively to store these high-frequency datas, shorten data read time and do not carry out systematic research, in this, still belong to a blank.
summary of the invention:
The object of the invention is retrieval and storage means that a kind of cold-rolling process High-frequency Data is provided for above-mentioned Problems existing, play and effectively can store high-frequency data, and shorten the time of reading high-frequency data table data, thus process control of cold rolling system is enable better to serve scene.
Above-mentioned object is realized by following technical scheme:
The retrieval of cold-rolling process High-frequency Data and storage means, the method comprises:
(1) according to place tables of data time field, first tables of data is divided into multiple data partition, is stored in different table spaces or identical table space, the memory object that table space is corresponding different;
(2) on the basis of data partition, subregion index is set up;
(3), when the data adopting partitioned storage mode to store being inquired about, scan corresponding subregion and complete inquiry.
The retrieval of described cold-rolling process High-frequency Data and storage means, the method also comprises restores data filing and individual data subregion, specific operation process is: operate in and separated from partition table data partition in logic, and generates the data temporary table of this subregion; Derive this partition data file of backup, and under partitioned file is packed automatically being stored in designated disk space; Automatically delete the data temporary table of this subregion, complete partition table and separate and backup procedure.
The retrieval of described cold-rolling process High-frequency Data and storage means, described is monthly divide according to the time of tables of data time Field-Partitions, place.
Beneficial effect:
1. the present invention is divided into multiple data partition (monthly dividing) according to place tables of data time field tables of data, and be stored in different table spaces or identical table space, table space may correspond to different memory object.After having had table subregion, data retrieval operation no longer for whole table, only for the data partition of specifying.Table subregion has following advantage: improve query performance: only can search for the subregion oneself be concerned about to the inquiry of zone object, improves retrieval rate.Strengthen availability: if certain subregion of table breaks down, show still can use in the data of other subregions; Easy to maintenance: if certain subregion of table breaks down, to need repair data, only repair this subregion; Balanced I/O: can different partition map to disk to balance I/O, improve whole system performance.
2. the present invention sets up subregion index, subregion index is different from traditional global index, the significant advantage that it brings is, subregion index, by multiple index partition, forms, and the key assignments in each index partition points to a unique data partition data accordingly, therefore when utility command is carried out proceeding to of data and produced, database performance can be improved significantly, and when carrying out the transfer of partition data Trimmed sums subregion, not need to rebuild subregion index.To the tables of data be based upon on partition table basis in process control of cold rolling system item, all employ the concept of subregion index.
3., when the present invention inquires about the data adopting partitioned storage mode to store, only need scan corresponding subregion just can complete inquiry, is not only data base administrator and provides higher scalability and dirigibility, and improve performance and the control ability of database.Provide conveniently for data filing and individual data subregion restore.Archiving process does not need database to go offline to operate, use the ATTACH PARTITION of ALTER TABLE statement and DETACH PARTITION clause can fast and easily, operation data being proceeded to or produces, compared with common Delete cancel statement, there is the advantages such as fast, the timely recovery space of speed.
4. method of the present invention trying out through plum steel process control of cold rolling system, the data that large degree of convergence improves insert the speed of process control of cold rolling system database, reduce the fault in the communication of both sides' text, improve the efficiency of data filing, recovery, enhance the security of data, alleviate the pressure of unit O&M.Through front and back contrast, after adopting new high-frequency data storage mode, the time of reading each high frequency table data can obtain required data from the original required 5 minutes time foreshortened to less than 5 seconds.Concerning user, needed 5 minutes ability application function out (such as total length curve) in the past, just result can be found less than the time of 5 seconds now, and the historical data inquired within 24 months that can also be very fast, greatly improve the efficiency of analysis of history data, can better analyze by subsidiary products quality objection, thus the use process control of cold rolling system enabling user convenient.
embodiment:
The retrieval of cold-rolling process High-frequency Data of the present invention and storage means, the method comprises:
(1) according to place tables of data time field, first tables of data is divided into multiple data partition, is stored in different table spaces or identical table space, the memory object that table space is corresponding different;
(2) on the basis of data partition, subregion index is set up;
(3), when the data adopting partitioned storage mode to store being inquired about, scan corresponding subregion and complete inquiry.
The retrieval of described cold-rolling process High-frequency Data and storage means, the method also comprises restores data filing and individual data subregion, specific operation process is: operate in and separated from partition table data partition in logic, and generates the data temporary table of this subregion; Derive this partition data file of backup, and under partitioned file is packed automatically being stored in designated disk space; Automatically delete the data temporary table of this subregion, complete partition table and separate and backup procedure.
The retrieval of described cold-rolling process High-frequency Data and storage means, described is monthly divide according to the time of tables of data time Field-Partitions, place.
Below be only most preferred embodiment of the present invention, method of the present invention includes but not limited to the technical scheme disclosed in above-described embodiment, and unaccomplished matter of the present invention belongs to the common practise of those skilled in the art.
Claims (3)
1. the retrieval of cold-rolling process High-frequency Data and a storage means, is characterized in that: the method comprises:
(1) according to place tables of data time field, first tables of data is divided into multiple data partition, is stored in different table spaces or identical table space, the memory object that table space is corresponding different;
(2) on the basis of data partition, subregion index is set up;
(3), when the data adopting partitioned storage mode to store being inquired about, scan corresponding subregion and complete inquiry.
2. the retrieval of cold-rolling process High-frequency Data according to claim 1 and storage means, it is characterized in that: the method also comprises restores data filing and individual data subregion, specific operation process is: operate in and separated from partition table data partition in logic, and generates the data temporary table of this subregion; Derive this partition data file of backup, and under partitioned file is packed automatically being stored in designated disk space; Automatically delete the data temporary table of this subregion, complete partition table and separate and backup procedure.
3. the retrieval of cold-rolling process High-frequency Data according to claim 1 and storage means, is characterized in that: described is monthly divide according to the time of tables of data time Field-Partitions, place.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310252685.4A CN104239362A (en) | 2013-06-24 | 2013-06-24 | High frequency data retrieval and storage method for use in cold rolling process |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310252685.4A CN104239362A (en) | 2013-06-24 | 2013-06-24 | High frequency data retrieval and storage method for use in cold rolling process |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104239362A true CN104239362A (en) | 2014-12-24 |
Family
ID=52227442
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310252685.4A Pending CN104239362A (en) | 2013-06-24 | 2013-06-24 | High frequency data retrieval and storage method for use in cold rolling process |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104239362A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106293995A (en) * | 2015-05-15 | 2017-01-04 | 中芯国际集成电路制造(上海)有限公司 | The data backup system of manufacturing execution system and method |
| CN110377624A (en) * | 2019-07-31 | 2019-10-25 | 象辑知源（武汉）科技有限公司 | A kind of storage and querying method to the geographic information data with multidimensional properties such as time and spaces |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1719441A (en) * | 2005-06-10 | 2006-01-11 | 上海宝信软件股份有限公司 | Date acquiring method realized by using object oriented data base development |
| CN101388009A (en) * | 2007-09-11 | 2009-03-18 | 北京思软科技有限公司 | Method for optimizing supper-large data quantity processing ability by dynamic table name |
| CN101763350A (en) * | 2008-12-30 | 2010-06-30 | 无锡虹业自动化工程有限公司 | Implementation method of high-capacity and high-speed configuration software report forms |
| US20120209894A1 (en) * | 2010-12-20 | 2012-08-16 | International Business Machines Corporation | Partition file system for virtual machine memory management |
| CN103069351A (en) * | 2010-08-09 | 2013-04-24 | 东芝三菱电机产业系统株式会社 | Data retrieval system |
-
2013
- 2013-06-24 CN CN201310252685.4A patent/CN104239362A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1719441A (en) * | 2005-06-10 | 2006-01-11 | 上海宝信软件股份有限公司 | Date acquiring method realized by using object oriented data base development |
| CN101388009A (en) * | 2007-09-11 | 2009-03-18 | 北京思软科技有限公司 | Method for optimizing supper-large data quantity processing ability by dynamic table name |
| CN101763350A (en) * | 2008-12-30 | 2010-06-30 | 无锡虹业自动化工程有限公司 | Implementation method of high-capacity and high-speed configuration software report forms |
| CN103069351A (en) * | 2010-08-09 | 2013-04-24 | 东芝三菱电机产业系统株式会社 | Data retrieval system |
| US20120209894A1 (en) * | 2010-12-20 | 2012-08-16 | International Business Machines Corporation | Partition file system for virtual machine memory management |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106293995A (en) * | 2015-05-15 | 2017-01-04 | 中芯国际集成电路制造(上海)有限公司 | The data backup system of manufacturing execution system and method |
| CN106293995B (en) * | 2015-05-15 | 2019-07-02 | 中芯国际集成电路制造(上海)有限公司 | The data backup system and method for manufacturing execution system |
| CN110377624A (en) * | 2019-07-31 | 2019-10-25 | 象辑知源（武汉）科技有限公司 | A kind of storage and querying method to the geographic information data with multidimensional properties such as time and spaces |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102682108B (en) | Row and line mixed database storage method | |
| CN102890721B (en) | Based on database building method and the system of row memory technology | |
| CN103678339B (en) | Data backflow method and system and data access method and system in relational database | |
| CN104407879B (en) | A kind of power network sequential big data loaded in parallel method | |
| CN102402617A (en) | Easily compressed database index storage system using fragments and sparse bitmap, and corresponding construction, scheduling and query processing methods | |
| CN103617232A (en) | Paging inquiring method for HBase table | |
| CN104731796A (en) | Data storage computing method and system | |
| CN102722584B (en) | Data storage system and method | |
| CN102609491A (en) | Column-storage oriented area-level data compression method | |
| CN104035956A (en) | Time-series data storage method based on distributive column storage | |
| SG140510A1 (en) | System and method for database indexing, searching and data retrieval | |
| CN101751475B (en) | Method for compressing section records and device therefor | |
| CN104765754A (en) | Data storage method and device | |
| CN106407191A (en) | Data processing method and server | |
| CN106250409A (en) | Data query method and device | |
| CN103235811A (en) | Data storage method and device | |
| CN104834650A (en) | Method and system for generating effective query tasks | |
| CN108009290A (en) | A kind of data modeling and storage method of track traffic command centre gauze big data | |
| CN105787090A (en) | Index building method and system of OLAP system of electric data | |
| CN104239362A (en) | High frequency data retrieval and storage method for use in cold rolling process | |
| CN107291881A (en) | Massive logs storage and querying method based on HBase | |
| CN101937464B (en) | Ciphertext search method based on word-for-word indexing | |
| CN102542029A (en) | Block compressed storage method of industrial data | |
| CN104699815A (en) | Data processing method and system | |
| CN108304527A (en) | A kind of data extraction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20141224 |
|
| RJ01 | Rejection of invention patent application after publication |