CN103440282A - Test data storage system and method - Google Patents
Test data storage system and method Download PDFInfo
- Publication number
- CN103440282A CN103440282A CN201310351678XA CN201310351678A CN103440282A CN 103440282 A CN103440282 A CN 103440282A CN 201310351678X A CN201310351678X A CN 201310351678XA CN 201310351678 A CN201310351678 A CN 201310351678A CN 103440282 A CN103440282 A CN 103440282A
- Authority
- CN
- China
- Prior art keywords
- data
- file
- time domain
- frequency domain
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a test data storage system and a test data storage method. The test data storage system comprises a data storage platform and a data storage center, wherein the data storage platform comprises a data classification module, an attachment storage module, an index information generation module, a parsing rule module, a non-time-domain and non-frequency-domain data storage module and a time domain and frequency domain data storage module; the data storage center comprises a relational database and a file database; the parsing rule module is used for dividing structural data into non-time-domain and non-frequency-domain data and time domain and frequency domain data; the relational database is used for storing a matched form menu established according to the type of test data; the file database is used for storing file data. The test data storage system and the test data storage method solve the technical problems of large volume of remote test run data and complexity in management, and can realize the parsing and database storage of multi-parameter time domain and frequency domain files, obtain the file form with multiple parameters through parsing and establish basis for the follow-up curve plotting.
Description
Technical field
The invention belongs to engine test Data Digital field, be specifically related to a kind of for the rocket engine ground test date storage method.
Background technology
Along with test mission sharply increases, make data and the existing firing test data data of test accumulation for many years sharply expand to make us unthinkable speed.Test figure is being carried a large amount of valuable Test Information, and the experimenter's of having condensed time and efforts is the memory of Multiple Type development of engine and important stock of knowledge resource.But prior art lacks effective test data management system, cause the maintenance of test run test figure and effectively utilize all very difficultly, the integrality of data, consistance, security and availability also are difficult to ensure.The data Management problem is becoming the new bottleneck that affects the firing test data analysis-by-synthesis and effectively utilize.The characteristics of current test figure:
(1) mass data continued to increase
Test obtains the measurement data such as motor power, temperature, pressure, flow, vibration.Measure passage tens tunnels at least, the sample frequency on multichannel ，Mei up to a hundred road does not wait to several ten thousand points of per second from per second is several at most.Annual such test is no less than 10 times, and at present, by the accumulation of data decades, total original data volume (stable state+dynamic) has approached 1T(1024G), and such data volume also will produce continuously.
(2) off-gauge data layout
The raw data file form produced by the data acquisition system (DAS) standard that is lack of consistency, and along with the variation of tested object and the development of testing software, the content recorded in data file and form are also constantly changing, caused the data file in one period must have a kind of process software to answer in contrast, the data file of different times and process software are incompatible mutually.Therefore to the standardization of data, should be the primary problem solved of firing test data management system.
(3) isolatism of data message
The dispersion of data causes resource to be difficult to share, and between overall and subordinate unit, data are difficult to exchange and retrieval, quote, form data silo one by one, bring a large amount of duplication of labour, to the performance boost of space flight hydrodynamic system, ensure that Development Schedule affects greatly.
Therefore, in the urgent need to setting up the management platform of a long-range firing test data, its top priority is to the data storage administration of being classified, for later use and depth analysis lay the first stone, to improve ability and the level of data message management.The method realizes fast, classifies and store for mass file data.
Summary of the invention
Large in order to solve existing long-range firing test data amount, the technical matters of complex management, the present invention puies forward a kind of test figure storage system and method.
Technical solution of the present invention:
A kind of data-storage system, its special character is: comprise data storing platform and data storage center, described data storing platform comprises Data classification module, annex memory module, index information generation module, resolution rules module, non-time domain, frequency domain data parsing module and time domain and frequency domain data parsing module;
Described data storage center comprises relational database and document data bank;
Described Data classification module is used for receiving data source, and the data source received is divided into to structural data and unstructured data according to the difference of data layout;
Described annex memory module realizes the raw data memory function in unstructured data storage and structural data, the unstructured data storage is passed through with the attachment version uploading data, document data bank is opened up certain storage space in disk, carries out file name establishment and data and writes;
Described index information generation module is used for unstructured data and structural data spanned file index information, and is stored in successively in relational database; Described file index information comprises file size, file tunnel name and file store path;
Whether described resolution rules module is for being divided into non-time domain, frequency domain data and time domain, frequency domain data with time domain and spectrum correlation by the structural data basis, and non-time domain, frequency domain data are sent to non-time domain, frequency domain data memory module, time domain, frequency domain data are sent to time domain, frequency domain data memory module simultaneously;
Described non-time domain, frequency domain data memory module be for non-time domain, frequency domain data, carries out after the data content extraction writing in the corresponding list of table space of relational database;
Described time domain, frequency domain data memory module are set up certain storage space in document data bank, generate respectively the * .tdmdf formatted file of the total parameter quantity formed with time row and each Argument List according to number of parameters, described file name comprises old file name+parameter code name+date;
Described relational database is set up the list of coupling according to the test figure type for storage, and by the file index information corresponding stored that receives in corresponding list;
Described document data bank store file data.
A kind of date storage method, its special character is: comprise the following steps:
1] building database:
Described database comprises relational database and document data bank,
Set up table space in relational database, and the list of coupling is set up in storage according to the test figure type;
2] data storing platform receives data source, and the Data classification module will receive data source and be divided into structural data and unstructured data according to the difference of data layout;
3] the annex memory module realizes the raw data memory function in unstructured data storage and structural data, the unstructured data storage is passed through with the attachment version uploading data, document data bank is opened up certain storage space in disk, carries out file name establishment and data and writes;
The index information generation module is unstructured data and structural data spanned file index information, and is stored in successively in the corresponding list of relational database; File index information comprises file size, file tunnel name and file store path;
4] whether the resolution rules module is divided into non-time domain, frequency domain data and time domain, frequency domain data with time domain and spectrum correlation by the structural data basis, and non-time domain, frequency domain data are sent to non-time domain, frequency domain data memory module, time domain, frequency domain data are sent to time domain, frequency domain data memory module simultaneously;
5] non-time domain, frequency domain data memory module be for non-time domain, frequency domain data, carries out after the data content extraction writing in the corresponding list of table space of relational database;
Time domain, frequency domain data memory module are set up certain storage space in the database file internal memory, generate respectively the * .tdmdf formatted file of the total parameter quantity formed with time row and each Argument List, and described file name comprises old file name+parameter code name+date.
The present invention has advantages of:
1, the present invention, by the test figure classification, is divided into structural data, the large class of unstructured data two by it; Realize the unified storage of test figure in database, file system, file layout comprises * .UFF, * .txt, * .doc, * .mdb form and near data (compressed package, picture, audio frequency, video format); Realize that speed becomes the quick storage of large file (>=50Mbytes);
2, the present invention can realize the parsing warehouse-in of multiparameter time domain, frequency domain file, and resolves to the document form of a plurality of parameters, is follow-up Drawing of Curve, sets up basis; Realize the slow varying parameter Data Analysis warehouse-in of non-time domain, frequency domain file, for the follow-up data compare of analysis is set up basis simultaneously;
3, the present invention realizes that the accessory class data are uploaded in file system fast, for annexes such as subsequent pictures, video, audio frequency, checks and sets up basis; Memory function be embodied as that data are processed and data analysis lay the data basis, significant to raising test figure information system management level and data management efficiency.
The accompanying drawing explanation
The platform structure figure that Fig. 1 is storage system of the present invention;
The schematic diagram that Fig. 2 is storage system of the present invention;
Fig. 3 is data Stored Procedure schematic diagram of the present invention;
Fig. 4 is * .txt file operation interface;
Fig. 5 is * .mdb formatted file operation interface;
Fig. 6 is that * .doc file is uploaded;
Fig. 7 is that compressed package files is uploaded;
Fig. 8 is that picture, video type file are uploaded;
Fig. 9 is curve display access figure.
Embodiment
1 platform building, as shown in Figure 1,
System management ability: support the data Management ability that 200T is above;
Operating system: Windows server2003;
The radix database: Oracle10g takes;
Storage mode: adopt relational database+document data bank array mode storage;
Test data management system: system provide a set of complete, for memory model and the service of test figure characteristics, wherein both comprised the Large-scale Relational Database system for the storage organization test figure, also comprised for storing the project file system of unstructured data.
2 Data Sources
Test figure, by sources comprise test preceding document data (test mission book, test resource, test direction book), set-up procedure data (quality record, account), test findings data (gradual, fast parameter certificate).
3 create relational database
Set up table space, set up the list of coupling according to the test figure type.
4 Ultrahighs
Storage realizes realizing by Data classification module, annex memory module, index stores module, index information generation module, resolution rules module, non-time domain, frequency domain data memory module, time domain, frequency domain data memory module.Wherein each intermodule relation, be shown in Fig. 2.The major function of each module is as follows:
4.1 Data classification module: data are classified, for each classification system phasing, answer the stores processor mode.Data comprise .UFF, * .txt, * .doc, * .mdb form and attachment data (compressed package, picture, audio frequency, video format) data.Difference by file content is divided into structural data and the large class of unstructured data two.
4.2 annex memory module: mainly realize the raw data memory function in unstructured data storage and structural data, the unstructured data storage is passed through with the attachment version uploading data, file system is opened up certain storage space in disk, carries out file name establishment and data and writes.
4.3 index information generation module: generating indexes information purpose is associated with test mission.Structuring time domain, frequency domain data generate parameter reference information, comprising: file parameters title, start time, frequency, concluding time, frequency; Remainder data spanned file index information comprises: file size, file (passage) title, file path.
4.4 resolution rules module: mainly for structural data, formulate corresponding resolution rules, for resolving after rules selection.Structural data needs resolution data to carry out rules selection, and resolution rules mainly contains two large classes, and a class is the supplemental characteristic resolution rules of time domain, frequency domain, and another kind of is non-time domain, frequency domain data resolution rules.
4.5 non-time domain, frequency domain data parsing module: mainly for the non-time domain in structural data, frequency domain resolution data, carry out data content extract after in the corresponding list of table space of write into Databasce, create the respective associated list and carry out the data storage.
4.6 time domain, frequency domain data parsing module: mainly for the time domain in institutional data, frequency domain resolution data, generate a plurality of file names (old file name+parameter code name+date) according to the file parameters number, set up certain storage space in file system, set up a plurality of one-parameter Data Filenames (* .tdmdf) in the file system respective paths, generate respectively the * .tdmdf formatted file of the total parameter quantity formed with time row and each Argument List.Time domain, frequency domain document definition: take time or frequency as abscissa axis, the 2-D data that the supplemental characteristic of take is axis of ordinates.
5 storage operation steps
The data storage mainly relates to the storage of two aspect data, and wherein whole operation steps schematic diagram, be shown in Fig. 3.The storing step of concrete each several part is as follows:
5.1 structural data storage
5.5.1 time domain, frequency domain data
Identify time domain, data masterplate resolution rules that frequency domain data is corresponding by resolution rules, the rule corresponding such as time domain data (complete close * .txt file) is " full ciphertext data ", after receiving " selection " instruction, call windows file directory systemic-function, select corresponding " complete close * .txt " file data, and receive " uploading raw data " instruction, and, after receiving " importing " instruction, the index information generation module generates parameter reference information in database; Time domain, frequency domain memory module resolution data content generate the * .tdmdf file of multiparameter in file system according to the parameter name in file; On annex, transmission module is uploaded the raw data file of this document.Operation interface is shown in Fig. 4.
5.5.2 non-time domain, frequency domain data
Identify non-time domain, data masterplate resolution rules that frequency domain data is corresponding by resolution rules, such as the 11-001P6000.mdb file.Corresponding rule is " P6000.mdb ", after receiving " selection " instruction, call windows file directory systemic-function, select corresponding " 11-001P6000.mdb " file data, and reception " uploading raw data " instruction, and, after receiving " importing " instruction, the index information generation module is the spanned file index information in database; Non-time domain, frequency domain memory module resolution data content are stored data content in the corresponding list of database; On annex, transmission module is uploaded the raw data file of this document.Operation interface is shown in Fig. 5.
5.5.3 unstructured data storage
Upload with attachment version, the descriptor of file is provided simultaneously and defines the type of information, be convenient to by its descriptor fast finding file.File type comprises the data file of non-structured * .doc file and compressed package form, and picture, audio frequency, video file.Fig. 6 is seen, Fig. 7 and Fig. 8 in corresponding upload operation interface.
6 data accesses
The Drawing of Curve of take in accessing, as example is introduced, enters platform by " access " interface, and selection need be carried out the parameter of Drawing of Curve, after " selection " completes, carry out " curve demonstration " instruction, software carries out Drawing of Curve for this time domain data parameter, and design sketch is shown in Fig. 9.
Claims (2)
1. a data-storage system, it is characterized in that: comprise data storing platform and data storage center, described data storing platform comprises Data classification module, annex memory module, index information generation module, resolution rules module, non-time domain, frequency domain data parsing module and time domain and frequency domain data parsing module;
Described data storage center comprises relational database and document data bank;
Described Data classification module is used for receiving data source, and the data source received is divided into to structural data and unstructured data according to the difference of data layout;
Described annex memory module realizes the raw data memory function in unstructured data storage and structural data, the unstructured data storage is passed through with the attachment version uploading data, document data bank is opened up certain storage space in disk, carries out file name establishment and data and writes;
Described index information generation module is used for unstructured data and structural data spanned file index information, and is stored in successively in relational database; Described file index information comprises file size, file tunnel name and file store path;
Whether described resolution rules module is for being divided into non-time domain, frequency domain data and time domain, frequency domain data with time domain and spectrum correlation by the structural data basis, and non-time domain, frequency domain data are sent to non-time domain, frequency domain data memory module, time domain, frequency domain data are sent to time domain, frequency domain data memory module simultaneously;
Described non-time domain, frequency domain data memory module be for non-time domain, frequency domain data, carries out after the data content extraction writing in the corresponding list of table space of relational database;
Described time domain, frequency domain data memory module are set up certain storage space in document data bank, generate respectively the * .tdmdf formatted file of the total parameter quantity formed with time row and each Argument List according to number of parameters, described file name comprises old file name+parameter code name+date;
Described relational database is set up the list of coupling according to the test figure type for storage, and by the file index information corresponding stored that receives in corresponding list;
Described document data bank store file data.
2. the date storage method of data-storage system according to claim 1 is characterized in that: comprise the following steps:
1] building database:
Described database comprises relational database and document data bank,
Set up table space in relational database, and the list of coupling is set up in storage according to the test figure type;
2] data storing platform receives data source, and the Data classification module will receive data source and be divided into structural data and unstructured data according to the difference of data layout;
3] the annex memory module realizes the raw data memory function in unstructured data storage and structural data, the unstructured data storage is passed through with the attachment version uploading data, document data bank is opened up certain storage space in disk, carries out file name establishment and data and writes;
The index information generation module is unstructured data and structural data spanned file index information, and is stored in successively in the corresponding list of relational database; File index information comprises file size, file tunnel name and file store path;
4] whether the resolution rules module is divided into non-time domain, frequency domain data and time domain, frequency domain data with time domain and spectrum correlation by the structural data basis, and non-time domain, frequency domain data are sent to non-time domain, frequency domain data memory module, time domain, frequency domain data are sent to time domain, frequency domain data memory module simultaneously;
5] non-time domain, frequency domain data memory module be for non-time domain, frequency domain data, carries out after the data content extraction writing in the corresponding list of table space of relational database;
Time domain, frequency domain data memory module are set up certain storage space in the database file internal memory, generate respectively the * .tdmdf formatted file of the total parameter quantity formed with time row and each Argument List, and described file name comprises old file name+parameter code name+date.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310351678.XA CN103440282B (en) | 2013-08-13 | 2013-08-13 | A kind of test data storage system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310351678.XA CN103440282B (en) | 2013-08-13 | 2013-08-13 | A kind of test data storage system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103440282A true CN103440282A (en) | 2013-12-11 |
| CN103440282B CN103440282B (en) | 2016-09-14 |
Family
ID=49693974
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310351678.XA Active CN103440282B (en) | 2013-08-13 | 2013-08-13 | A kind of test data storage system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103440282B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103942260A (en) * | 2014-03-21 | 2014-07-23 | 深圳海联讯科技股份有限公司 | Permission data indexing technology for structured data |
| CN104298726A (en) * | 2014-09-26 | 2015-01-21 | 惠州市亿能电子有限公司 | BMS data storage system and method based on database |
| CN106528641A (en) * | 2016-10-13 | 2017-03-22 | 许继集团有限公司 | Data storage method and device and communication gateway machine |
| CN107944029A (en) * | 2017-12-12 | 2018-04-20 | 清华大学 | A kind of method and apparatus of the time series data storage based on frequency domain character |
| CN110609502A (en) * | 2019-09-26 | 2019-12-24 | 武汉市珞珈俊德地信科技有限公司 | Assembled map data processing system |
| CN114118008A (en) * | 2022-01-21 | 2022-03-01 | 西安羚控电子科技有限公司 | Data comparison system and method based on BS architecture |
| CN115438514A (en) * | 2022-11-07 | 2022-12-06 | 大尧信息科技(湖南)有限公司 | Reconfigurable experiment system based on cloud server |
| WO2023123287A1 (en) * | 2021-12-30 | 2023-07-06 | 深圳晶泰科技有限公司 | Molecular data storage method and device, and molecular data application method and device |
| CN117481667A (en) * | 2023-10-24 | 2024-02-02 | 沈阳工业大学 | Electroencephalogram signal acquisition system |
| CN118535533A (en) * | 2024-07-26 | 2024-08-23 | 北京摩弈信息科技有限公司 | Data management method and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070038591A1 (en) * | 2005-08-15 | 2007-02-15 | Haub Andreas P | Method for Intelligent Browsing in an Enterprise Data System |
| CN102567204A (en) * | 2011-12-28 | 2012-07-11 | 云海创想信息技术（无锡）有限公司 | Method for managing automatic testing data |
-
2013
- 2013-08-13 CN CN201310351678.XA patent/CN103440282B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070038591A1 (en) * | 2005-08-15 | 2007-02-15 | Haub Andreas P | Method for Intelligent Browsing in an Enterprise Data System |
| CN102567204A (en) * | 2011-12-28 | 2012-07-11 | 云海创想信息技术（无锡）有限公司 | Method for managing automatic testing data |
Non-Patent Citations (1)
| Title |
|---|
| 宋铭利: "试验数据管理系统的设计与实现", 《计算机工程与设计》 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103942260A (en) * | 2014-03-21 | 2014-07-23 | 深圳海联讯科技股份有限公司 | Permission data indexing technology for structured data |
| CN104298726A (en) * | 2014-09-26 | 2015-01-21 | 惠州市亿能电子有限公司 | BMS data storage system and method based on database |
| CN104298726B (en) * | 2014-09-26 | 2018-07-03 | 惠州市亿能电子有限公司 | A kind of BMS data-storage systems and its method based on database |
| CN106528641B (en) * | 2016-10-13 | 2020-01-10 | 许继集团有限公司 | Data storage method and device and communication gateway machine |
| CN106528641A (en) * | 2016-10-13 | 2017-03-22 | 许继集团有限公司 | Data storage method and device and communication gateway machine |
| CN107944029A (en) * | 2017-12-12 | 2018-04-20 | 清华大学 | A kind of method and apparatus of the time series data storage based on frequency domain character |
| CN107944029B (en) * | 2017-12-12 | 2018-10-30 | 清华大学 | A kind of method and apparatus of the time series data storage based on frequency domain character |
| CN110609502A (en) * | 2019-09-26 | 2019-12-24 | 武汉市珞珈俊德地信科技有限公司 | Assembled map data processing system |
| WO2023123287A1 (en) * | 2021-12-30 | 2023-07-06 | 深圳晶泰科技有限公司 | Molecular data storage method and device, and molecular data application method and device |
| CN114118008A (en) * | 2022-01-21 | 2022-03-01 | 西安羚控电子科技有限公司 | Data comparison system and method based on BS architecture |
| CN114118008B (en) * | 2022-01-21 | 2022-05-10 | 西安羚控电子科技有限公司 | Data comparison system and method based on BS framework |
| CN115438514A (en) * | 2022-11-07 | 2022-12-06 | 大尧信息科技(湖南)有限公司 | Reconfigurable experiment system based on cloud server |
| CN117481667A (en) * | 2023-10-24 | 2024-02-02 | 沈阳工业大学 | Electroencephalogram signal acquisition system |
| CN118535533A (en) * | 2024-07-26 | 2024-08-23 | 北京摩弈信息科技有限公司 | Data management method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103440282B (en) | 2016-09-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103440282A (en) | Test data storage system and method | |
| US10643181B2 (en) | System and method for a big data analytics enterprise framework | |
| CN110019004B (en) | Data processing method, device and system | |
| CN103020281B (en) | A kind of data storage and retrieval method based on spatial data numerical index | |
| CN111291016A (en) | Layered mixed storage and indexing method for mass remote sensing image data | |
| CN103678491A (en) | Method based on Hadoop small file optimization and reverse index establishment | |
| CN111258978A (en) | Data storage method | |
| CN108121778B (en) | Heterogeneous data exchange and cleaning system and method | |
| CN101093489A (en) | Method and device for image search | |
| CN106326292A (en) | Data structure and file aggregation and reading methods and apparatuses | |
| CN110727406A (en) | Data storage scheduling method and device | |
| CN105608126A (en) | Method and apparatus for establishing secondary indexes for massive databases | |
| CN108604162A (en) | Optimize the access to creation data | |
| CN103984507A (en) | Storage configuration and optimizing strategy for bioinformatics high-performance computing platform | |
| CN111459943A (en) | Data processing method, device, system, equipment and storage medium | |
| CN117171108B (en) | Virtual model mapping method and system | |
| Zuo et al. | Bandwidth and energy efficient image sharing for situation awareness in disasters | |
| CN111061806B (en) | Storage method and networked access method for distributed massive geographic tiles | |
| CN111460012A (en) | Spark-based meteorological historical station leather-following data visualization method and system | |
| CN103778223A (en) | Pervasive word-reciting system based on cloud platform and construction method thereof | |
| CN111913926A (en) | Cloud platform storage method based on Hadoop | |
| Incipini et al. | Databases performance evaluation for IoT systems: The Scrovegni chapel use case | |
| KR102372732B1 (en) | Electronic device performing emulation-based forensic analysis and method of performing forensic analysis using the same | |
| CN110955709B (en) | Data processing method and device and electronic equipment | |
| Pan et al. | Research on mass image data storage method for data center |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |