CN110059153A - A kind of method for interchanging data for real-time data processing system - Google Patents
A kind of method for interchanging data for real-time data processing system Download PDFInfo
- Publication number
- CN110059153A CN110059153A CN201910246356.6A CN201910246356A CN110059153A CN 110059153 A CN110059153 A CN 110059153A CN 201910246356 A CN201910246356 A CN 201910246356A CN 110059153 A CN110059153 A CN 110059153A
- Authority
- CN
- China
- Prior art keywords
- data
- address
- area
- real
- dictionary
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
- G06F16/316—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/374—Thesaurus
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of method for interchanging data for real-time data processing system, include the following steps, write corresponding data dictionary；In continuous sheet of storage region, definition data structure is estimated when using the data structure, the data volume of required processing according to data dictionary；Initial data based on the received is searched according to data structure；After the first address for finding corresponding initial data, the data type according to defined in data dictionary is converted, and obtains the actual value of data.In real time data processing, the related definition of the data of processing and data can be all stored in the data structure using the data structure.The data access method provided by using the structure, it is possible to reduce data interchange format or content frequently change bring software modification workload, improve data processing, exchange and the efficiency used.The structure can also be shared by way of computer shared drive simultaneously, the Real Data Exchangs between realization process.
Description
Technical field
The present invention relates to the data structures and data of the data exchange of multiple two-dimensional table classes of real-time data processing system
Exchange method.
Background technique
In some data processing fields, data are often handled with the formal classification of two-dimensional table, such as some type of number
According to being determined with X-Y, X is table number, and Y is which data in the table.In data handling, generally using two-dimensional array or
Person continuous sheet of memory stores real time data.There are two deficiencies for this processing method, first is that using one-dimensional or two-dimensional array
When storing data, when data type changes, need to safeguard array in code level；Second is that storage of array side
The data of formula can only indicate the value or simple some meaning descriptions of data itself.
Summary of the invention
In order to solve the problems, such as that some data exchanges in real time data processing, the present invention establish a kind of two-dimensional linear
The storing data structure of structure, passes through some simple computations and positioning searching, it will be able to access data value and each generic attribute
Information, and support numerous types of data, it is easy to realize in different operating system.
In consideration of it, the technical solution adopted by the present invention is that: a kind of data exchange side for real-time data processing system
Method, comprising the following steps:
(1) corresponding data dictionary is write according to application scenarios；In the data dictionary to the table number of each data and
Serial number is defined, and includes code name, length and type in table, and other information can increase and decrease according to actual needs.
(2) in continuous sheet of storage region, definition data structure is estimated according to data dictionary and is using the data
When structure, the data volume of required processing, and suitable memory headroom is opened up according to data volume is related to.Generally use gross data
1.5 times of capacity.
(3) initial data based on the received, is searched according to data structure.
(4) after the first address for finding corresponding initial data, the data type according to defined in data dictionary is turned
It changes, obtains the actual value of data.
The data structure includes following 6 parts:
The number of table of the table quantity for being stored in stored table first address information area；
It is said that heading address information area is for storing every table in the first address in original data region sum number area pellucida, in number
According to the positioning in access process, realizing table；
Code name quantity is used to store the number of mapping table in data code name and X-Y map section；
Data code name and X-Y map section are used to store the corresponding relationship of data code name and data table number, serial number；
Received initial data area is for storing the received original data region without post-production；
Data illustrate area's data type defined in dictionary for storing data.
The present invention in the continuous sheet of storage region of computer, define plurality of data structures combination, realize data value,
The matching of the relevant informations such as data meaning stores.The access interface of volume of data is provided simultaneously, avoids directly operating memory band
The hidden danger such as the corrupted data, the structure destruction that come, greatly improve the ease for use and Information Security of evidence.
In real time data processing, the correlation of the data of processing and data can be determined using data structure of the present invention
It is adopted to be all stored in the data structure.By using the data access method that the structure provides, such as pass through code name access number
According to, it is possible to reduce data interchange format or content are frequently changed, and to data application layer bring software modification workload, are mentioned
High data processing, exchange and the efficiency used.The structure can also be carried out by way of computer shared drive simultaneously
It is shared, Real Data Exchangs between realization process.
Detailed description of the invention
Fig. 1 data dictionary structural schematic diagram；
Fig. 2 data profile schematic diagram；
Fig. 3 data dictionary reads flow chart；
Fig. 4 searches specified original data flow diagram；
Fig. 5 searches specified data attribute flow chart.
Specific embodiment
To make the object, technical solutions and advantages of the present invention clearer, below in conjunction with attached drawing to embodiment party of the present invention
Formula is described in further detail.
The scheme that the present invention uses is 6 parts to be divided into, referring to attached drawing in continuous sheet of storage region
2.Wherein:
Table quantity: the number for the table stored in heading address information area.
Heading address information area: it is said that every table is stored in the first address in original data region sum number area pellucida in the region,
During data access, the positioning (positioning of X) of table is realized.
Code name quantity: in data code name and X-Y map section, the number of mapping table.
Data code name and X-Y map section: the corresponding relationship of data code name and data table number, serial number is stored in the region.
Received initial data area: the region is the received original data region without post-production.Received data
It can be continuous, be also possible to discontinuous, illustrate that the relevant information in area is determined by data.
Data illustrate area: storing data name, code name title, data-bias, length, the type, list of each data
The contents such as position, theoretical value.Data dictionary sequence consensus in the contents of the section and Fig. 1, can carry out according to practical situations
Increase and decrease.Wherein data-bias stores the data in receiving data field, the deflected length (positioning of Y) in corresponding table.
Each regional structure is defined with C language as follows:
Heading address of cache area:
Data code name and X-Y map section:
Data illustrate area:
The corresponding data dictionary structure of notebook data is as shown in Figure 1, table number, serial number in data dictionary to each data
It is defined, code name, length, type are entry requireds, and other information can increase and decrease according to actual needs, or design.
The initialization of data structure, the load process of data dictionary are as shown in Figure 3:
The capacity and dependency structure body that each area size is handled according to actual needs in data structure estimate point
Match, when reading dictionary, is read out analysis by row.One mark is set before the end of each tables of data or new data table
Will judges the mark when reading dictionary.If it is new tables of data, then recalculate Xin Biao initial data area's first address and
Data illustrate area's first address.If not new tables of data, then after adding relevant information to old table.
Data search utilizes
1, tables of data initial data area first address is searched
Corresponding table initial data area first address is searched in heading address of cache area.
2, it searches tables of data and illustrates data field first address
Corresponding table data are searched in heading address of cache area illustrates area's first address.
3, the initial data of each data is searched
As shown in figure 4, being searched if it is by data code name, by code name and X-Y correspondence mappings area content, by code name
It is converted into X-Y form, by corresponding heading address and data offset address, initial data first address is obtained after being added.
Specifically, illustrating area's first address according to the data that X information finds corresponding data table；Number in corresponding data table is found according to Y information
According to detailed attributes；Offset address of the data in initial data area is found out in detailed attributes；It is found further according to X information
Initial data area first address；Data field first address obtains specific data address plus offset address.
4, the detailed attributes of each data are searched
As shown in figure 5, illustrate that area's first address and offset are positioned by corresponding data, the attribute length of each data
Be it is fixed, offset is obtained by Y attribute length.Specifically, finding corresponding data table data according to X information illustrates the first ground in area
Location；The detailed attributes of data in corresponding data table are found according to Y information.
5, data use
After the first address for finding corresponding initial data, the data type according to defined in dictionary, by the original of 16 systems
Beginning data are converted, and the actual value of data is obtained.
Claims (7)
1. a kind of method for interchanging data for real-time data processing system, which comprises the following steps:
(1) corresponding data dictionary is write according to application scenarios；
(2) in continuous sheet of storage region, definition data structure is estimated according to data dictionary and is using the data structure
When, the data volume of required processing, and suitable memory headroom is opened up according to data volume is related to；
(3) initial data based on the received, is searched according to data structure；
(4) after the first address for finding corresponding initial data, the data type according to defined in data dictionary is converted, and is obtained
To the actual value of data.
2. a kind of method for interchanging data for real-time data processing system according to claim 1, it is characterised in that: in institute
It states in data dictionary and the table number and serial number of each data is defined, include code name, length and type in table.
3. according to claim 1 or a kind of 2 method for interchanging data for real-time data processing system, it is characterised in that: described
Data structure includes table quantity, heading address information area, code name quantity, data code name and X-Y map section, received initial data
It is said that area sum number area pellucida.
4. according to where a kind of 3 method for interchanging data for real-time data processing system are required, it is characterised in that: institute
State the number of table of the table quantity for storing in stored table first address information area；
It is said that heading address information area is for storing every table in the first address in original data region sum number area pellucida, in data
In access process, the positioning of table is realized；
The code name quantity is used to store the number of mapping table in data code name and X-Y map section；
The data code name and X-Y map section are used to store the corresponding relationship of data code name and data table number, serial number；
The received initial data area is for storing the received original data region without post-production；
The data illustrate area's data type defined in dictionary for storing data.
5. according to where a kind of 4 method for interchanging data for real-time data processing system are required, it is characterised in that: institute
It states and estimates when using the data structure, when the data volume of required processing, using 1.5 times of gross data capacity.
6. special according to where a kind of 1 or 2 or 4 or 5 method for interchanging data for real-time data processing system are required
Sign is: the step (3) includes the initial data for searching each data, specific steps are as follows:
Data, which are found, according to X information illustrates area's first address；The detailed attributes of corresponding data are found according to Y information；In detailed attributes
In find out offset address of the data in initial data area；Initial data area first address is found further according to X information；The first ground in data field
Location obtains specific data address plus offset address.
7. special according to where a kind of 1 or 2 or 4 or 5 method for interchanging data for real-time data processing system are required
Sign is: the step (3) includes the data that illustrate for searching each data, specific steps are as follows: finds data explanation according to X information
Area's first address；The detailed attributes of corresponding data are found according to Y information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910246356.6A CN110059153B (en) | 2019-03-29 | 2019-03-29 | Data exchange method for real-time data processing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910246356.6A CN110059153B (en) | 2019-03-29 | 2019-03-29 | Data exchange method for real-time data processing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110059153A true CN110059153A (en) | 2019-07-26 |
| CN110059153B CN110059153B (en) | 2023-09-26 |
Family
ID=67317823
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910246356.6A Active CN110059153B (en) | 2019-03-29 | 2019-03-29 | Data exchange method for real-time data processing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110059153B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1652112A (en) * | 2005-03-02 | 2005-08-10 | 北京北方烽火科技有限公司 | Implementing method of data dictionary under embedded environment |
| CN102081644A (en) * | 2010-11-25 | 2011-06-01 | 浙江鸿程计算机系统有限公司 | Data storage method for storing data and data meanings separately |
| CN102722540A (en) * | 2012-05-23 | 2012-10-10 | 中兴通讯股份有限公司 | Method and device for processing data in real-time memory database system |
| CN103150397A (en) * | 2013-03-26 | 2013-06-12 | 北京经纬恒润科技有限公司 | Establishing method for data index, and data searching method and system |
| CN104462280A (en) * | 2014-11-27 | 2015-03-25 | 国网上海市电力公司 | Data information storage and fast inquiry method used for embedded system |
| CN104915383A (en) * | 2015-05-19 | 2015-09-16 | 中国电子科技集团公司第二十八研究所 | General method for automatic object adapting in database access |
| EP2924594A1 (en) * | 2014-03-25 | 2015-09-30 | Sap Se | Data encoding and corresponding data structure in a column-store database |
| CN106980665A (en) * | 2017-03-21 | 2017-07-25 | 广东神马搜索科技有限公司 | Data dictionary implementation method, device and data dictionary management system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102332030A (en) * | 2011-10-17 | 2012-01-25 | 中国科学院计算技术研究所 | Data storing, managing and inquiring method and system for distributed key-value storage system |
| CN103970855B (en) * | 2014-05-06 | 2017-05-10 | 车智互联（北京）科技有限公司 | Device and method for processing data |
| CN108228606A (en) * | 2016-12-14 | 2018-06-29 | 北京国双科技有限公司 | The wiring method and device of data |
-
2019
- 2019-03-29 CN CN201910246356.6A patent/CN110059153B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1652112A (en) * | 2005-03-02 | 2005-08-10 | 北京北方烽火科技有限公司 | Implementing method of data dictionary under embedded environment |
| CN102081644A (en) * | 2010-11-25 | 2011-06-01 | 浙江鸿程计算机系统有限公司 | Data storage method for storing data and data meanings separately |
| CN102722540A (en) * | 2012-05-23 | 2012-10-10 | 中兴通讯股份有限公司 | Method and device for processing data in real-time memory database system |
| CN103150397A (en) * | 2013-03-26 | 2013-06-12 | 北京经纬恒润科技有限公司 | Establishing method for data index, and data searching method and system |
| EP2924594A1 (en) * | 2014-03-25 | 2015-09-30 | Sap Se | Data encoding and corresponding data structure in a column-store database |
| CN104462280A (en) * | 2014-11-27 | 2015-03-25 | 国网上海市电力公司 | Data information storage and fast inquiry method used for embedded system |
| CN104915383A (en) * | 2015-05-19 | 2015-09-16 | 中国电子科技集团公司第二十八研究所 | General method for automatic object adapting in database access |
| CN106980665A (en) * | 2017-03-21 | 2017-07-25 | 广东神马搜索科技有限公司 | Data dictionary implementation method, device and data dictionary management system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110059153B (en) | 2023-09-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20150370545A1 (en) | Object storage and synchronization hooks for occasionally-connected devices | |
| CN103577440A (en) | Data processing method and device in non-relational database | |
| Chen et al. | An efficient algorithm for graph edit distance computation | |
| CN107391034A (en) | A kind of duplicate data detection method based on local optimization | |
| CN101551807A (en) | Multilevel index technology for file database | |
| US8001069B2 (en) | Method for using windows of similar equivalence signatures (areas of presentation spaces) in assessing the similarity of sets of data | |
| CN103186510A (en) | Document format transforming method and device | |
| US11995059B2 (en) | Database index and database query processing method, apparatus, and device | |
| US8161051B2 (en) | Method and apparatus for data processing with index search | |
| CN106682110A (en) | Video file storing and managing system and method based on Hash grid index | |
| CN104391908A (en) | Locality sensitive hashing based indexing method for multiple keywords on graphs | |
| US7792866B2 (en) | Method and system for querying structured documents stored in their native format in a database | |
| CN102542041A (en) | Method and system for processing raster data | |
| CN109002425A (en) | Acquisition methods, terminal device and the medium of enterprise's upstream-downstream relationship | |
| CN104408041A (en) | A method for storing GPS data | |
| CN101266607A (en) | High dimension data index method based on maximum clearance space mappings | |
| CN104142979A (en) | Index method for realizing RFID (Radio Frequency Identification Devices) tag storage management | |
| CN110059153A (en) | A kind of method for interchanging data for real-time data processing system | |
| CN100487697C (en) | Searching method by using modified hash method | |
| CN103902693A (en) | Method of read-optimized memory database T-tree index structure | |
| CN102855115A (en) | Method and system for displaying character label based on splicing wall | |
| CN102360370A (en) | Centralized data management system and method | |
| CN101840426B (en) | GML document storage and enquiry method based on coordinate separation | |
| CN114996270A (en) | Method and device for inquiring paging data | |
| JP2002024761A (en) | Image processing device, image processing method, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |