CN107357854A - The method, apparatus and computer-readable storage medium of data storage - Google Patents
The method, apparatus and computer-readable storage medium of data storage Download PDFInfo
- Publication number
- CN107357854A CN107357854A CN201710510129.0A CN201710510129A CN107357854A CN 107357854 A CN107357854 A CN 107357854A CN 201710510129 A CN201710510129 A CN 201710510129A CN 107357854 A CN107357854 A CN 107357854A
- Authority
- CN
- China
- Prior art keywords
- data
- stored
- length
- time
- tag
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1744—Redundancy elimination performed by the file system using compression, e.g. sparse files
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The embodiments of the invention provide a kind of method, apparatus of data storage and computer-readable storage medium, this method includes：Data to be stored are compressed according to data format, the data format includes tag number, time tag, the quality of data, next Data Position and data value；Store the data after the compression.As can be seen here, the method for data storage can take into account both data compression and decompression in the embodiment of the present invention, realize numerous types of data, and can realize that data insertion and quick-searching etc. require.
Description
Technical field
The present invention relates to process control field, the method, apparatus and computer for relating more specifically to a kind of data storage are deposited
Storage media.
Background technology
Need to take into account data compression rate for storage of history data P, while the data for needing to consider to be compressed can be quick
Ground retrieves.Due to being conflict between data compression and decompression, and current data-compressed form can not take into account both.
The content of the invention
The present invention is proposed in view of above mentioned problem.The invention provides a kind of method, apparatus of data storage and meter
Calculation machine storage medium, both data compression and decompression can be taken into account, and can realize that data insertion and quick-searching etc. require.
According to the first aspect of the invention, there is provided a kind of method of data storage, including：
Data to be stored are compressed according to data format, the data format includes tag number, time tag, data
Quality, next Data Position and data value；
Store the data after the compression.
Exemplarily, in addition to：The tag number, the time tag in the data format and it is described under
One Data Position carries out data retrieval.
Exemplarily, in addition to：According to the time tag, the data of inquiry particular moment or special time period.
Exemplarily, when being stored, more than first storing at the time of represented by the time tag of the data stored afterwards
At the time of represented by the time tag of data.
Exemplarily, the data to be stored are WORD categorical datas, and the length of the data value is 1 byte；Or institute
It is double-length floating data to state data to be stored, and the length of the data value is 8 bytes.
Exemplarily, the length of the tag number is 10 bytes, and the length of the time tag is 3 bytes, the data
Quality includes the integrity flag that length is the deleted marker of 1 and length is 7, and the length of next Data Position is
4 bytes.
Second aspect, there is provided a kind of device of data storage, including：
Compression module, for data to be stored to be compressed according to data format, the data format include tag number,
Time tag, the quality of data, next Data Position and data value；
Memory module, for storing the data after the compression.
Exemplarily, in addition to：Module is retrieved, for the tag number in the data format, the time
Label and next Data Position carry out data retrieval.
Exemplarily, in addition to：Enquiry module, for according to the time tag, inquiring about particular moment or special time
The data of section.
The method that the device is implemented for the data storage of aforementioned first aspect and its various examples.
The third aspect, there is provided a kind of device of data storage, including memory, processor and it is stored in the memory
Computer program that is upper and running on the processor, first aspect and each is realized during the computing device described program
The step of example methods described.
Fourth aspect, there is provided a kind of computer-readable storage medium, be stored thereon with computer program, described program is processed
The step of device realizes first aspect and each example methods described when performing.
As can be seen here, the method for data storage can take into account both data compression and decompression in the embodiment of the present invention, real
Existing numerous types of data, and can realize that data insertion and quick-searching etc. require.
Brief description of the drawings
The embodiment of the present invention is described in more detail in conjunction with the accompanying drawings, above-mentioned and other purpose of the invention,
Feature and advantage will be apparent.Accompanying drawing is used for providing further understanding the embodiment of the present invention, and forms explanation
A part for book, it is used to explain the present invention together with the embodiment of the present invention, is not construed as limiting the invention.In the accompanying drawings,
Identical reference number typically represents same parts or step.
Fig. 1 is an indicative flowchart of the method for the data storage of the embodiment of the present invention；
Fig. 2 is the preservation form of the WORD categorical datas of the embodiment of the present invention；
Fig. 3 is the preservation form of the double-length floating data of the embodiment of the present invention；
Fig. 4 is a schematic block diagram of the device of the data storage of the embodiment of the present invention.
Embodiment
Become apparent in order that obtaining the object, technical solutions and advantages of the present invention, root is described in detail below with reference to accompanying drawings
According to the example embodiment of the present invention.Obviously, described embodiment is only the part of the embodiment of the present invention, rather than this hair
Bright whole embodiments, it should be appreciated that the present invention is not limited by example embodiment described herein.Described in the present invention
The embodiment of the present invention, those skilled in the art's all other embodiment resulting in the case where not paying creative work
It should all fall under the scope of the present invention.
Fig. 1 is an indicative flowchart of the method for the data storage of the embodiment of the present invention.Method shown in Fig. 1 can be with
Including：
S101, data to be stored are compressed according to data format, the data format includes tag number, the time marks
Label, the quality of data, next Data Position and data value.
S102, store the data after the compression.
It should be noted that although Fig. 1 shows the process of the data storage, those skilled in the art with the step of S101 and S102
It is understood that S102 can be performed while S101 is performed, that is to say, that compression and storage can be performed in parallel, such energy
The enough efficiency for ensureing storage.
Exemplarily, the length of the tag number is 10 bytes (Byte), and the length of the time tag is 3 bytes, institute
Stating the quality of data includes the integrity flag that length is the deleted marker of 1 (bit) and length is 7, next data
The length of position is 4 bytes.
Wherein, the mark for the data value that tag number can be in the data format.It should be noted that when data increase tag number,
The data are newest.
Wherein, time tag can be used to indicate that the current time for storing the data to be stored, and be deposited after being appreciated that
At the time of at the time of represented by the time tag of the data of storage more than represented by the time tag of the data first stored.Namely
Say, when data are preserved, new data time is always greater than data time before.Therefore, the time tag can be used
In the priority for representing the stored time.
Wherein, next Data Position represents the tag number that next data value after the data value is stored, tool
Body, can be the storage location of the initial value of the tag number of next data value.
It should be noted that the embodiment of the present invention to the time tag in data format, the quality of data, next Data Position it is suitable
Sequence is not construed as limiting.For example, the field of the quality of data can be located at before the field of time tag, Fig. 2 and Fig. 3 institutes as after
Show.For example, the field of next Data Position can be located at before or after the field of time tag.
The embodiment of the present invention can support numerous types of data, such as can support WORD categorical datas, double-precision floating point
Categorical data.
As a kind of implementation, the data to be stored in S101 are WORD categorical datas, and the length of the data value is 1
Byte.The preservation form of WORD categorical datas is illustrated in figure 2, tag number of the data format shown in Fig. 2 including 10 bytes,
The deleted marker of 1, the integrality of 7, the time of 3 bytes, the data value of the subsequent point position of 4 bytes and 1 byte, it is grown
Degree is 19 bytes altogether.
As another implementation, the data to be stored in S101 are double-length floating data, the data value
Length be 8 bytes.The preservation form of double-length floating data is illustrated in figure 3, the data format shown in Fig. 3 includes
The tag number of 10 bytes, the deleted marker of 1, the integrality of 7, the time of 3 bytes, the subsequent point position of 4 bytes and 8 words
The data value of section, its length are 26 bytes altogether.
It should be noted that the subsequent point position in Fig. 2 and Fig. 3 is foregoing next Data Position.
Alternatively, after the method shown in Fig. 1, the data stored can also be retrieved.By thus according to institute
The data format of setting is compressed rear and stored, and can quickly and easily be found by next Data Position next
Individual data value, therefore the speed of retrieval can be greatly enhanced, specifically, the retrieval of the historical data to being stored can be improved
Speed.As can be seen here, the embodiment of the present invention has taken into account data compression rate and quick-searching simultaneously.
Exemplarily, can be according to the tag number in the data format, the time tag and described next
Individual Data Position carries out data retrieval.That is, can be using tag number, time tag and next Data Position as index
To be retrieved, so as to obtain data corresponding with indexing.
Exemplarily, the data of particular moment or special time period can according to the time tag, be inquired about.Namely
Say, data can be polled by two ways：For a kind of mode to inquire about the data at some moment, another way is to inquire about certain
The data of one period.For example, some moment can be expressed as t0, some period can be expressed as [t1,
t2]。
As can be seen here, the method for data storage can take into account both data compression and decompression in the embodiment of the present invention, real
Existing numerous types of data, and can realize that data insertion and quick-searching etc. require.
Fig. 4 is a schematic block diagram of the device of the data storage of the embodiment of the present invention.Device 40 shown in Fig. 4 can be with
Including：Compression module 401 and memory module 402.
Compression module 401, for being compressed data to be stored according to data format, the data format includes label
Number, time tag, the quality of data, next Data Position and data value.
Memory module 402, for storing the data after the compression.
Exemplarily, device 40 can also include：Module is retrieved, for the label in the data format
Number, the time tag and the next Data Position carry out data retrieval.
Exemplarily, device 40 can also include：Enquiry module, for according to the time tag, inquiring about particular moment
Or the data of special time period.
Exemplarily, memory module 402 is when being stored, big at the time of represented by the time tag of the data stored afterwards
At the time of represented by time tag in the data first stored.
Exemplarily, the data to be stored are WORD categorical datas, and the length of the data value is 1 byte.
Exemplarily, the data to be stored are double-length floating data, and the length of the data value is 8 bytes.
Exemplarily, the length of the tag number is 10 bytes, and the length of the time tag is 3 bytes, the data
Quality includes the integrity flag that length is the deleted marker of 1 and length is 7, and the length of next Data Position is
4 bytes.
The method that device 40 shown in Fig. 4 can realize the data storage shown in earlier figures 1, to avoid repeating, here not
Repeat again.
In addition, the embodiment of the present invention additionally provides the device of another data storage, including memory, processor and storage
The computer program run on the memory and on the processor, earlier figures are realized during computing device described program
Shown in 1 the step of method.
In addition, the embodiment of the present invention additionally provides a kind of computer-readable storage medium, computer program is stored thereon with.Work as institute
When stating computer program by computing device, it is possible to achieve shown in earlier figures 1 the step of method.For example, computer storage is situated between
Matter is computer-readable recording medium.
As can be seen here, the method for data storage can take into account both data compression and decompression in the embodiment of the present invention, real
Existing numerous types of data, and can realize that data insertion and quick-searching etc. require.
Although describe example embodiment by reference to accompanying drawing here, it should be understood that above-mentioned example embodiment is merely exemplary
, and be not intended to limit the scope of the invention to this.Those of ordinary skill in the art can carry out various changes wherein
And modification, it is made without departing from the scope of the present invention and spiritual.All such changes and modifications are intended to be included in appended claims
Within required the scope of the present invention.
Those of ordinary skill in the art are it is to be appreciated that the list of each example described with reference to the embodiments described herein
Member and algorithm steps, it can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually
Performed with hardware or software mode, application-specific and design constraint depending on technical scheme.Professional and technical personnel
Described function can be realized using distinct methods to each specific application, but this realization is it is not considered that exceed
The scope of the present invention.
In addition, it will be appreciated by those of skill in the art that although some embodiments described herein include other embodiments
In included some features rather than further feature, but the combination of the feature of different embodiments means in of the invention
Within the scope of and form different embodiments.For example, in detail in the claims, embodiment claimed it is one of any
Mode it can use in any combination.
The all parts embodiment of the present invention can be realized with hardware, or to be run on one or more processor
Software module realize, or realized with combinations thereof.It will be understood by those of skill in the art that it can use in practice
Microprocessor or digital signal processor (DSP) realize some moulds in article analytical equipment according to embodiments of the present invention
The some or all functions of block.The present invention is also implemented as the part or complete for performing method as described herein
The program of device (for example, computer program and computer program product) in portion.Such program for realizing the present invention can store
On a computer-readable medium, or can the form with one or more signal.Such signal can be from internet
Download and obtain on website, either provide on carrier signal or provided in the form of any other.
The foregoing is only a specific embodiment of the invention or the explanation to embodiment, protection of the invention
Scope is not limited thereto, any one skilled in the art the invention discloses technical scope in, can be easily
Expect change or replacement, should all be included within the scope of the present invention.Protection scope of the present invention should be with claim
Protection domain is defined.
Claims (11)
- A kind of 1. method of data storage, it is characterised in that including：Data to be stored are compressed according to data format, the data format includes tag number, time tag, data matter Amount, next Data Position and data value；Store the data after the compression.
- 2. according to the method for claim 1, it is characterised in that also include：The tag number, the time tag and next Data Position in the data format carry out data Retrieval.
- 3. according to the method for claim 1, it is characterised in that also include：According to the time tag, the data of inquiry particular moment or special time period.
- 4. according to the method for claim 1, it is characterised in that when being stored, the time tag of the data stored afterwards At the time of at the time of represented more than represented by the time tag of the data first stored.
- 5. according to the method for claim 1, it is characterised in thatThe data to be stored are WORD categorical datas, and the length of the data value is 1 byte；Or the data to be stored are double-length floating data, the length of the data value is 8 bytes.
- 6. according to the method for claim 1, it is characterised in that the length of the tag number is 10 bytes, the time mark The length of label is 3 bytes, and the quality of data includes the integrity flag that length is the deleted marker of 1 and length is 7, institute The length for stating next Data Position is 4 bytes.
- A kind of 7. device of data storage, it is characterised in that including：Compression module, for being compressed data to be stored according to data format, the data format includes tag number, time Label, the quality of data, next Data Position and data value；Memory module, for storing the data after the compression.
- 8. device according to claim 7, it is characterised in that also include：Module is retrieved, for the tag number in the data format, the time tag and next number Data retrieval is carried out according to position.
- 9. device according to claim 7, it is characterised in that also include：Enquiry module, for according to the time tag, inquiring about the data of particular moment or special time period.
- 10. a kind of device of data storage, including memory, processor and it is stored on the memory and in the processor The computer program of upper operation, it is characterised in that realized during the computing device described program any in claim 1 to 6 The step of item methods described.
- 11. a kind of computer-readable storage medium, is stored thereon with computer program, it is characterised in that described program is held by processor The step of method any one of claim 1 to 6 is realized during row.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710510129.0A CN107357854A (en) | 2017-06-28 | 2017-06-28 | The method, apparatus and computer-readable storage medium of data storage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710510129.0A CN107357854A (en) | 2017-06-28 | 2017-06-28 | The method, apparatus and computer-readable storage medium of data storage |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107357854A true CN107357854A (en) | 2017-11-17 |
Family
ID=60274100
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710510129.0A Pending CN107357854A (en) | 2017-06-28 | 2017-06-28 | The method, apparatus and computer-readable storage medium of data storage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107357854A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1242846A (en) * | 1997-10-21 | 2000-01-26 | 富士通株式会社 | File processing method, data processing device, and storage medium |
| US20050047666A1 (en) * | 2003-08-21 | 2005-03-03 | Mitchell Joan L. | Browsing JPEG images using MPEG hardware chips |
| CN101853287A (en) * | 2010-05-24 | 2010-10-06 | 南京高普科技有限公司 | Data compression quick retrieval file system and method thereof |
| CN106557272A (en) * | 2015-09-30 | 2017-04-05 | 中国科学院软件研究所 | A kind of efficient sensor historic data archiving method |
| CN106846543A (en) * | 2016-12-23 | 2017-06-13 | 鸿秦（北京）科技有限公司 | Data compression device and implementation method in a kind of new-energy automobile black box |
-
2017
- 2017-06-28 CN CN201710510129.0A patent/CN107357854A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1242846A (en) * | 1997-10-21 | 2000-01-26 | 富士通株式会社 | File processing method, data processing device, and storage medium |
| US20050047666A1 (en) * | 2003-08-21 | 2005-03-03 | Mitchell Joan L. | Browsing JPEG images using MPEG hardware chips |
| CN101853287A (en) * | 2010-05-24 | 2010-10-06 | 南京高普科技有限公司 | Data compression quick retrieval file system and method thereof |
| CN106557272A (en) * | 2015-09-30 | 2017-04-05 | 中国科学院软件研究所 | A kind of efficient sensor historic data archiving method |
| CN106846543A (en) * | 2016-12-23 | 2017-06-13 | 鸿秦（北京）科技有限公司 | Data compression device and implementation method in a kind of new-energy automobile black box |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102307957B1 (en) | Stream selection for multi-stream storage | |
| CN104462141B (en) | Method, system and the storage engines device of a kind of data storage and inquiry | |
| US20150127621A1 (en) | Use of solid state storage devices and the like in data deduplication | |
| CN103870514B (en) | Data de-duplication method and device | |
| CN107045531A (en) | A kind of system and method for optimization HDFS small documents access | |
| CN107688591B (en) | A kind of actuarial treating method and apparatus | |
| CN104408190B (en) | Data processing method and device based on Spark | |
| JP2012513069A (en) | Method and apparatus for content-aware data partitioning and data deduplication | |
| CN109564545A (en) | Method and apparatus for compressing address | |
| CN105607960A (en) | Repairing method and device of file system directory tree | |
| CN106600708B (en) | A kind of processing method and system of map | |
| CN103632250A (en) | Quick sales order sorting, grouping and screening method | |
| EP2946295A1 (en) | Grouping chunks of data into a compression region | |
| CN107707600A (en) | A kind of date storage method and device | |
| CN108052643A (en) | Date storage method, device and storage engines based on LSM Tree structures | |
| CN104750432B (en) | A kind of date storage method and device | |
| CN107741889A (en) | Document recovery method, system, readable storage medium storing program for executing and terminal | |
| CA2398148A1 (en) | System and method for managing bi-directional relationships between objects | |
| CN107330119A (en) | Caching data processing method, electronic equipment, computer-readable storage medium | |
| CN108090125A (en) | A kind of data de-duplication method and device of non-query formulation | |
| CN107239485A (en) | Database operation method, apparatus and system | |
| CN105159925B (en) | A kind of data-base cluster data distributing method and system | |
| CN108681866A (en) | Processing method, system, equipment and the storage medium of waybill | |
| CN111930556A (en) | Backup method and device for distributed storage data | |
| CN107357854A (en) | The method, apparatus and computer-readable storage medium of data storage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20171117 |
|
| WD01 | Invention patent application deemed withdrawn after publication |