CN108595714A - A kind of date storage method, medium and equipment based on data base querying - Google Patents
A kind of date storage method, medium and equipment based on data base querying Download PDFInfo
- Publication number
- CN108595714A CN108595714A CN201810457079.9A CN201810457079A CN108595714A CN 108595714 A CN108595714 A CN 108595714A CN 201810457079 A CN201810457079 A CN 201810457079A CN 108595714 A CN108595714 A CN 108595714A
- Authority
- CN
- China
- Prior art keywords
- data
- server
- tables
- alive
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The present invention provides a kind of date storage method, medium and equipment based on data base querying.The method, including：By in relation data tables of data and blob files distinguish；Select the alive data in the tables of data；It will be in alive data storage at least one server.By by relation data tables of data and blob files distinguish, relatively active data in reselection tables of data, it will be in alive data storage at least one server, in this way, the number that data between different server are transferred can be reduced, and then reduce the bandwidth of data processing, improve the speed of data processing.
Description
Technical field
The present invention relates to field of database query technology, and in particular to a kind of data storage side based on data base querying
Method, medium and equipment.
Background technology
When handling concurrent data inquiry, tables of data is stored separately in by row or by row in different servers, and
The data stored in each server do not repeat, in this way, if a server needs other servers when handling data
In data, then need to transfer the data from other servers, undergo from the sequence of network interface card-CPU- memories-disk/hard disk,
The data needed are obtained, compare occupied bandwidth, and the speed of data processing can be reduced.
Invention content
For the defects in the prior art, the present invention provides a kind of date storage method based on data base querying, can
The number that data are transferred between reduction different server, and then reduce the bandwidth of data processing, improve the speed of data processing.
In a first aspect, the present invention provides a kind of date storage methods based on data base querying, including：
By in relation data tables of data and blob files distinguish；
Select the alive data in the tables of data；
It will be in alive data storage at least one server.
Optionally, the alive data in the selection tables of data, including：
Count the number that each field in the tables of data per data is accessed within a preset period of time；
According to the accessed number, the alive data in the tables of data is selected.
Optionally, in the storage at least one server by the alive data, including：
Count the memory space and idleness of each server within a preset period of time；
It, will be in alive data storage at least one server according to the memory space and idleness.
Optionally, further include：
The blob files are uniformly distributed and are stored in each server.
Optionally, each field per data in the statistics tables of data is accessed within a preset period of time
After the step of number, further include：
According to the accessed number, histogram is generated, and export the histogram.
Second aspect, the present invention provide a kind of computer readable storage medium, are stored thereon with computer program, the program
A kind of date storage method based on data base querying is realized when being executed by processor.
The third aspect, the present invention provide a kind of data storage device based on data base querying, including：Memory, processing
On a memory and the computer program that can run on a processor, when processor execution described program, is realized for device and storage
A kind of date storage method based on data base querying.
The present invention provides a kind of date storage method based on data base querying, by by relation data tables of data and
Blob files distinguish, relatively active data in reselection tables of data, by alive data storage at least one service
In device, in such manner, it is possible to reduce the number that data between different server are transferred, and then reduces the bandwidth of data processing, improve number
According to the speed of processing.
The present invention provides a kind of computer readable storage medium and a kind of data storage device based on data base querying, with
It is above-mentioned it is a kind of based on the date storage method of data base querying for identical inventive concept, advantageous effect having the same.
Description of the drawings
It, below will be to specific in order to illustrate more clearly of the specific embodiment of the invention or technical solution in the prior art
Embodiment or attached drawing needed to be used in the description of the prior art are briefly described.In all the appended drawings, similar element
Or part is generally identified by similar reference numeral.In attached drawing, each element or part might not be drawn according to actual ratio.
Fig. 1 is a kind of flow chart of data storage based on data base querying provided by the invention；
Fig. 2 is a kind of principle schematic of the conventional method of distributed data management provided by the invention.
Specific implementation mode
The embodiment of technical solution of the present invention is described in detail below in conjunction with attached drawing.Following embodiment is only used for
Clearly illustrate technical scheme of the present invention, therefore be intended only as example, and the protection of the present invention cannot be limited with this
Range.
It should be noted that unless otherwise indicated, technical term or scientific terminology used in this application should be this hair
The ordinary meaning that bright one of ordinary skill in the art are understood.
The present invention provides a kind of date storage method, medium and equipment based on data base querying.Below in conjunction with the accompanying drawings
The embodiment of the present invention is illustrated.
First embodiment：
Referring to FIG. 1, Fig. 1 is a kind of date storage method based on data base querying that the specific embodiment of the invention provides
Flow chart, a kind of date storage method based on data base querying provided in this embodiment, including：
Step S101：By in relation data tables of data and blob files distinguish.
Step S102：Select the alive data in the tables of data.
Step S103：It will be in alive data storage at least one server.
The present invention is suitable for computer concurrent processing SQL query statement, that is, when relation data amount is more than computer simultaneously
Send out number.
General server is when handling data, from other servers when called data, as shown in Fig. 2, need undergo from
The sequence of network interface card-CPU- memories-disk/hard disk obtains data.CPU needs elder generation in the data during processing handles book server
By data from storage is extracted in disk/hard disk in memory, then the data are transferred from memory, then handled.
The present invention is directed to the data storage method in disk/hard disk and carries out method improvement, can improve the speed of data processing
Degree.
By by relation data tables of data and blob files distinguish, it is relatively active in reselection tables of data
Data store alive data at least one server, in such manner, it is possible to reduce time that data between different server are transferred
Number, and then reduce the bandwidth of data processing, improve the speed of data processing.
Wherein, in tables of data include a plurality of data, include the corresponding data of multiple fields per data.
Wherein, alive data is possibly stored in a server, can also be stored in multiple servers, it is preferred that
It is stored with alive data in each server.
In a specific embodiment provided by the invention, the alive data in the selection tables of data, including：System
Count the number that each field in the tables of data per data is accessed within a preset period of time；According to accessed time
Number, selects the alive data in the tables of data.
First, the number that each field in statistics table per data is accessed within a preset period of time；Then, sentence
Whether the accessed number of disconnected each field per data is more than preset times, if more than corresponding data are then judged to live
Jump data；Finally, the alive data that accessed number is more than to preset times is cut out, can in cutting alive data
In a manner of being combined using longitudinally cutting and transverse cuts, it is cut into alive data.
Wherein, alive data may include the data of multiple fields in a plurality of data.
It, can will be accessed after the number that each field counted per data is accessed within a preset period of time
Number generates histogram, and access times are observed convenient for user.
It is described by alive data storage at least one server in a specific embodiment provided by the invention
In, including：Count the memory space and idleness of each server within a preset period of time；According to the memory space and free time
Rate, will be in alive data storage at least one server.
In the memory space and idleness of statistical server, can only count in preset time period, with preset time
Data in section are as reference.
When the memory space of server is larger, can all store all alive datas in the hard disk of server,
It can choose in partial data or the total data storage to server in alive data.When server memory space compared with
It is small, then it can choose in the partial data storage to server in alive data.
It, then can will be in alive data storage to common server when the idleness of server is higher；If server
Idleness is relatively low, substantially all in working condition when, alive data can be stored in each server.
By that by alive data storage to multiple servers, storage data can be repeated in the server, improve data
The efficiency of inquiry.
In a specific embodiment provided by the invention, can also include：The blob files are uniformly distributed storage
In each server.
In the present invention, since blob file committed memories are larger, then need blob files being uniformly distributed be stored in it is each
In a server, the blob files stored in each server is made not repeat, in such manner, it is possible to reduce the occupancy of server hard disc
Space.When needs, which are applicable in, does not have to the blob files in server, then transferred from other servers.
More than, it is a kind of date storage method based on data base querying provided by the invention.
Second embodiment：
In above-mentioned first embodiment, a kind of date storage method based on data base querying is provided, in conjunction with above-mentioned
First embodiment, second embodiment of the invention provide a kind of computer readable storage medium, are stored thereon with computer program, should
A kind of date storage method based on data base querying that above-mentioned first embodiment provides is realized when program is executed by processor.
3rd embodiment：
In conjunction with a kind of date storage method based on data base querying that first embodiment provides, the present invention also provides one kind
Data storage device based on data base querying, including：Memory, processor and storage are on a memory and can be on a processor
The computer program of operation, the processor realize that one kind that above-mentioned first embodiment provides is based on data when executing described program
The date storage method of library inquiry.
In the specification of the present invention, numerous specific details are set forth.It is to be appreciated, however, that the embodiment of the present invention can be with
It puts into practice without these specific details.In some instances, well known method, structure and skill is not been shown in detail
Art, so as not to obscure the understanding of this description.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or spy described in conjunction with this embodiment or example
Point is included at least one embodiment or example of the invention.In the present specification, schematic expression of the above terms are not
It must be directed to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be in office
It can be combined in any suitable manner in one or more embodiments or example.In addition, without conflicting with each other, the skill of this field
Art personnel can tie the feature of different embodiments or examples described in this specification and different embodiments or examples
It closes and combines.
Finally it should be noted that：The above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent
Present invention has been described in detail with reference to the aforementioned embodiments for pipe, it will be understood by those of ordinary skill in the art that：Its according to
So can with technical scheme described in the above embodiments is modified, either to which part or all technical features into
Row equivalent replacement；And these modifications or replacements, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution
The range of scheme should all cover in the claim of the present invention and the range of specification.
Claims (7)
1. a kind of date storage method based on data base querying, which is characterized in that including：
By in relation data tables of data and blob files distinguish；
Select the alive data in the tables of data；
It will be in alive data storage at least one server.
2. according to the method described in claim 1, it is characterized in that, alive data in the selection tables of data, including：
Count the number that each field in the tables of data per data is accessed within a preset period of time；
According to the accessed number, the alive data in the tables of data is selected.
3. according to the method described in claim 1, it is characterized in that, described by alive data storage at least one service
In device, including：
Count the memory space and idleness of each server within a preset period of time；
It, will be in alive data storage at least one server according to the memory space and idleness.
4. according to the method described in claim 1, it is characterized in that, further including：
The blob files are uniformly distributed and are stored in each server.
5. according to the method described in claim 2, it is characterized in that, each per data in the statistics tables of data
After the step of number that field is accessed within a preset period of time, further include：
According to the accessed number, histogram is generated, and export the histogram.
6. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is held by processor
The method described in one of claim 1-5 is realized when row.
7. a kind of data storage device based on data base querying, including：Memory, processor and storage are on a memory and can
The computer program run on a processor, which is characterized in that the processor realizes claim 1-5 when executing described program
One of described in method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810457079.9A CN108595714A (en) | 2018-05-14 | 2018-05-14 | A kind of date storage method, medium and equipment based on data base querying |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810457079.9A CN108595714A (en) | 2018-05-14 | 2018-05-14 | A kind of date storage method, medium and equipment based on data base querying |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108595714A true CN108595714A (en) | 2018-09-28 |
Family
ID=63637430
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810457079.9A Pending CN108595714A (en) | 2018-05-14 | 2018-05-14 | A kind of date storage method, medium and equipment based on data base querying |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108595714A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102043830A (en) * | 2010-11-17 | 2011-05-04 | 北京握奇数据系统有限公司 | Method and device for processing BLOB (Binary Large Object) type data and intelligent card |
| CN103473276A (en) * | 2013-08-26 | 2013-12-25 | 广东电网公司电力调度控制中心 | Storage method of very large data and distributed database system and retrieval method thereof |
| CN103793487A (en) * | 2014-01-20 | 2014-05-14 | 中国建设银行股份有限公司 | Method and system for centrally extracting and storing electronic seal impression account data |
| US20140143256A1 (en) * | 2012-11-19 | 2014-05-22 | Jonathan Mooser | Searching entity-key associations using in-memory objects |
| CN105426377A (en) * | 2014-09-23 | 2016-03-23 | 中国移动通信集团广西有限公司 | Data management method, apparatus and system |
| CN105930370A (en) * | 2016-04-13 | 2016-09-07 | 曙光信息产业(北京)有限公司 | Data monitoring method and device |
-
2018
- 2018-05-14 CN CN201810457079.9A patent/CN108595714A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102043830A (en) * | 2010-11-17 | 2011-05-04 | 北京握奇数据系统有限公司 | Method and device for processing BLOB (Binary Large Object) type data and intelligent card |
| US20140143256A1 (en) * | 2012-11-19 | 2014-05-22 | Jonathan Mooser | Searching entity-key associations using in-memory objects |
| CN103473276A (en) * | 2013-08-26 | 2013-12-25 | 广东电网公司电力调度控制中心 | Storage method of very large data and distributed database system and retrieval method thereof |
| CN103793487A (en) * | 2014-01-20 | 2014-05-14 | 中国建设银行股份有限公司 | Method and system for centrally extracting and storing electronic seal impression account data |
| CN105426377A (en) * | 2014-09-23 | 2016-03-23 | 中国移动通信集团广西有限公司 | Data management method, apparatus and system |
| CN105930370A (en) * | 2016-04-13 | 2016-09-07 | 曙光信息产业(北京)有限公司 | Data monitoring method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11403275B2 (en) | Incremental reclustering of database tables using reclustering-count levels | |
| US7895210B2 (en) | Methods and apparatuses for information analysis on shared and distributed computing systems | |
| US20090307329A1 (en) | Adaptive file placement in a distributed file system | |
| US7472108B2 (en) | Statistics collection using path-value pairs for relational databases | |
| CN102819586A (en) | Uniform Resource Locator (URL) classifying method and equipment based on cache | |
| US20220083246A1 (en) | Variable Length Deduplication of Stored Data | |
| US20220405285A1 (en) | Cutoffs for pruning of database queries | |
| CN105653647A (en) | Information acquisition method and system of SQL (Structured Query Language) statement | |
| US9117005B2 (en) | Statistics collection using path-value pairs for relational databases | |
| CN106649800A (en) | Solr-based Chinese search method | |
| US9852171B2 (en) | Index masking based on insert contention ratio | |
| US8700822B2 (en) | Parallel aggregation system | |
| CN108595714A (en) | A kind of date storage method, medium and equipment based on data base querying | |
| CN105447183A (en) | MPP framework database cluster sequence system and sequence management method | |
| CN104572723A (en) | File access method and file access device | |
| US11061932B2 (en) | Methods and system for managing storage with multiple simultaneous data temperature models | |
| Noble et al. | Optimize T-SQL | |
| Dessers et al. | Organizing care integration. A social systems based approach | |
| CN108062310A (en) | A kind of method and system of distributed data retrieval result Pagination Display | |
| EP2657862A1 (en) | Parallel set aggregation | |
| JP2000148549A (en) | Data storage management device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180928 |