CN104765749B - A kind of date storage method and device - Google Patents
A kind of date storage method and device Download PDFInfo
- Publication number
- CN104765749B CN104765749B CN201410006688.4A CN201410006688A CN104765749B CN 104765749 B CN104765749 B CN 104765749B CN 201410006688 A CN201410006688 A CN 201410006688A CN 104765749 B CN104765749 B CN 104765749B
- Authority
- CN
- China
- Prior art keywords
- attribute
- monitoring data
- bit
- data
- coding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application discloses a kind of date storage method and device, to when solving that monitoring data amount is larger in the prior art, database cannot store monitoring data in time, or even the problem of delay machine occur.This method determines attribute-bit set belonging to the attribute-bit carried in monitoring data, as specified attribute logo collection, according to the attribute-bit set set in advance for each database server in distributed data base, by the supervising data storage in the corresponding database server of the specified attribute logo collection.Pass through the above method, the different monitoring data of attribute-bit set belonging to attribute-bit can be respectively stored in database server different in distributed data base, and since the I/O ability of distributed data base is stronger compared to the I/O ability of single machine database, even if therefore the data volume of monitoring data is larger, distributed data base can also timely receive and store monitoring data, can effectively avoid database delay machine.
Description
Technical field
This application involves field of computer technology more particularly to a kind of date storage methods and device.
Background technique
Each target device in cluster device is supervised currently, generalling use the cluster monitoring systems such as Ganglia
Control acquires monitoring data, and in the database by supervising data storage.
In the prior art, general using RRD(Round Robin Database) database stores monitoring data, prison
The memory system architecture for controlling data is as shown in Figure 1.
Fig. 1 is the memory system architecture schematic diagram of monitoring data in the prior art, in Fig. 1, by Ganglia cluster monitoring
Monitoring server (Fig. 1 is not shown) in system is monitored the target device in cluster device, acquires each target device
Monitoring data, after each monitoring server in Ganglia cluster monitoring system collects monitoring data, the monitoring data that will obtain
It is sent to RRD database purchase.
But when the quantity of the target device in the cluster device of monitoring more (such as more than 5000 target devices),
The data volume for the monitoring data that cluster monitoring system obtains is also very huge, and since RRD database is the data of a single machine
Library, input/output (Input/Output, I/O) ability is limited, therefore when the data volume of monitoring data is larger, is limited by
The I/O ability of RRD database, RRD database cannot will timely receive the monitoring data and storage that cluster monitoring system is sent,
Even there is the case where RRD database delay machine.
Summary of the invention
The embodiment of the present application provides a kind of date storage method and device, to solve in the prior art when monitoring data
When data volume is larger, database cannot receive and store monitoring data in time, or even the problem of delay machine occur.
A kind of date storage method provided by the embodiments of the present application, the method are applied to include multiple database services
In the distributed data base of device, each database server is corresponding with the attribute-bit set divided in advance, the method
Include:
Monitoring data is obtained, carries attribute-bit in the monitoring data；
Attribute-bit set belonging to the attribute-bit carried in the monitoring data obtained is determined, as specified attribute
Logo collection；
By the supervising data storage in the corresponding database server of the specified attribute logo collection.
A kind of data storage device provided by the embodiments of the present application, described device are applied to include multiple database services
In the distributed data base of device, each database server is corresponding with the attribute-bit set divided in advance, described device
Include:
It obtains module and carries attribute-bit in the monitoring data for obtaining monitoring data；
Determining module, for determining attribute-bit collection belonging to the attribute-bit carried in the monitoring data obtained
It closes, as specified attribute logo collection；
Memory module is used for the supervising data storage in the corresponding database service of the specified attribute logo collection
In device.
The embodiment of the present application provides a kind of date storage method and device, and this method obtains monitoring data, and determination acquires
Monitoring data in attribute-bit set belonging to the attribute-bit that carries, as specified attribute logo collection, and according to preparatory
For the attribute-bit set of each database server setting in distributed data base, by the supervising data storage in institute
It states in the corresponding database server of specified attribute logo collection.By the above method, for attribute mark belonging to attribute-bit
Know and gather for different monitoring datas, these monitoring datas can be respectively stored in database clothes different in distributed data base
It is engaged in device, and since the I/O ability of distributed data base is stronger compared to the I/O ability of single machine database, even if monitoring
The data volume of data is larger, and distributed data base can also timely receive and store monitoring data, can effectively avoid database and delays
Machine.
Detailed description of the invention
The drawings described herein are used to provide a further understanding of the present application, constitutes part of this application, this Shen
Illustrative embodiments and their description please are not constituted an undue limitation on the present application for explaining the application.In the accompanying drawings:
Fig. 1 is the memory system architecture schematic diagram of monitoring data in the prior art；
Fig. 2 is data storage procedure provided by the embodiments of the present application；
Fig. 3 is data storage device structural schematic diagram provided by the embodiments of the present application；
Fig. 4 is provided by the embodiments of the present application to be illustrated so that data storage device shown in Fig. 3 is independent equipment as an example
Data storage system architecture schematic diagram.
Specific embodiment
Since the I/O ability of distributed data base depends on the quantity of database server in distributed data base, data
The quantity of library server is more, and the I/O ability of distributed data base is higher, and theoretically the I/O ability of distributed data base is not deposited
In bottleneck problem, therefore, monitoring data is stored using distributed data base in the embodiment of the present application, a large amount of monitoring datas will be stored
The I/O to be consumed shares in each database server in distributed data base, allows monitoring data by timely
Receive and storage, also can effectively avoid database due to I/O scarce capacity delay machine.
To keep the purposes, technical schemes and advantages of the application clearer, below in conjunction with the application specific embodiment and
Technical scheme is clearly and completely described in corresponding attached drawing.Obviously, described embodiment is only the application one
Section Example, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art are not doing
Every other embodiment obtained under the premise of creative work out, shall fall in the protection scope of this application.
Fig. 2 is data storage procedure provided by the embodiments of the present application, and wherein the data storage procedure is applied to include more
In the distributed data base of a database server, each database server and the attribute-bit set pair divided in advance
It answers, specifically includes the following steps:
S201: monitoring data is obtained, carries attribute-bit in the monitoring data.
In the embodiment of the present application, the collected monitoring data of cluster monitoring system can be obtained by data storage device.
Wherein, cluster monitoring system includes but is not limited to Ganglia cluster monitoring system, and data storage device can be located at cluster monitoring
In system, it is also possible to the equipment independently of cluster monitoring system, is below that an independent equipment is with data storage device
Example is illustrated.
S202: attribute-bit set belonging to the attribute-bit carried in the monitoring data obtained is determined, as specified category
Property logo collection.
In the embodiment of the present application, it is carried in the monitoring data that cluster monitoring system is monitored target device
Attribute-bit, the attribute-bit can be formed by the device identification of the target device and the index identifier combination of monitor control index.
For example, it is assumed that cluster monitoring system will the memory service condition to some target device be monitored, then the target
The device identification of equipment can be the Internet protocol (Int of the target deviceernet ProtocoL, IP) address or media access
Control (Media Access ControL, MAC) address, monitor control index is " memory use ", and can presetting monitor control index, " memory makes
With " corresponding index is identified as 001, then cluster monitoring system is monitored the memory service condition of the target device, obtains
Monitoring data in the attribute-bit that carries be " IP address or MAC Address+001 of the target device ".
Further, can be previously according to the attribute-bits being likely to occur all in monitoring data in the embodiment of the present application, it will
All attribute-bits being likely to occur are divided into several attribute-bit set.
For example, the target device to be monitored includes equipment 0~9 totally 10, monitor control index includes index 0~2 totally 3, then
All attribute-bits being likely to occur share 10 × 3=30, be respectively as follows: device identification+index 0 of equipment 0 index mark,
The index of device identification+index 1 of equipment 0 identifies, the index of device identification+index 2 of equipment 0 identifies, the equipment mark of equipment 1
The index of knowledge+index 0 identifies, the index of device identification+index 1 of equipment 1 identifies, the finger of device identification+index 2 of equipment 1
The index of mark mark ... device identification+index 2 of equipment 9 identifies.This 30 attribute-bits being likely to occur can be drawn at random
It is divided into several attribute-bit set, the quantity for the attribute-bit for including in each attribute-bit set can be identical, can also be with
It is different.
After all attribute-bits being likely to occur are divided into several attribute-bit set in advance, data storage device is then
It can determine attribute-bit set belonging to the attribute-bit of the monitoring data carrying obtained from cluster monitoring system, it will be determining
The attribute-bit set is determined as specified attribute logo collection.
S203: by the supervising data storage in the corresponding database server of the specified attribute logo collection.
In the embodiment of the present application, due to subsequent by supervising data storage in distributed data base, to break through single machine
The I/O bottleneck of database, and including multiple database servers in distributed data base therefore can be according to above-mentioned divided
Good attribute-bit set presets each database server pass corresponding with attribute-bit set in distributed data base
System.Wherein, a database server can correspond to multiple attribute-bit set, but an attribute-bit set cannot correspond to it is multiple
Database server.
After having preset the corresponding attribute-bit set of each database server, then can according to by step S202 determine
Specified attribute logo collection (category belonging to the attribute-bit carried in the monitoring data that specified attribute logo collection as acquires
Property logo collection), monitoring data is stored using the corresponding database server of the specified attribute logo collection.
Specifically, the data storage device in the embodiment of the present application may be located in distributed data base, if data
Storage device is located in distributed data base, then after specified attribute logo collection being determined by step S202, data storage dress
The corresponding database server storage monitoring data of the specified attribute logo collection can be directlyed adopt by setting, if data storage device
In cluster monitoring system or independent equipment, then the monitoring data can be sent to this in distributed data base and specified
The corresponding database server storage of attribute-bit set.
As it can be seen that date storage method described herein is applied in distributed storage scene.By the above method,
For the different monitoring data of the corresponding database server of attribute-bit set belonging to the attribute-bit, these monitoring numbers
According to will be respectively stored in database server different in distributed data base, and due to the I/O of distributed data base
Ability is stronger compared to the I/O ability of single machine database, therefore, even if the data volume of monitoring data is larger, distributed data base
Monitoring data can also be timely received and stored, can effectively avoid database delay machine.
In the embodiment of the present application, the attribute-bit carried in the monitoring data, which is specifically as follows, compiles in collaboration with code, the conjunction
Coding includes the device coding of target device and the index coding of monitor control index.Specifically, the attribute mark carried in monitoring data
The device identification of target device in knowledge can be for the preset device coding of the target device, and the index of monitor control index identifies
It can be for the preset index coding of the monitor control index, then after data storage device acquires monitoring data, it may be determined that monitoring
What the device coding of the target device carried in data and the index coding of monitor control index were constituted compiles in collaboration with code, as the monitoring number
According to the attribute-bit of middle carrying.
For example, the device identification of a target device can be preset as the binary number of 4 bytes, as shown in table 1.
Table 1
In table 1, since 1 byte has 8, theoretically can be to 2 using device coding as shown in Table 132It is a
Target device is monitored.
The index mark of one monitor control index can also be preset as the binary number of 4 bytes, as shown in table 2.
Table 2
It is similar with table 1, using in index coding theory as shown in Table 2 can be directed to a target device 232A prison
Control index is monitored.
After the method encoded using above-mentioned default device coding and index, compiled in collaboration with by what device coding and index coding formed
Code then can be the binary number for 8 bytes being made of device coding and index coding, as shown in table 3.
Table 3
As can be seen from Table 3, carried in monitoring data compile in collaboration with code be one by device coding (4 byte binary number) and
Index encodes 8 byte binary numbers of (4 byte binary number) composition, and therefore, the conjunction being theoretically likely to occur is encoded to 264It is a.
Since the attribute-bit carried in monitoring data is to compile in collaboration with code using what the above method determined, in advance to institute
It is possible that the method that the attribute-bit occurred divides attribute-bit set accordingly can be with are as follows: in advance to it is all be likely to occur compile in collaboration with
Code, which divides, closes coding range.Wherein, division methods can specifically be divided using dichotomy, and the application is not construed as limiting this.
The method phase of each database server and the corresponding relationship of ATTRIBUTE INDEX set in default distributed data base
It should can be with are as follows: each database server in default distributed data base and the corresponding relationship for closing coding range.Wherein, one
Database server can correspond to multiple conjunction coding ranges, but a conjunction coding range can not correspond to multiple database servers.
Then the method for step S202 as shown in Figure 2 is specifically as follows: according to the target carried in the monitoring data of acquisition
The device coding of equipment and the index coding of monitor control index, determine and are compiled in collaboration with by what the device coding and index coding were constituted
Code, according to the conjunction coding range divided in advance, determines that this compiles in collaboration with the conjunction coding range that code is fallen into, and compiles in collaboration with a yard model as specified
It encloses.The method of step S203 as shown in Figure 2 is specifically as follows: being taken according to each database in preset distributed data base
Business device and the corresponding relationship for closing coding range, using the determining corresponding database server storage monitoring of specified conjunction coding range
Data.
Further, it is contemplated that device identification and index mark are preset in such a way that default device coding and index encode,
The range for compiling in collaboration with code being likely to occur may be excessively huge, if sequence encodes each target device set device, sequentially
To each monitor control index setting index coding, even when conjunction coding range has been divided, in all monitoring datas monitored
What is carried compiles in collaboration with code and can may also only fall in the conjunction coding range of some or certain several minority, in this way, these monitor numbers
According to that will be stored in the database server of corresponding some or certain several minority, cannot also reach largely to be supervised
Control data are shared in each database server, and the effect of I/O bottleneck is broken through.Therefore, it is being set in the embodiment of the present application
When standby coding and index coding, not using the method for sequence set device coding and index coding, and it is randomly generated each
The device coding of target device generates the index coding of each monitor control index at random, makes to be made of device coding and index coding
Compile in collaboration with code and can uniformly appear in each conjunction coding range of division.
Continue to use the example above and be illustrated, it is assumed that the target device to be monitored there are 5000, and each target device will monitor
10 achievement datas, then actual monitored to monitoring data there was only 50000 kinds.And the conjunction for using method as shown in table 3 to determine
Coding, the conjunction that theory is likely to occur are encoded to 264It is a that (the conjunction coding range being likely to occur is 0~264- 1), even if by these
It compiles in collaboration with code and is divided equally into 216A conjunction coding range, then each quantity for compiling in collaboration with code for including in coding range of closing is still 248It is a.
Obviously, 248Total amount 50000 of this quantity much larger than the type of actual monitoring data.If right in default device coding
Each target device is encoded according to the method set device of 0,1,2 ... this sequence sequence settings, in pre-set level coding,
To each monitor control index according to the method setting index coding of 0,1,2 ... this sequence sequence settings, then actual monitored obtains
50000 kinds of monitoring datas in carry this 50000 kinds compile in collaboration with code just and can all fall into the same conjunction coding range, the conjunction
Coding range is exactly 0~2 divided48- 1 this range.And since a conjunction coding range only corresponds to a database server,
Therefore, this 50000 kinds of monitoring datas monitored will be stored in the same database server.
In order to avoid the generation of above situation, in the embodiment of the present application in default device coding, set for a target
It is standby, generate device coding of the binary number as the target device of 4 bytes at random, for a monitor control index, also with
The binary number that machine generates 4 bytes is encoded as the index of the monitor control index, and guarantees that the same device coding only corresponds to
One target device, the same index coding only correspond to the same monitor control index, so that it may realize that 50000 kinds of monitoring datas carry
This 50000 kinds different codes of compiling in collaboration be to be evenly distributed in theoretically all conjunction coding ranges (0~2 being likely to occur64- 1)
In, rather than concentrate in the range of some very little.
For a database server preset corresponding conjunctions coding range when, also will at random by division some or
Certain several yard range of compiling in collaboration with is preset as the corresponding conjunction coding range of the database server.Wherein, it can be taken according to each database
The corresponding conjunction coding range of performance presetting database server of business device, if the performance of a database server is relatively
It is good, then the greater number of conjunction coding range of division random can be preset as that the database server is corresponding to compile in collaboration with code by this
Range then can be random by the small number of of division by this, whereas if the performance of a database server is relatively poor
It closes coding range and is preset as the corresponding conjunction coding range of the database server.
In this way, being averagely divided into 2 to theoretically all codes of compiling in collaboration with being likely to occur16A conjunction coding range be (each division
Close includes 2 in coding range48It is a to compile in collaboration with code) after, actual 50000 kinds different compile in collaboration with code will uniformly appear in this 216
In a different conjunction coding range, without being concentrated in 0~248- 1 this close coding range in, thus, it is subsequent then can be by this
50000 kinds of different monitoring datas are respectively stored in different database servers, are used in distributed data base with reaching
Multiple database servers share a large amount of monitoring data, break through the purpose of the I/O bottleneck of single machine database.
Further, it is contemplated that when the data volume of monitoring data is larger, relevant database stores monitoring data and needs
A large amount of storage resource is expended (this is because relevant database is the database that one kind indexes more, to need in storing data
Store multiple indexes and the corresponding relationships of data), therefore, in order to save storage resource, in the embodiment of the present application, for storing prison
The distributed data base for controlling data uses non-relational distributed data base, i.e., the distributed data base singly indexed.
Specifically, the non-relational database in the embodiment of the present application can be Hbase database.Due in the prior art
Using RRD database purchase monitoring data, and if time that RRD database stores certain monitoring datas it is longer (such as 3 months),
Then these monitoring datas can be merged by specified rule and (e.g., be identified to the identical device identification of carrying and index by RRD database
Monitoring data in monitoring value be averaged, merge into a monitoring data), will lead in this way merge after monitoring data lose
The details for having lost original monitoring data causes the accuracy decline of the monitoring data of storage.And Hbase database is one due to it
A distributed data base, theoretically the capacity of distributed data base can infinitely be expanded by increasing the quantity of database server
Hold, therefore, Hbase database would not also lose monitoring data without merging to the longer monitoring data of storage time
Details, can effectively improve storage monitoring data accuracy.
Certainly, non-relational distributed data base described in the embodiment of the present application is not only limited in Hbase database, only
If the non-relational distributed data base not merged to the longer data of storage time.
Furthermore, it is contemplated that the database that monitoring data is stored in the embodiment of the present application is non-relational distributed data base,
Single search index function can only be provided in subsequent query process, in step S203 as shown in Figure 2, data storage device
The method of the corresponding database server storage monitoring data of the specified attribute logo collection can in using distributed data base
With are as follows: the monitoring data is sent to this in distributed data base and referred to by the attribute-bit to carry in the monitoring data for index
Determine the corresponding database server storage of attribute-bit set.
Specifically, data storage device can adjust the data format of monitoring data are as follows: with the attribute mark wherein carried
Knowledge be to index, using the monitoring value wherein carried as the data format of detailed data, then will have adjusted the monitoring number after data format
It is stored according to corresponding database server is sent to.
It is subsequent, when data storage device receives the inquiry request of query monitor data, then it can determine inquiry request
The attribute-bit of the monitoring data to be checked of middle carrying, and be index with the attribute-bit of the monitoring data to be checked, to be checked
It askes in the corresponding database server of attribute-bit set belonging to the attribute-bit of monitoring data, inquires the monitoring number to be checked
According to.That is, device coding and index coding by including in index, user can easily inquire some prison of some target device
Control the monitoring data of index.
It certainly, can be to be carried in monitoring data other than the attribute-bit that carries in monitoring data is index
Attribute-bit and timestamp (temporal information that timestamp indicates the acquisition monitoring) are collectively as index, as shown in table 4.
Table 4
In the data format shown in table 4, index includes: the device coding and 4 byte int by 4 byte integers (int type)
The timestamp of attribute-bit and 8 byte longs (long type) that the index coding of type is constituted.Detailed data is that 16 bytes are double
The monitoring value of precision float (double type).
Monitoring data is adjusted using data format as shown in table 4, and monitoring data adjusted is sent to accordingly
After database server storage, if data storage device receives the inquiry request of query monitor data, it can determine that this is looked into
Ask the attribute-bit and time segment information of monitoring data to be checked carried in request, and from the attribute of the monitoring data to be checked
In the corresponding database server of attribute-bit set belonging to mark, query time stamp falls into the to be checked of the time segment information
Monitoring data.That is, some monitor control index of some target device according to index as shown in table 4, can be inquired in some period
Interior monitoring data.
It should be noted that since the non-relational distributed data base singly indexed is in index when inquiring data
The vertical sequence of field successively search (for example, shown in table 4 index in field be successively from front to back equipment compile
Code, index coding, timestamp), for the first character section in index, it can be reserved for the field and corresponding database server
Corresponding relationship then can be used dichotomy and continue to search for subsequent field in index, therefore, for first character in index
The inquiry velocity of section is very fast.
And the quantity of the target device due in practical applications, needing to monitor is much larger than the monitor control index for needing to monitor
Therefore, in the embodiment of the present application quantity improves the speed of subsequent query data, attribute mark that can be most by quantity in index
Know as the first character section in index.As shown in table 4, since the quantity 5000 for the target device for needing to monitor is much larger than monitoring
The quantity 10 of index, therefore, using the most device coding of quantity as the first character section in index in table 4, by negligible amounts
Index coding as index in second field.To, during subsequent query data, non-relational distribution number
It can quickly be navigated on certain database servers according to the first character section (device coding) in index according to library, further according to
Two fields (index coding), specifically navigate on a database server, finally according to time segment information, timestamp are fallen
The data to be checked entered in the period are extracted and are returned.
The above are date storage methods provided by the embodiments of the present application, are based on same thinking, and the embodiment of the present application also mentions
For a kind of data storage device, as shown in Figure 3.
Fig. 3 is data storage device structural schematic diagram provided by the embodiments of the present application, and described device is deposited applied to distribution
Scene is stored up, carries attribute-bit in monitoring data, divides each attribute-bit set in advance, and every in default distributed data base
The corresponding relationship of a database server and the attribute-bit set of division, described device specifically include:
It obtains module 301 and carries attribute-bit in the monitoring data for obtaining monitoring data；
Determining module 302, for determining attribute-bit belonging to the attribute-bit carried in the monitoring data obtained
Set, as specified attribute logo collection；
Memory module 303 is used for the supervising data storage in the corresponding database of the specified attribute logo collection
In server.
Wherein, the attribute-bit carried in the monitoring data is to compile in collaboration with code, and the code of compiling in collaboration with includes setting for target device
The index of standby coding and monitor control index encodes.
The memory module 303 is specifically used for, and is index with the attribute-bit carried in the monitoring data, by the prison
Control data are sent to the corresponding database server storage of specified attribute logo collection described in the distributed data base,
In, the distributed data base includes non-relational distributed data base.
Described device further include:
Enquiry module 304, for determining and being taken in the inquiry request when receiving the inquiry request of query monitor data
The attribute-bit of the monitoring data to be checked of band is index with the attribute-bit of the monitoring data to be checked, from described to be checked
It askes in the corresponding database server of attribute-bit set belonging to the attribute-bit of monitoring data, inquires the monitoring to be checked
Data.
Specifically, above-mentioned data storage device as shown in Figure 3 can be located in cluster monitoring system, distribution may be alternatively located at
In formula database, independent equipment can also be.
Fig. 4 is provided by the embodiments of the present application to be illustrated so that data storage device shown in Fig. 3 is independent equipment as an example
Data storage system architecture schematic diagram, the system include: cluster monitoring system, data storage device, distributed data base, inquiry
Client.Wherein:
Cluster monitoring system acquires monitoring data for being monitored to each target device；
Data storage device, for obtaining the collected monitoring data of cluster monitoring system, according to the monitoring of acquisition
The attribute-bit carried in data, and each attribute-bit set divided in advance are determined and are carried in the monitoring data
Attribute-bit set belonging to attribute-bit, as specified attribute logo collection, according in advance in distributed data base
The attribute-bit set of each database server setting, determines the corresponding database service of the specified attribute logo collection
The monitoring data is sent to the database server determined in the distributed data base and stored by device；
Distributed data base, the monitoring data sent for receiving and storing data storage device；
Inquiring client terminal, for sending inquiry request to data storage device；
Data storage device is also used to, when receiving the inquiry request for the query monitor data that inquiring client terminal is sent,
The attribute-bit of the monitoring data to be checked carried in the inquiry request is determined, with the attribute mark of the monitoring data to be checked
Know for index, the corresponding database server of attribute-bit set belonging to the attribute-bit from the monitoring data to be checked
In, inquire the monitoring data to be checked
The embodiment of the present application provides a kind of date storage method and device, and this method obtains monitoring data, and determination acquires
Monitoring data in attribute-bit set belonging to the attribute-bit that carries, as specified attribute logo collection, and according to preparatory
For the attribute-bit set of each database server setting in distributed data base, by the supervising data storage in institute
It states in the corresponding database server of specified attribute logo collection.By the above method, for attribute mark belonging to attribute-bit
Know and gather for different monitoring datas, these monitoring datas can be respectively stored in database clothes different in distributed data base
It is engaged in device, and since the I/O ability of distributed data base is stronger compared to the I/O ability of single machine database, even if monitoring
The data volume of data is larger, and distributed data base can also timely receive and store monitoring data, can effectively avoid database and delays
Machine.
In a typical configuration, calculating equipment includes one or more processors (CPU), input/output interface, net
Network interface and memory.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/or
The forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable medium
Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves
State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable
Programmable read only memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM),
Digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape magnetic disk storage or other magnetic storage devices
Or any other non-transmission medium, can be used for storage can be accessed by a computing device information.As defined in this article, it calculates
Machine readable medium does not include temporary computer readable media (transitory media), such as the data-signal and carrier wave of modulation.
It should also be noted that, the terms "include", "comprise" or its any other variant are intended to nonexcludability
It include so that the process, method, commodity or the equipment that include a series of elements not only include those elements, but also to wrap
Include other elements that are not explicitly listed, or further include for this process, method, commodity or equipment intrinsic want
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including described want
There is also other identical elements in the process, method of element, commodity or equipment.
It will be understood by those skilled in the art that embodiments herein can provide as method, system or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the application
Form.It is deposited moreover, the application can be used to can be used in the computer that one or more wherein includes computer usable program code
The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
The above description is only an example of the present application, is not intended to limit this application.For those skilled in the art
For, various changes and changes are possible in this application.All any modifications made within the spirit and principles of the present application are equal
Replacement, improvement etc., should be included within the scope of the claims of this application.
Claims (7)
1. a kind of date storage method, which is characterized in that the method is applied to the distribution for including multiple database servers
In formula database, each database server is corresponding with the attribute-bit set divided in advance, which comprises
The device coding of each target device is generated at random, generates the index coding of each monitor control index at random；
The monitoring data being monitored to target device is obtained, carries attribute-bit in the monitoring data；
The index of the device coding and monitor control index that determine the target device carried in the monitoring data encodes constituted conjunction
Coding, the attribute-bit carried as the monitoring data；
It determines attribute-bit set belonging to the attribute-bit carried in the monitoring data obtained, is identified as specified attribute
Set, comprising: according to the conjunction coding range divided in advance, the conjunction coding range that code is fallen into is compiled in collaboration with described in determination, as specified
Close coding range；
By the supervising data storage in the corresponding database server of the specified attribute logo collection, comprising: according to pre-
If distributed data base in each database server and close the corresponding relationship of coding range, specified compiled in collaboration with using determining
The corresponding database server of code range stores monitoring data.
2. the method as described in claim 1, which is characterized in that the device coding for setting each target device at random in advance, with
Machine sets the index coding of each monitor control index.
3. the method as described in claim 1, which is characterized in that by the supervising data storage in the specified attribute identification sets
It closes in corresponding database server, specifically includes:
The monitoring data is sent to the distributed data base for index by the attribute-bit to carry in the monitoring data
Described in the corresponding database server storage of specified attribute logo collection.
4. method as claimed in claim 3, which is characterized in that the method also includes:
When receiving the inquiry request of query monitor data, the monitoring data to be checked carried in the inquiry request is determined
Attribute-bit；
It is to index with the attribute-bit of the monitoring data to be checked, belonging to the attribute-bit from the monitoring data to be checked
In the corresponding database server of attribute-bit set, the monitoring data to be checked is inquired.
5. a kind of data storage device, which is characterized in that described device is applied to the distribution for including multiple database servers
In formula database, each database server is corresponding with the attribute-bit set divided in advance, and described device includes:
Module is obtained, for generating the device coding of each target device at random, the index for generating each monitor control index at random is compiled
Code obtains the monitoring data being monitored to target device, carries attribute-bit in the monitoring data；
The index of the device coding and monitor control index that determine the target device carried in the monitoring data encodes constituted conjunction
Coding, the attribute-bit carried as the monitoring data；
Determining module is made for determining attribute-bit set belonging to the attribute-bit carried in the monitoring data obtained
For specified attribute logo collection, comprising: according to the conjunction coding range divided in advance, determine described in compile in collaboration with that code fallen into compile in collaboration with code
Range, as specified conjunction coding range；
Memory module is used for the supervising data storage in the corresponding database server of the specified attribute logo collection
In, comprising: according to the corresponding relationship of each database server and conjunction coding range in preset distributed data base, use
The corresponding database server of determining specified conjunction coding range stores monitoring data.
6. device as claimed in claim 5, which is characterized in that the memory module is specifically used for, in the monitoring data
The attribute-bit of carrying is index, and the monitoring data is sent to specified attribute identification sets described in the distributed data base
Close corresponding database server storage.
7. device as claimed in claim 6, which is characterized in that described device further include:
Enquiry module, for when receiving the inquiry request of query monitor data, determine carried in the inquiry request to
The attribute-bit of query monitor data is index with the attribute-bit of the monitoring data to be checked, from the monitoring to be checked
In the corresponding database server of attribute-bit set belonging to the attribute-bit of data, the monitoring data to be checked is inquired.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410006688.4A CN104765749B (en) | 2014-01-07 | 2014-01-07 | A kind of date storage method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410006688.4A CN104765749B (en) | 2014-01-07 | 2014-01-07 | A kind of date storage method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104765749A CN104765749A (en) | 2015-07-08 |
| CN104765749B true CN104765749B (en) | 2019-04-16 |
Family
ID=53647582
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410006688.4A Active CN104765749B (en) | 2014-01-07 | 2014-01-07 | A kind of date storage method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104765749B (en) |
Families Citing this family (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106371333A (en) * | 2015-07-20 | 2017-02-01 | 阿里巴巴集团控股有限公司 | Controlling method, apparatus, and system for intelligent device |
| CN105490835B (en) * | 2015-11-20 | 2019-03-01 | 北京百度网讯科技有限公司 | Information monitoring method and device |
| CN106844375A (en) * | 2015-12-04 | 2017-06-13 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN107203915B (en) * | 2016-03-16 | 2020-07-21 | 阿里巴巴集团控股有限公司 | Data storage method and device |
| CN106454248A (en) * | 2016-10-25 | 2017-02-22 | 上海银晨智能识别科技有限公司 | Processing system and processing method of face monitoring data in wide range |
| CN106953755B (en) * | 2017-03-17 | 2020-06-09 | 中国科学院沈阳自动化研究所 | Energy monitoring method and system |
| CN107622079A (en) * | 2017-07-28 | 2018-01-23 | 阿里巴巴集团控股有限公司 | Data storage, querying method and device |
| CN108334549A (en) * | 2017-12-28 | 2018-07-27 | 海尔优家智能科技（北京）有限公司 | A kind of device data storage method, extracting method, storage platform and extraction platform |
| CN108304484A (en) * | 2017-12-29 | 2018-07-20 | 北京城市网邻信息技术有限公司 | Key word matching method and device, electronic equipment and readable storage medium storing program for executing |
| CN108829798B (en) * | 2018-06-05 | 2024-02-02 | 平安科技（深圳）有限公司 | Data storage method and system based on distributed database |
| CN109189812A (en) * | 2018-08-14 | 2019-01-11 | 中国平安人寿保险股份有限公司 | A kind of index querying method, computer readable storage medium and server |
| CN109388651B (en) * | 2018-09-19 | 2020-11-10 | 中国联合网络通信集团有限公司 | Data processing method and device |
| CN109710682A (en) * | 2018-12-31 | 2019-05-03 | 深圳市道通科技股份有限公司 | A kind of diagnostic data processing method, device, diagnostic device and server |
| CN111722973B (en) * | 2019-03-20 | 2023-05-23 | 顺丰科技有限公司 | Event timeout monitoring method, system and storage medium |
| CN111309548B (en) * | 2020-02-03 | 2024-04-12 | 中国银联股份有限公司 | Timeout monitoring method and device and computer readable storage medium |
| CN111787284A (en) * | 2020-07-16 | 2020-10-16 | 济南浪潮数据技术有限公司 | Data acquisition system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101346931A (en) * | 2005-12-22 | 2009-01-14 | 安讯士有限公司 | Monitoring system and method for connecting a monitoring device to a service server |
| CN101631237A (en) * | 2009-08-05 | 2010-01-20 | 青岛海信网络科技股份有限公司 | Video monitoring data storing and managing system |
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN102314480A (en) * | 2011-07-05 | 2012-01-11 | 万达信息股份有限公司 | Distributed data storage method specific to mass data |
| CN102333206A (en) * | 2011-08-15 | 2012-01-25 | 杭州华三通信技术有限公司 | Monitoring storage system and method for improving reliability of monitoring storage |
| CN103220167A (en) * | 2013-03-25 | 2013-07-24 | 中标软件有限公司 | Distributed monitoring system and data storing method thereof |
| CN103366573A (en) * | 2013-07-10 | 2013-10-23 | 中兴智能交通（无锡）有限公司 | Vehicle running information tracking method and system based on cloud computing |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102137145B (en) * | 2010-12-03 | 2015-03-18 | 华为技术有限公司 | Method, device and system for managing distributed contents |
-
2014
- 2014-01-07 CN CN201410006688.4A patent/CN104765749B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101346931A (en) * | 2005-12-22 | 2009-01-14 | 安讯士有限公司 | Monitoring system and method for connecting a monitoring device to a service server |
| CN101631237A (en) * | 2009-08-05 | 2010-01-20 | 青岛海信网络科技股份有限公司 | Video monitoring data storing and managing system |
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN102314480A (en) * | 2011-07-05 | 2012-01-11 | 万达信息股份有限公司 | Distributed data storage method specific to mass data |
| CN102333206A (en) * | 2011-08-15 | 2012-01-25 | 杭州华三通信技术有限公司 | Monitoring storage system and method for improving reliability of monitoring storage |
| CN103220167A (en) * | 2013-03-25 | 2013-07-24 | 中标软件有限公司 | Distributed monitoring system and data storing method thereof |
| CN103366573A (en) * | 2013-07-10 | 2013-10-23 | 中兴智能交通（无锡）有限公司 | Vehicle running information tracking method and system based on cloud computing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104765749A (en) | 2015-07-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104765749B (en) | A kind of date storage method and device | |
| TWI682652B (en) | Data processing method and device | |
| JP6716727B2 (en) | Streaming data distributed processing method and apparatus | |
| CN107395659A (en) | A kind of method and device of service handling and common recognition | |
| CN102857578B (en) | A kind of file uploading method of network hard disc, system and net dish client | |
| CN104702625A (en) | Method and device for scheduling access request in CDN (Content Delivery Network) | |
| CN104298541A (en) | Data distribution algorithm and data distribution device for cloud storage system | |
| JP7313382B2 (en) | Frequent Pattern Analysis of Distributed Systems | |
| CN105005624A (en) | Method and device for generating index document ID | |
| CN107203532A (en) | Construction method, the implementation method of search and the device of directory system | |
| CN110413845A (en) | Resource storage method and device based on Internet of Things operating system | |
| CN105320702A (en) | Analysis method and device for user behavior data and smart television | |
| CN108399175B (en) | Data storage and query method and device | |
| CN113835823A (en) | Resource scheduling method and device, electronic equipment and computer readable storage medium | |
| CN111935306A (en) | Node scheduling method and device | |
| CN108810089B (en) | Information pushing method and device and storage medium | |
| CN110784336A (en) | Multi-device intelligent timing delay scene setting method and system based on Internet of things | |
| CN104283966A (en) | Data distribution algorithm and device of cloud storage system | |
| CN104168174A (en) | Method and apparatus for information transmission | |
| CN108664322A (en) | Data processing method and system | |
| CN107391541A (en) | A kind of real time data merging method and device | |
| CN109981726A (en) | A kind of distribution method of memory node, server and system | |
| TWI694393B (en) | Data processing method and device | |
| CN110019783A (en) | Attribute term clustering method and device | |
| CN108810299B (en) | Information analysis method, medium and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |