CN104765749A - Data storage method and device - Google Patents
Data storage method and device Download PDFInfo
- Publication number
- CN104765749A CN104765749A CN201410006688.4A CN201410006688A CN104765749A CN 104765749 A CN104765749 A CN 104765749A CN 201410006688 A CN201410006688 A CN 201410006688A CN 104765749 A CN104765749 A CN 104765749A
- Authority
- CN
- China
- Prior art keywords
- attribute
- bit
- monitor data
- data
- index
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data storage method and device. The data storage method and device are used for solving the problem of the prior art that a database is incapable of timely storing monitoring data even occurs a down phenomenon when the monitoring data volume is large. The data storage method includes that confirming an attribute identity set of attribute identities in the monitoring data, using as an appointed attribute identity set, and storing the monitoring data in a database server corresponding to the appointed attribute identity set according to an attribute identity set preset for each database server in a distributed database. By means of said method, different monitoring data of the attribute identity set of the attribute identities can be respectively stored in different database servers of the distributed database, because the I/O ability of the distributed database is stronger than the I/O ability of a stand-alone database, even if the monitoring data volume is large, the distributed database can timely receive and store monitoring data, and the database down can be effectively avoided.
Description
Technical field
The application relates to field of computer technology, particularly relates to a kind of date storage method and device.
Background technology
At present, the cluster monitoring systems such as such as Ganglia are usually adopted to monitor each target device in cluster device, acquisition monitoring data, and by supervising data storage in a database.
In the prior art, general adopting RRD(Round Robin Database) database stores monitor data, and the memory system architecture of monitor data is as shown in Figure 1.
Fig. 1 is the memory system architecture schematic diagram of monitor data in prior art, in FIG, by the monitoring server (Fig. 1 is not shown) in Ganglia cluster monitoring system, the target device in cluster device is monitored, gather the monitor data of each target device, after each monitoring server in Ganglia cluster monitoring system collects monitor data, the monitor data obtained is sent to RRD database purchase.
But, when quantity more (more than 5000 target devices) of the target device in the cluster device monitored, the data volume of the monitor data that cluster monitoring system obtains is also very huge, and due to RRD database be the database of a unit, its I/O (Input/Output, I/O) limited in one's ability, therefore when the data volume of monitor data is larger, be limited by the I/O ability of RRD database, RRD database can not receive monitor data that cluster monitoring system sends timely and store, and even occur that RRD database is delayed the situation of machine.
Summary of the invention
The embodiment of the present application provides a kind of date storage method and device, and in order to solve in prior art when the data volume of monitor data is larger, database can not receive in time and store monitor data, even occurs the problem of the machine of delaying.
A kind of date storage method that the embodiment of the present application provides, described method is applied in the distributed data base including multiple database server, and each described database server is corresponding with the attribute-bit set divided in advance, and described method comprises:
Obtain monitor data, in described monitor data, carry attribute-bit;
Determine the attribute-bit set belonging to attribute-bit of carrying in the described monitor data obtained, as the set of specified attribute mark;
By described supervising data storage in the database server that the set of described specified attribute mark is corresponding.
A kind of data storage device that the embodiment of the present application provides, described application of installation is in the distributed data base including multiple database server, and each described database server is corresponding with the attribute-bit set divided in advance, and described device comprises:
Obtaining module, for obtaining monitor data, in described monitor data, carrying attribute-bit;
Determination module, for determining the attribute-bit set in the described monitor data that obtains belonging to the attribute-bit that carries, as the set of specified attribute mark;
Memory module, for gathering described supervising data storage in corresponding database server in described specified attribute mark.
The embodiment of the present application provides a kind of date storage method and device, the method obtains monitor data, determine the attribute-bit set belonging to attribute-bit of carrying in the monitor data acquired, as the set of specified attribute mark, and according in advance for the attribute-bit set of each database server setting in distributed data base, by described supervising data storage in the database server that the set of described specified attribute mark is corresponding.Pass through said method, for the monitor data that the attribute-bit set belonging to attribute-bit is different, these monitor datas can be stored in database servers different in distributed data base respectively, and due to the I/O ability of distributed data base stronger compared to the I/O ability of unit database, even if therefore the data volume of monitor data is larger, distributed data base also can receive timely and store monitor data, can effectively avoid database to delay machine.
Accompanying drawing explanation
Accompanying drawing described herein is used to provide further understanding of the present application, and form a application's part, the schematic description and description of the application, for explaining the application, does not form the improper restriction to the application.In the accompanying drawings:
Fig. 1 is the memory system architecture schematic diagram of monitor data in prior art;
The data storage procedure that Fig. 2 provides for the embodiment of the present application;
The data storage device structural representation that Fig. 3 provides for the embodiment of the present application;
What Fig. 4 provided for the embodiment of the present application is the independently data storage system architecture schematic diagram that illustrates of equipment for the data storage device shown in Fig. 3.
Embodiment
I/O ability due to distributed data base depends on the quantity of database server in distributed data base, the quantity of database server is more, the I/O ability of distributed data base is higher, there is not bottleneck problem in the I/O ability of distributed data base in theory, therefore, distributed data library storage monitor data is adopted in the embodiment of the present application, the I/O that a large amount of monitor data of storage will consume is shared in each database server in distributed data base, monitor data can be received and stores timely, also can effectively avoid database due to I/O scarce capacity machine of delaying.
For making the object of the application, technical scheme and advantage clearly, below in conjunction with the application's specific embodiment and corresponding accompanying drawing, technical scheme is clearly and completely described.Obviously, described embodiment is only some embodiments of the present application, instead of whole embodiments.Based on the embodiment in the application, those of ordinary skill in the art are not making the every other embodiment obtained under creative work prerequisite, all belong to the scope of the application's protection.
The data storage procedure that Fig. 2 provides for the embodiment of the present application, wherein this data storage procedure is applied in the distributed data base including multiple database server, each described database server is corresponding with the attribute-bit set divided in advance, specifically comprises the following steps:
S201: obtain monitor data, carry attribute-bit in described monitor data.
In the embodiment of the present application, the monitor data that collects of cluster monitoring system is obtained by data storage device.Wherein, cluster monitoring system includes but not limited to Ganglia cluster monitoring system, data storage device can be arranged in cluster monitoring system, also can be the equipment independent of cluster monitoring system, below for data storage device be one independently equipment be described.
S202: determine the attribute-bit set belonging to attribute-bit of carrying in the monitor data obtained, as the set of specified attribute mark.
In the embodiment of the present application, cluster monitoring system is monitored target device in the monitor data obtained and is carried attribute-bit, and this attribute-bit can be formed by the index identifier combination of the device identification of this target device and monitor control index.
Such as, suppose that cluster monitoring system will be monitored the internal memory service condition of certain target device, then the device identification of this target device can be the Internet protocol (Int of this target device
ern
et Pr
ot
ocol, IP) address or medium education (M
edi
aa
ccessc
ontr
ol, MAC) address, monitor control index is " internal memory use ", the index that predeterminable monitor control index " internal memory use " is corresponding is designated 001, then the internal memory service condition of cluster monitoring system to this target device is monitored, and the attribute-bit carried in the monitor data obtained is " the IP address of this target device or MAC Address+001 ".
Further, in advance according to the attribute-bit likely occurred in monitor data, the attribute-bit likely occurred can be divided into several attribute-bit set in the embodiment of the present application.
Such as, the target device monitored comprises equipment 0 ~ 9 totally 10, monitor control index comprises index 0 ~ 2 totally 3, the attribute-bit then likely occurred has 10 × 3=30, be respectively: the index mark of the device identification+index 0 of equipment 0, the index mark of the device identification+index 1 of equipment 0, the index mark of the device identification+index 2 of equipment 0, the index mark of the device identification+index 0 of equipment 1, the index mark of the device identification+index 1 of equipment 1, the index mark of the device identification+index 2 of equipment 1 ... the index mark of the device identification+index 2 of equipment 9.At random these 30 attribute-bits that may occur can be divided into several attribute-bit set, the quantity of the attribute-bit comprised in each attribute-bit set can be identical, also can be different.
After in advance the attribute-bit likely occurred being divided into several attribute-bit set, data storage device then can determine the attribute-bit set belonging to the attribute-bit that the monitor data obtained from cluster monitoring system carries, and this attribute-bit set determined is defined as the set of specified attribute mark.
S203: by described supervising data storage in the database server that the set of described specified attribute mark is corresponding.
In the embodiment of the present application, due to follow-up will by supervising data storage in distributed data base, to break through the I/O bottleneck of unit database, and distributed data base comprises multiple database server, therefore, can according to above-mentioned ready-portioned attribute-bit set, each database server in default distributed data base and the corresponding relation of attribute-bit set.Wherein, a database server may correspond to multiple attribute-bit set, but an attribute-bit set can not corresponding multiple database server.
After having preset attribute-bit set corresponding to each database server, then can gather according to the specified attribute mark determined by step S202 (set of specified attribute mark is the attribute-bit set belonging to attribute-bit of carrying in the monitor data acquired), the database server adopting this specified attribute to identify set correspondence stores monitor data.
Concrete, data storage device in the embodiment of the present application can also be arranged in distributed data base, if data storage device is arranged in distributed data base, after then determining the set of specified attribute mark by step S202, data storage device directly can adopt this specified attribute to identify database server corresponding to set and store monitor data, if data storage device is arranged in cluster monitoring system, or independently equipment, then this monitor data can be sent to the database server that in distributed data base, the set of this specified attribute mark is corresponding to store.
Visible, the date storage method described in the application is applied in distributed storage scene.Pass through said method, for the monitor data that the database server that the attribute-bit set belonging to attribute-bit is corresponding is different, these monitor datas will be respectively stored in database servers different in distributed data base, and due to the I/O ability of distributed data base stronger compared to the I/O ability of unit database, therefore, even if the data volume of monitor data is comparatively large, distributed data base also can receive timely and store monitor data, can effectively avoid database to delay machine.
In the embodiment of the present application, the attribute-bit carried in described monitor data is specifically as follows compiles in collaboration with code, described in compile in collaboration with the index coding that code comprises the device coding of target device and monitor control index.Particularly, the device identification of the target device in the attribute-bit carried in monitor data can be the device coding preset for this target device, the index mark of monitor control index can be the index coding preset for this monitor control index, after then data storage device acquires monitor data, that can determine that the index coding of the device coding of the target device carried in monitor data and monitor control index forms compiles in collaboration with code, as the attribute-bit carried in this monitor data.
Such as, the device identification of a target device can be preset as the binary number of 4 bytes, as shown in table 1.
Table 1
In Table 1, because 1 byte has 8, therefore, adopt device coding as shown in table 1 in theory can to 2
32individual target device is monitored.
The index mark of a monitor control index also can be preset as the binary number of 4 bytes, as shown in table 2.
Table 2
Similar with table 1, adopting can for 2 of a target device in index coding theory as shown in table 2
32individual monitor control index is monitored.
After adopting the method for above-mentioned default device coding and index coding, that is encoded formed by device coding and index compiles in collaboration with code and then can be encoded by device coding and index the binary number of 8 bytes formed, as shown in table 3.
Table 3
As can be seen from Table 3, the code of compiling in collaboration with carried in monitor data is the 8 byte binary numbers be made up of device coding (4 byte binary number) and index coding (4 byte binary number), and therefore, the conjunction that may occur in theory is encoded to 2
64individual.
Due to the attribute-bit carried in monitor data be adopt said method determine compile in collaboration with code, therefore, dividing that the method for attribute-bit set is corresponding to the attribute-bit that likely occurs in advance can be for: divide close coding range to the code of compiling in collaboration with likely occurred in advance.Wherein, division methods specifically can adopt dichotomy to divide, and the application is not construed as limiting this.
Each database server in default distributed data base is corresponding to the method for the corresponding relation of ATTRIBUTE INDEX set can be: preset each database server in distributed data base and the corresponding relation closing coding range.Wherein, a database server may correspond to multiple conjunction coding range, but a conjunction coding range not may correspond to multiple database server.
Then the method for step S202 is as shown in Figure 2 specifically as follows: according to the device coding of the target device carried in the monitor data obtained and the index coding of monitor control index, determine by this device coding and index encode form compile in collaboration with code, according to the conjunction coding range divided in advance, determine this conjunction coding range of compiling in collaboration with code and falling into, close coding range as appointment.The method of step S203 is as shown in Figure 2 specifically as follows: according to each database server in the distributed data base preset and the corresponding relation closing coding range, adopts the appointment determined to close database server corresponding to coding range and stores monitor data.
Further, consider that the mode of encoding with default device coding and index presets device identification and index mark, the scope of compiling in collaboration with code that may occur may be too huge, if encoding to each target device set device of order, order each monitor control index setting index is encoded, even if so divided conjunction coding range, that carries in all monitor datas monitored compiles in collaboration with code and also may only fall within the conjunction coding range of some or certain several minority, like this, these monitor datas will be stored in the database server of some or certain several minority accordingly, also just can not reach shares in each database server by a large amount of monitor datas, break through the effect of I/O bottleneck.Therefore, in the embodiment of the present application when default device coding and index coding, do not adopt the method for order set device coding and index coding, but the device coding of each target device of stochastic generation, the index coding of each monitor control index of stochastic generation, what make to be encoded formed by device coding and index compiles in collaboration with code and can appear at uniformly in each conjunction coding range of division.
Continue along being described by example, suppose that the target device that will monitor has 5000, each target device will monitor 10 achievement datas, then actual monitored to monitor data only have 50000 kinds.And adopt method as shown in table 3 to determine compile in collaboration with code, the conjunction that its theory may occur is encoded to 2
64it is individual that (the conjunction coding range that may occur is 0 ~ 2
64-1), even if these are compiled in collaboration with code be on average divided into 2
16individual conjunction coding range, then the quantity of compiling in collaboration with code comprised in each conjunction coding range is still 2
48individual.Obviously, 2
48this quantity is much larger than the total amount 50000 of the kind of the monitor data of reality.If when default device coding, to each target device according to 0,1,2 ... the method set device coding of this order sequence setting, when pre-set level is encoded, to each monitor control index according to 0,1,2 ... the method setting index coding of this order sequence setting, compile in collaboration with code for these 50000 kinds that carry in 50000 kinds of monitor datas that then actual monitored obtains and just all can fall into same conjunction coding range, this conjunction coding range is exactly divide 0 ~ 2
48-1 this scope.And due to an a conjunction coding range only corresponding database server, therefore, these the 50000 kinds of monitor datas monitored will be stored in same database server.
In order to avoid the generation of above-mentioned situation, in the embodiment of the present application when default device coding, for a target device, the binary number of stochastic generation 4 bytes is as the device coding of this target device, for a monitor control index, also the binary number of stochastic generation 4 bytes is encoded as the index of this monitor control index, and ensure a same device coding only corresponding target device, same index coding only corresponding same monitor control index, these 50000 kinds that just can realize that 50000 kinds of monitor datas carry different codes of compiling in collaboration with are evenly distributed in the conjunction coding range (0 ~ 2 likely occurred in theory
64-1), instead of concentrate in certain very little scope.
When presetting corresponding conjunction coding range for a database server, also want random and certain or certain the several yard scope of compiling in collaboration with divided is preset as conjunction coding range corresponding to this database server.Wherein, conjunction coding range that can be corresponding according to the performance presetting database server of each database server, if the performance of a database server is relatively better, then the conjunction coding range of a greater number of division random can be preset as conjunction coding range corresponding to this database server by this, otherwise, if the performance of a database server is relatively poor, then the conjunction coding range of the lesser amt of division random can be preset as conjunction coding range corresponding to this database server by this.
Like this, on average 2 are divided into the code of compiling in collaboration with likely occurred in theory
16individual conjunction coding range (comprises 2 in the conjunction coding range of each division
48individually compile in collaboration with code) after, actual 50000 kinds different compile in collaboration with code will appear at uniformly this 2
16in individual different conjunction coding range, and be not concentrated in 0 ~ 2
48in-1 this conjunction coding range, thus, follow-up, can these 50000 kinds different monitor datas be stored in different database servers respectively, adopt the multiple database servers in distributed data base to share a large amount of monitor datas to reach, break through the object of the I/O bottleneck of unit database.
Further, consider when the data volume of monitor data is larger, relational data library storage monitor data needs the storage resources of at substantial, and (this is because relevant database is a kind of database of many indexes, the corresponding relation storing multiple index and data is needed) when storing data, therefore, in order to save storage resources, in the embodiment of the present application, distributed data base for storing monitor data adopts non-relational distributed data base, i.e. the distributed data base of single index.
Concrete, the non-relational database in the embodiment of the present application can be Hbase database.Owing to adopting RRD database purchase monitor data in prior art, if and RRD database stores the time longer (as 3 months) of some monitor data, then RRD database can by these monitor datas by specified rule carry out merging (as, the monitoring value of carrying in the monitor data of identical device mark and index mark is averaged, merge into a monitor data), the monitor data after merging can be caused like this to lost the details of original monitor data, cause the accuracy of the monitor data of storage to decline.And Hbase database is a distributed data base due to it, the capacity of distributed data base can by increasing the quantity of database server and unlimited dilatation in theory, therefore, Hbase database merges without the need to the monitor data longer to storage time, also would not lose the details of monitor data, effectively can improve the accuracy storing monitor data.
Certainly, the non-relational distributed data base described in the embodiment of the present application has more than and is limited to Hbase database, as long as not to the non-relational distributed data base that the data that storage time is longer merge.
In addition, consider in the embodiment of the present application the relational distributed data base of database right and wrong storing monitor data, single search index function can only be provided in follow-up query script, in step S203 as shown in Figure 2, the method that data storage device is adopting the database server of this specified attribute mark set correspondence in distributed data base to store monitor data can be: the attribute-bit carried in this monitor data is index, this monitor data is sent to the database server storage that in distributed data base, the set of this specified attribute mark is corresponding.
Concrete, the data layout of monitor data can be adjusted to by data storage device: with the attribute-bit wherein carried be index, the data layout that is detailed data with the monitoring value of wherein carrying, then send to corresponding database server to store by have adjusted the monitor data after data layout.
Follow-up, when data storage device receives the inquiry request of query monitor data, then can determine the attribute-bit of the monitor data to be checked carried in inquiry request, and with the attribute-bit of this monitor data to be checked for index, from the database server that the attribute-bit set belonging to the attribute-bit of monitor data to be checked is corresponding, inquire about this monitor data to be checked.That is, the device coding comprised by index and index coding, user can inquire about the monitor data of certain monitor control index of certain target device easily.
Certainly, except with the attribute-bit carried in monitor data for except index, the attribute-bit that can also carry in monitor data and timestamp (timestamp represents the temporal information of this this monitoring of acquisition) common as index, as shown in table 4.
Table 4
In data layout shown in table 4, index comprises: the timestamp of the attribute-bit that forms and the 8 byte long integers (long type) of being encoded by the device coding of 4 byte integers (int type) and the index of 4 byte int types.Detailed data is the monitoring value of 16 byte double-precision floating point types (double type).
Adopt data layout adjustment monitor data as shown in table 4, and after sending to corresponding database server to store the monitor data after adjustment, if data storage device receives the inquiry request of query monitor data, then can determine attribute-bit and the time period information of the monitor data to be checked carried in this inquiry request, and from database server corresponding to the attribute-bit set belonging to the attribute-bit of this monitor data to be checked, query time stamp falls into the monitor data to be checked of this time period information.That is, according to index as shown in table 4, the monitor data of certain monitor control index within certain time period of certain target device can be inquired about.
It should be noted that, non-relational distributed data base due to single index when data query be with the vertical order of the field in index search successively (such as, field in index shown in table 4 is device coding, index coding, timestamp from front to back successively), for the first character section in index, the corresponding relation of this field and corresponding database server can be preserved, for field follow-up in index, dichotomy then can be adopted to continue to search, therefore, be quickly for the inquiry velocity of first character section in index.
And due in actual applications, need the quantity of the target device of monitoring much larger than the quantity needing the monitor control index monitored, therefore, in order to improve the speed of subsequent query data in the embodiment of the present application, can using attribute-bits maximum for quantity in index as the first character section in index.As shown in table 4, owing to needing the quantity 10 of quantity 5000 much larger than monitor control index of the target device of monitoring, therefore, in table 4 using device codings maximum for quantity as the first character section in index, using the index of negligible amounts coding as the field of second in index.Thus, in the process of subsequent query data, non-relational distributed data base can according to the first character section (device coding) in index, quick position is on some database server, again according to second field (index coding), specifically navigate on a database server, finally according to time period information, the data to be checked fallen into by timestamp in this time period extract and return.
Be the date storage method that the embodiment of the present application provides above, based on same thinking, the embodiment of the present application also provides a kind of data storage device, as shown in Figure 3.
The data storage device structural representation that Fig. 3 provides for the embodiment of the present application, described application of installation is in distributed storage scene, attribute-bit is carried in monitor data, divide each attribute-bit set in advance, and the corresponding relation of the attribute-bit set of each database server preset in distributed data base and division, described device specifically comprises:
Obtaining module 301, for obtaining monitor data, in described monitor data, carrying attribute-bit;
Determination module 302, for determining the attribute-bit set in the described monitor data that obtains belonging to the attribute-bit that carries, as the set of specified attribute mark;
Memory module 303, for gathering described supervising data storage in corresponding database server in described specified attribute mark.
Wherein, the attribute-bit carried in described monitor data for compiling in collaboration with code, described in compile in collaboration with the index coding that code comprises the device coding of target device and monitor control index.
Described memory module 303 specifically for, with the attribute-bit carried in described monitor data for index, sent to by described monitor data specified attribute described in described distributed data base to identify database server corresponding to set to store, wherein, described distributed data base comprises non-relational distributed data base.
Described device also comprises:
Enquiry module 304, for when receiving the inquiry request of query monitor data, determine the attribute-bit of the monitor data to be checked carried in described inquiry request, with the attribute-bit of described monitor data to be checked for index, from the database server that the attribute-bit set belonging to the attribute-bit of described monitor data to be checked is corresponding, inquire about described monitor data to be checked.
Concrete, above-mentioned data storage device as shown in Figure 3 can be arranged in cluster monitoring system, also can be arranged in distributed data base, can also be independently equipment.
What Fig. 4 provided for the embodiment of the present application is the independently data storage system architecture schematic diagram that illustrates of equipment for the data storage device shown in Fig. 3, and this system comprises: cluster monitoring system, data storage device, distributed data base, inquiring client terminal.Wherein:
Cluster monitoring system is used for monitoring each target device, acquisition monitoring data;
Data storage device, for obtaining the monitor data that cluster monitoring system collects, according to the attribute-bit carried in the described monitor data obtained, and each attribute-bit set divided in advance, determine the attribute-bit set belonging to attribute-bit of carrying in described monitor data, as the set of specified attribute mark, according in advance for the attribute-bit set of each database server setting in distributed data base, determine the database server that the set of described specified attribute mark is corresponding, described monitor data is sent to established data storehouse server stores in described distributed data base,
Distributed data base, for receiving and storing the monitor data that data storage device sends;
Inquiring client terminal, for sending inquiry request to data storage device;
Data storage device also for, when receiving the inquiry request of the query monitor data that inquiring client terminal is sent, determine the attribute-bit of the monitor data to be checked carried in described inquiry request, with the attribute-bit of described monitor data to be checked for index, from the database server that the attribute-bit set belonging to the attribute-bit of described monitor data to be checked is corresponding, inquire about described monitor data to be checked
The embodiment of the present application provides a kind of date storage method and device, the method obtains monitor data, determine the attribute-bit set belonging to attribute-bit of carrying in the monitor data acquired, as the set of specified attribute mark, and according in advance for the attribute-bit set of each database server setting in distributed data base, by described supervising data storage in the database server that the set of described specified attribute mark is corresponding.Pass through said method, for the monitor data that the attribute-bit set belonging to attribute-bit is different, these monitor datas can be stored in database servers different in distributed data base respectively, and due to the I/O ability of distributed data base stronger compared to the I/O ability of unit database, even if therefore the data volume of monitor data is larger, distributed data base also can receive timely and store monitor data, can effectively avoid database to delay machine.
In one typically configuration, computing equipment comprises one or more processor (CPU), input/output interface, network interface and internal memory.
Internal memory may comprise the volatile memory in computer-readable medium, and the forms such as random access memory (RAM) and/or Nonvolatile memory, as ROM (read-only memory) (ROM) or flash memory (flash RAM).Internal memory is the example of computer-readable medium.
Computer-readable medium comprises permanent and impermanency, removable and non-removable media can be stored to realize information by any method or technology.Information can be computer-readable instruction, data structure, the module of program or other data.The example of the storage medium of computing machine comprises, but be not limited to phase transition internal memory (PRAM), static RAM (SRAM), dynamic RAM (DRAM), the random access memory (RAM) of other types, ROM (read-only memory) (ROM), Electrically Erasable Read Only Memory (EEPROM), fast flash memory bank or other memory techniques, read-only optical disc ROM (read-only memory) (CD-ROM), digital versatile disc (DVD) or other optical memory, magnetic magnetic tape cassette, tape magnetic rigid disk stores or other magnetic storage apparatus or any other non-transmitting medium, can be used for storing the information can accessed by computing equipment.According to defining herein, computer-readable medium does not comprise temporary computer readable media (transitory media), as data-signal and the carrier wave of modulation.
Also it should be noted that, term " comprises ", " comprising " or its any other variant are intended to contain comprising of nonexcludability, thus make to comprise the process of a series of key element, method, commodity or equipment and not only comprise those key elements, but also comprise other key elements clearly do not listed, or also comprise by the intrinsic key element of this process, method, commodity or equipment.When not more restrictions, the key element limited by statement " comprising ... ", and be not precluded within process, method, commodity or the equipment comprising described key element and also there is other identical element.
It will be understood by those skilled in the art that the embodiment of the application can be provided as method, system or computer program.Therefore, the application can adopt the form of complete hardware embodiment, completely software implementation or the embodiment in conjunction with software and hardware aspect.And the application can adopt in one or more form wherein including the upper computer program implemented of computer-usable storage medium (including but not limited to magnetic disk memory, CD-ROM, optical memory etc.) of computer usable program code.
The foregoing is only the embodiment of the application, be not limited to the application.To those skilled in the art, the application can have various modifications and variations.Any amendment done within all spirit in the application and principle, equivalent replacement, improvement etc., within the right that all should be included in the application.
Claims (9)
1. a date storage method, is characterized in that, described method is applied in the distributed data base including multiple database server, and each described database server is corresponding with the attribute-bit set divided in advance, and described method comprises:
Obtain monitor data, in described monitor data, carry attribute-bit;
Determine the attribute-bit set belonging to attribute-bit of carrying in the described monitor data obtained, as the set of specified attribute mark;
By described supervising data storage in the database server that the set of described specified attribute mark is corresponding.
2. the method for claim 1, is characterized in that, the attribute-bit carried in described monitor data for compiling in collaboration with code, described in compile in collaboration with the index coding that code comprises the device coding of target device and monitor control index.
3. method as claimed in claim 2, is characterized in that, in advance the device coding of each target device of random setting, the index coding of each monitor control index of random setting.
4. the method for claim 1, is characterized in that, by described supervising data storage in the database server that the set of described specified attribute mark is corresponding, specifically comprises:
With the attribute-bit carried in described monitor data for index, sent to by described monitor data specified attribute described in described distributed data base to identify database server corresponding to set and store.
5. method as claimed in claim 4, it is characterized in that, described method also comprises:
When receiving the inquiry request of query monitor data, determine the attribute-bit of the monitor data to be checked carried in described inquiry request;
With the attribute-bit of described monitor data to be checked for index, from the database server that the attribute-bit set belonging to the attribute-bit of described monitor data to be checked is corresponding, inquire about described monitor data to be checked.
6. a data storage device, is characterized in that, described application of installation is in the distributed data base including multiple database server, and each described database server is corresponding with the attribute-bit set divided in advance, and described device comprises:
Obtaining module, for obtaining monitor data, in described monitor data, carrying attribute-bit;
Determination module, for determining the attribute-bit set in the described monitor data that obtains belonging to the attribute-bit that carries, as the set of specified attribute mark;
Memory module, for gathering described supervising data storage in corresponding database server in described specified attribute mark.
7. device as claimed in claim 6, is characterized in that, the attribute-bit carried in described monitor data for compiling in collaboration with code, described in compile in collaboration with code and comprise the device coding of target device and the index coding of monitor control index.
8. device as claimed in claim 6, it is characterized in that, described memory module specifically for, with the attribute-bit carried in described monitor data for index, sent to by described monitor data specified attribute described in described distributed data base to identify database server corresponding to set and store.
9. device as claimed in claim 8, it is characterized in that, described device also comprises:
Enquiry module, for when receiving the inquiry request of query monitor data, determine the attribute-bit of the monitor data to be checked carried in described inquiry request, with the attribute-bit of described monitor data to be checked for index, from the database server that the attribute-bit set belonging to the attribute-bit of described monitor data to be checked is corresponding, inquire about described monitor data to be checked.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410006688.4A CN104765749B (en) | 2014-01-07 | 2014-01-07 | A kind of date storage method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410006688.4A CN104765749B (en) | 2014-01-07 | 2014-01-07 | A kind of date storage method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104765749A true CN104765749A (en) | 2015-07-08 |
| CN104765749B CN104765749B (en) | 2019-04-16 |
Family
ID=53647582
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410006688.4A Active CN104765749B (en) | 2014-01-07 | 2014-01-07 | A kind of date storage method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104765749B (en) |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105490835A (en) * | 2015-11-20 | 2016-04-13 | 北京百度网讯科技有限公司 | Information monitoring method and device |
| CN106371333A (en) * | 2015-07-20 | 2017-02-01 | 阿里巴巴集团控股有限公司 | Controlling method, apparatus, and system for intelligent device |
| CN106454248A (en) * | 2016-10-25 | 2017-02-22 | 上海银晨智能识别科技有限公司 | Processing system and processing method of face monitoring data in wide range |
| CN106844375A (en) * | 2015-12-04 | 2017-06-13 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN106953755A (en) * | 2017-03-17 | 2017-07-14 | 中国科学院沈阳自动化研究所 | A kind of energy monitoring method and system |
| CN107203915A (en) * | 2016-03-16 | 2017-09-26 | 阿里巴巴集团控股有限公司 | Date storage method and device |
| CN107622079A (en) * | 2017-07-28 | 2018-01-23 | 阿里巴巴集团控股有限公司 | Data storage, querying method and device |
| CN108304484A (en) * | 2017-12-29 | 2018-07-20 | 北京城市网邻信息技术有限公司 | Key word matching method and device, electronic equipment and readable storage medium storing program for executing |
| CN108334549A (en) * | 2017-12-28 | 2018-07-27 | 海尔优家智能科技（北京）有限公司 | A kind of device data storage method, extracting method, storage platform and extraction platform |
| CN109189812A (en) * | 2018-08-14 | 2019-01-11 | 中国平安人寿保险股份有限公司 | A kind of index querying method, computer readable storage medium and server |
| CN109388651A (en) * | 2018-09-19 | 2019-02-26 | 中国联合网络通信集团有限公司 | A kind of data processing method and device |
| CN109710682A (en) * | 2018-12-31 | 2019-05-03 | 深圳市道通科技股份有限公司 | A kind of diagnostic data processing method, device, diagnostic device and server |
| WO2019232933A1 (en) * | 2018-06-05 | 2019-12-12 | 平安科技（深圳）有限公司 | Data storage method and system employing distributed database |
| CN111309548A (en) * | 2020-02-03 | 2020-06-19 | 中国银联股份有限公司 | Timeout monitoring method and device and computer readable storage medium |
| CN111722973A (en) * | 2019-03-20 | 2020-09-29 | 顺丰科技有限公司 | Event timeout monitoring method, system and storage medium |
| CN111787284A (en) * | 2020-07-16 | 2020-10-16 | 济南浪潮数据技术有限公司 | Data acquisition system |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101346931A (en) * | 2005-12-22 | 2009-01-14 | 安讯士有限公司 | Monitoring system and method for connecting a monitoring device to a service server |
| CN101631237A (en) * | 2009-08-05 | 2010-01-20 | 青岛海信网络科技股份有限公司 | Video monitoring data storing and managing system |
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN102137145A (en) * | 2010-12-03 | 2011-07-27 | 华为技术有限公司 | Method, device and system for managing distributed contents |
| CN102314480A (en) * | 2011-07-05 | 2012-01-11 | 万达信息股份有限公司 | Distributed data storage method specific to mass data |
| CN102333206A (en) * | 2011-08-15 | 2012-01-25 | 杭州华三通信技术有限公司 | Monitoring storage system and method for improving reliability of monitoring storage |
| CN103220167A (en) * | 2013-03-25 | 2013-07-24 | 中标软件有限公司 | Distributed monitoring system and data storing method thereof |
| CN103366573A (en) * | 2013-07-10 | 2013-10-23 | 中兴智能交通（无锡）有限公司 | Vehicle running information tracking method and system based on cloud computing |
-
2014
- 2014-01-07 CN CN201410006688.4A patent/CN104765749B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101346931A (en) * | 2005-12-22 | 2009-01-14 | 安讯士有限公司 | Monitoring system and method for connecting a monitoring device to a service server |
| CN101631237A (en) * | 2009-08-05 | 2010-01-20 | 青岛海信网络科技股份有限公司 | Video monitoring data storing and managing system |
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN102137145A (en) * | 2010-12-03 | 2011-07-27 | 华为技术有限公司 | Method, device and system for managing distributed contents |
| CN102314480A (en) * | 2011-07-05 | 2012-01-11 | 万达信息股份有限公司 | Distributed data storage method specific to mass data |
| CN102333206A (en) * | 2011-08-15 | 2012-01-25 | 杭州华三通信技术有限公司 | Monitoring storage system and method for improving reliability of monitoring storage |
| CN103220167A (en) * | 2013-03-25 | 2013-07-24 | 中标软件有限公司 | Distributed monitoring system and data storing method thereof |
| CN103366573A (en) * | 2013-07-10 | 2013-10-23 | 中兴智能交通（无锡）有限公司 | Vehicle running information tracking method and system based on cloud computing |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106371333A (en) * | 2015-07-20 | 2017-02-01 | 阿里巴巴集团控股有限公司 | Controlling method, apparatus, and system for intelligent device |
| CN105490835A (en) * | 2015-11-20 | 2016-04-13 | 北京百度网讯科技有限公司 | Information monitoring method and device |
| CN105490835B (en) * | 2015-11-20 | 2019-03-01 | 北京百度网讯科技有限公司 | Information monitoring method and device |
| CN106844375A (en) * | 2015-12-04 | 2017-06-13 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN107203915A (en) * | 2016-03-16 | 2017-09-26 | 阿里巴巴集团控股有限公司 | Date storage method and device |
| CN107203915B (en) * | 2016-03-16 | 2020-07-21 | 阿里巴巴集团控股有限公司 | Data storage method and device |
| CN106454248A (en) * | 2016-10-25 | 2017-02-22 | 上海银晨智能识别科技有限公司 | Processing system and processing method of face monitoring data in wide range |
| CN106953755A (en) * | 2017-03-17 | 2017-07-14 | 中国科学院沈阳自动化研究所 | A kind of energy monitoring method and system |
| CN106953755B (en) * | 2017-03-17 | 2020-06-09 | 中国科学院沈阳自动化研究所 | Energy monitoring method and system |
| CN107622079A (en) * | 2017-07-28 | 2018-01-23 | 阿里巴巴集团控股有限公司 | Data storage, querying method and device |
| CN108334549A (en) * | 2017-12-28 | 2018-07-27 | 海尔优家智能科技（北京）有限公司 | A kind of device data storage method, extracting method, storage platform and extraction platform |
| CN108304484A (en) * | 2017-12-29 | 2018-07-20 | 北京城市网邻信息技术有限公司 | Key word matching method and device, electronic equipment and readable storage medium storing program for executing |
| WO2019232933A1 (en) * | 2018-06-05 | 2019-12-12 | 平安科技（深圳）有限公司 | Data storage method and system employing distributed database |
| CN109189812A (en) * | 2018-08-14 | 2019-01-11 | 中国平安人寿保险股份有限公司 | A kind of index querying method, computer readable storage medium and server |
| CN109388651A (en) * | 2018-09-19 | 2019-02-26 | 中国联合网络通信集团有限公司 | A kind of data processing method and device |
| CN109388651B (en) * | 2018-09-19 | 2020-11-10 | 中国联合网络通信集团有限公司 | Data processing method and device |
| CN109710682A (en) * | 2018-12-31 | 2019-05-03 | 深圳市道通科技股份有限公司 | A kind of diagnostic data processing method, device, diagnostic device and server |
| CN111722973A (en) * | 2019-03-20 | 2020-09-29 | 顺丰科技有限公司 | Event timeout monitoring method, system and storage medium |
| CN111722973B (en) * | 2019-03-20 | 2023-05-23 | 顺丰科技有限公司 | Event timeout monitoring method, system and storage medium |
| CN111309548A (en) * | 2020-02-03 | 2020-06-19 | 中国银联股份有限公司 | Timeout monitoring method and device and computer readable storage medium |
| CN111309548B (en) * | 2020-02-03 | 2024-04-12 | 中国银联股份有限公司 | Timeout monitoring method and device and computer readable storage medium |
| CN111787284A (en) * | 2020-07-16 | 2020-10-16 | 济南浪潮数据技术有限公司 | Data acquisition system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104765749B (en) | 2019-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104765749A (en) | Data storage method and device | |
| JP6716727B2 (en) | Streaming data distributed processing method and apparatus | |
| CN102968498A (en) | Method and device for processing data | |
| CN107515878B (en) | Data index management method and device | |
| CN104572727A (en) | Data querying method and device | |
| CN103067525A (en) | Cloud storage data backup method based on characteristic codes | |
| CN105512320A (en) | User ranking obtaining method and device and server | |
| CN104424229A (en) | Calculating method and system for multi-dimensional division | |
| CN104298541A (en) | Data distribution algorithm and data distribution device for cloud storage system | |
| CN104702625A (en) | Method and device for scheduling access request in CDN (Content Delivery Network) | |
| CN108874803A (en) | Date storage method, device and storage medium | |
| CN104731951A (en) | Data query method and device | |
| CN105279258B (en) | File storage method and system with balanced distribution | |
| CN107203532A (en) | Construction method, the implementation method of search and the device of directory system | |
| CN104486777A (en) | Method and device for processing data | |
| CN105045762A (en) | Management method and apparatus for configuration file | |
| CN108154024B (en) | Data retrieval method and device and electronic equipment | |
| CN108399175B (en) | Data storage and query method and device | |
| Xiong et al. | kBF: Towards approximate and bloom filter based key-value storage for cloud computing systems | |
| US8788470B2 (en) | Allocating and managing random identifiers using a shared index set across products | |
| CN105245624A (en) | Cloud storage file system | |
| JP6233846B2 (en) | Variable-length nonce generation | |
| CN112948531B (en) | Massive track query method, retrieval server and system | |
| CN104679821A (en) | Data access processing method and data access processing device | |
| CN104239307A (en) | User information storage method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |