CN106407205A - Data aggregation method and apparatus - Google Patents
Data aggregation method and apparatus Download PDFInfo
- Publication number
- CN106407205A CN106407205A CN201510455006.2A CN201510455006A CN106407205A CN 106407205 A CN106407205 A CN 106407205A CN 201510455006 A CN201510455006 A CN 201510455006A CN 106407205 A CN106407205 A CN 106407205A
- Authority
- CN
- China
- Prior art keywords
- characteristic
- index
- convergence
- dimension
- characteristic value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000002776 aggregation Effects 0.000 title claims abstract description 45
- 238000004220 aggregation Methods 0.000 title claims abstract description 45
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000004364 calculation method Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims description 17
- 238000012544 monitoring process Methods 0.000 description 50
- 238000010586 diagram Methods 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention relates to a data aggregation method and apparatus. The method comprises the steps of receiving an input condition sent by a user terminal; querying an index tree according to the input condition of the user terminal; obtaining an index characteristic matched with the input condition; obtaining a storage address, corresponding to the index characteristic, of a characteristic value of the index characteristic; obtaining the characteristic value of the index characteristic corresponding to the storage address of the characteristic value of the index characteristic from a characteristic value memory according to the storage address of the characteristic value of the index characteristic; and sending the characteristic value of the index characteristic to the user terminal. Therefore, the quantity of characteristic values called from the characteristic value memory is reduced, the calculation amount in the data aggregation process is reduced, and the data aggregation speed is increased.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data aggregation method and apparatus.
Background
At present, with the increasing popularity of network informatization, we are in a "big data" era with explosive growth of data. In the aspect of big data monitoring, a monitoring system collects characteristic information and characteristic values of each service in real time, for example, the characteristic information of the service may include, for example, a region where a service server is located, an operator to which the service server belongs, and/or a monitoring index of the service; the characteristic value is a specific numerical value corresponding to the monitoring index, and for example, if the monitoring index is the number of requests, the characteristic value is the number of requests per unit time. The monitoring system can store the collected characteristic value in the characteristic value memory, acquire the corresponding storage address identifier, allocate the corresponding identifier for the collected characteristic information, and store the storage address identifier, the characteristic information and the identifier of the characteristic information in the relation tree in a 'simple characteristic' form. When the monitoring system receives the input condition of the user terminal, the relation tree is inquired according to the input condition, the simple characteristic matched with the characteristic information in the input condition is obtained, and then the corresponding characteristic value is obtained from the characteristic value storage according to the storage address identification in the simple characteristic.
When the number of the simple characteristics matched with the characteristic information in the input condition is multiple, the corresponding characteristic value of each simple characteristic needs to be acquired, all the acquired characteristic values are converged, and the obtained convergence result is returned to the user terminal, so that the problem of low convergence speed is caused, and the real-time performance of displaying, alarming or fault analysis of the user terminal according to the convergence result is influenced.
Disclosure of Invention
The invention provides a data aggregation method and a data aggregation device, which are used for solving the problem of low aggregation speed in the prior art.
The first aspect of the present invention provides a data aggregation method, including:
receiving an input condition sent by a user terminal;
inquiring an index tree, and when acquiring the index characteristics matched with the input conditions, acquiring the storage address of the characteristic value of the index characteristics corresponding to the index characteristics;
acquiring a characteristic value of the index characteristic corresponding to a storage address of the characteristic value of the index characteristic from a characteristic value storage;
and sending the characteristic value of the index characteristic to the user terminal so that the user terminal can perform corresponding processing according to the characteristic value of the index characteristic.
Another aspect of the present invention provides a data convergence device, including:
the receiving module is used for receiving input conditions sent by the user terminal;
the query module is used for querying the index tree and acquiring the storage address of the characteristic value of the index characteristic corresponding to the index characteristic when the index characteristic matched with the input condition is acquired;
an obtaining module, configured to obtain, from a characteristic value storage, a characteristic value of an index characteristic corresponding to a storage address of the characteristic value of the index characteristic;
and the sending module is used for sending the characteristic value of the index characteristic to the user terminal so that the user terminal can correspondingly process according to the characteristic value of the index characteristic.
According to the method and the device, the input condition sent by the user terminal is received, the index tree is inquired according to the input condition of the user terminal, the index characteristic matched with the input condition is obtained, the storage address of the characteristic value of the index characteristic corresponding to the index characteristic is obtained, the characteristic value of the index characteristic corresponding to the storage address of the characteristic value of the index characteristic is obtained from the characteristic value storage according to the storage address of the characteristic value of the index characteristic, and the characteristic value of the index characteristic is sent to the user terminal, so that the number of the characteristic values called from the characteristic value storage is reduced, the calculation amount in the data aggregation process is reduced, and the data aggregation speed is improved.
Drawings
FIG. 1 is a flow chart of an embodiment of a data aggregation method provided by the present invention;
FIG. 2 is a diagram illustrating an index tree in the data aggregation method according to the present invention;
FIG. 3 is a flow chart of a data aggregation method according to another embodiment of the present invention;
FIG. 4 is a schematic structural diagram of an embodiment of a data aggregation apparatus provided in the present invention;
FIG. 5 is a schematic structural diagram of a data aggregation apparatus according to another embodiment of the present invention;
fig. 6 is a schematic structural diagram of another embodiment of a data aggregation device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a flowchart of an embodiment of a data aggregation method provided by the present invention, as shown in fig. 1, specifically including the following steps:
101. and receiving the input condition sent by the user terminal.
The execution main body of the invention is a data aggregation device, the data aggregation device can be a monitoring system, the monitoring system can collect the characteristic value and the characteristic information of each service, process and store the characteristic value and the characteristic information of each service, receive the input condition of the user terminal, inquire the pre-stored data according to the characteristic in the input condition of the user terminal, obtain the characteristic value matched with the input condition and provide the characteristic value for the user terminal, so that the user terminal can carry out fault analysis, alarm analysis, display and the like according to the characteristic value matched with the input condition.
The input conditions may include: characteristics of one or more dimensions. The dimension can be a service name, related software, a region where the service server is located, an operator to which the service server belongs, a task type of the service, a service interface, an error code, a machine room of the service server, a service convergence index and the like. The characteristic of the dimension of the region where the business server is located can be the regions of Shanghai, Beijing, Shenzhen and the like; the service server belongs to the operator dimension which is characterized by mobile, Unicom, telecom and other operators; the characteristics of the machine room dimension of the service server can be a machine room A, a machine room B, a machine room C and the like, wherein A, B, C can be the serial number of the machine room; the characteristics of the task type dimension can be uploading, downloading and the like; the service convergence index dimension can be characterized by the number of service requests, the number of successful requests, the success probability of the requests and the like in unit time; the service interface dimension may be characterized as interface a, interface b, interface c, etc., where a, b, c may be interface numbers. For example, the input condition may be { shanghai, telecommunications, room C }, { beijing, room C }, and the like. Currently, a monitoring system can support 25-dimensional features.
102. When the index characteristics matching the input conditions are acquired, the index tree is searched for, and the storage address of the characteristic value of the index characteristic corresponding to the index characteristic is acquired.
Wherein, the index tree includes: root nodes, trunk nodes, and leaf nodes. The leaf nodes of the index tree are simple characteristics, and the trunk nodes of the index tree are index characteristics.
Assuming that the total number of dimensions is N, the index properties in the index tree include: the storage address of the characteristic value of the index characteristic, the characteristic information of the index characteristic and the corresponding characteristic information identifier, wherein the characteristic information of the index characteristic comprises: n-1 dimensions of features. Simple properties in the index tree include: the storage address of the characteristic value of the simple characteristic, the characteristic information of the simple characteristic and the corresponding characteristic information identification, wherein the characteristic information of the simple characteristic comprises the following components: n dimensions of properties. Each dimension in the characteristic information of the simple characteristic can be a service name, related software, a region where the service server is located, an operator to which the service server belongs, a task type of the service, a service interface, an error code, a machine room of the service server, a monitoring index of the service, and the like. The characteristics of the monitoring index dimension can be the number of service requests, the number of successful requests and the like in unit time.
For example, assume that the total number of dimensions is 3, and the 3 dimensions are respectively: the region where the service server is located, the operator to which the service server belongs, and the machine room of the service server. The characteristic information of the simple characteristics in the index tree is { Shanghai, telecom, machine room B }, { Beijing, telecom, machine room C }, { Shenzhen, telecom, machine room B }, { Beijing, Unicom, machine room C }, { Shenzhen, Unicom, machine room C }, { Shanghai, telecom, machine room C }, respectively. The characteristic information of the index characteristic in the index tree is { telecom, machine room C }, { telecom, machine room B }, { UNICOM, machine room C }, respectively, and the structure of the index tree is shown in FIG. 2.
If the input condition is { telecom, computer room C }, the characteristic information of the index characteristic matched with the input condition is { telecom, computer room C }, and the monitoring system can acquire the storage address of the characteristic value of the index characteristic corresponding to the index characteristic. If the input condition is { computer room C }, the characteristic information of the index characteristic matched with the input condition is { telecommunication, computer room C } and { Unicom, computer room C }.
In this embodiment, when the characteristic of a certain dimension in the input condition is a summary characteristic, for example, when there is no characteristic of a regional dimension in the input condition, the characteristic information matched with the input condition is queried as an index characteristic of { telecommunications, room C }, instead of querying and acquiring simple characteristics of characteristic information of { shanghai, telecommunications, room C }, { shenzhen telecommunications, room C }, and acquiring characteristic values of the three simple characteristics for calculation, the number of characteristic values retrieved from the characteristic value memory is reduced, the amount of calculation in the data aggregation process is reduced, and the data aggregation speed is increased.
103. The characteristic value of the index characteristic corresponding to the storage address of the characteristic value of the index characteristic is acquired from the characteristic value storage.
Wherein the property values of the index property may be stored in a property value store. The property value store may be a non-relational database.
104. And sending the characteristic value of the index characteristic to the user terminal so that the user terminal can perform corresponding processing according to the characteristic value of the index characteristic.
In addition, if the number of the index characteristics matched with the input conditions is large, the monitoring system may perform aggregation calculation on the index characteristics first and then provide the aggregated index characteristics to the user terminal. The manner of convergence calculation may be determined according to the convergence index in the input condition. For example, if the aggregation indicator is the number of successful requests, the aggregation calculation is performed by summing the characteristic values of the index characteristics that match the input conditions. For example, if the aggregation index is an average number of requests, the aggregation calculation method is to average the characteristic values of the index characteristics matching the input conditions.
In this embodiment, by receiving an input condition sent by a user terminal, querying an index tree according to the input condition of the user terminal, obtaining an index characteristic matched with the input condition, obtaining a storage address of a characteristic value of the index characteristic corresponding to the index characteristic, obtaining a characteristic value of the index characteristic corresponding to the storage address of the characteristic value of the index characteristic from a characteristic value storage according to the storage address of the characteristic value of the index characteristic, and sending the characteristic value of the index characteristic to the user terminal, the number of the characteristic values retrieved from the characteristic value storage is reduced, the amount of calculation in a data aggregation process is reduced, and the data aggregation speed is increased.
Fig. 3 is a flowchart of another embodiment of the data aggregation method provided by the present invention, as shown in fig. 3, based on the embodiment shown in fig. 1, the method specifically includes the following steps:
301. a convergence dimension is determined.
In the first scenario, the convergence dimension may be a plurality of pre-configured dimensions, and the monitoring system may converge for each dimension. For example, the convergence dimension may be any one or more of a region convergence dimension, an operator convergence dimension, an interface convergence dimension, an error code convergence dimension, a machine room convergence dimension, and a task convergence dimension.
In another scenario, the convergence dimension may be a convergence dimension determined according to a convergence condition of each of the preconfigured plurality of dimensions. Wherein, the convergence condition of each dimension refers to the convergence number of each dimension. The convergence number refers to the number of index features obtained by summarizing at least one simple feature on a certain feature.
For example, if the property information of each simple property in the relationship tree is { shanghai, telecommunications, machine room B }, { beijing, telecommunications, machine room C }, { shenzhen, telecommunications, machine room B }, { beijing, unicom, machine room C }, { shenzhen, unicom, machine room C }, { shanghai, telecommunications, machine room C }, respectively. The convergence number obtained by convergence in the region dimension is 3, the convergence number obtained by convergence in the operator dimension is 5, and the convergence number obtained by convergence in the machine room dimension is 5. And determining the convergence dimension as the region dimension if the convergence number obtained by convergence on the region dimension is the minimum and the convergence condition is the best.
In this embodiment, the dimension with the smallest convergence number is selected as the summary dimension, which can reduce the number of index features, so that when the monitoring system receives the input condition of the user, the number of index features obtained by matching is smaller, thereby reducing the number of feature values called from the feature value memory, and reducing the calculation amount in the data aggregation process.
Before step 301, the monitoring system may collect characteristic information of each internal service such as Tencent, Baidu, Jingdong, or Google and characteristic values in each time period, store the characteristic values in a characteristic value memory, and store storage address identifiers of the characteristic values and identifiers of the characteristic information in a relationship tree in a simple characteristic form.
Further, in order to facilitate storage and calculation of the characteristic value, the characteristic information of the simple characteristic may further include: a time period. The time period may be 1 minute, 1 hour, 2 hours, a day, etc. For example, if the monitoring system collects the characteristic value and the characteristic information of each service every minute within the previous minute and the time period is 2 hours, the characteristic value of each simple characteristic may be a characteristic number group, and the characteristic value group includes 120 characteristic values within 2 hours.
When the monitoring system collects the characteristic value and characteristic information of a certain service for the first time, for example, at 8 a.m.: 01 when collecting the characteristic value and the characteristic information of a certain service, the monitoring system may create a new leaf node in the relation tree and create a storage area in the characteristic value storage for storing the characteristic value within 2 hours. When the monitoring system receives 8:02, 8:03, … …, 10:00, etc., the characteristic value of the service having the same characteristic information, the characteristic value of each time is stored in the area in sequence. When the monitoring system collects the characteristic value of the service with the same characteristic information again at 10:01, the monitoring system can create a new leaf node in the relation tree and create a new storage area in the characteristic value storage for storing the characteristic value collected between 10:01 and 12: 00.
302. According to the convergence dimension, carrying out convergence processing on each simple characteristic in the relation tree to obtain an index tree, wherein the index tree comprises: an index property, and a simple property under the attribute index property.
For example, if the property information of each simple property in the relationship tree is { shanghai, telecommunications, machine room B }, { beijing, telecommunications, machine room C }, { shenzhen, telecommunications, machine room B }, { beijing, unicom, machine room C }, { shenzhen, unicom, machine room C }, { shanghai, telecommunications, machine room C }, respectively. In the case where the convergence dimension is the region dimension, the index tree is generated as shown in fig. 2. The simple characteristics of the index characteristic with characteristic information of { telecommunication, computer room C } include: the characteristic information is three simple characteristics of { Beijing, telecom, computer room C }, { Shenzhen, telecom, computer room C } and { Shanghai, telecom, computer room C }.
303. And acquiring the characteristic value of the index characteristic corresponding to the index characteristic, and storing the characteristic value of the index characteristic in a characteristic value memory.
Step 303 may specifically include: acquiring a characteristic value of a simple characteristic under the attribute index characteristic; and calculating the characteristic value of the determined index characteristic according to the characteristic value of the simple characteristic under the attribute index characteristic.
Wherein the characteristic value of the simple characteristic can be acquired from the characteristic value memory based on the storage address of the characteristic value of the simple characteristic. The calculation method of the characteristic value of the index characteristic may be determined according to the characteristic of the monitoring index dimension in the characteristic information of the index characteristic. For example, if the characteristic of the monitoring index dimension is the request number, the characteristic value of the index characteristic is the sum of the characteristic values of the simple characteristics under the attribution index characteristic; if the characteristic of the monitoring index dimension is the average request number, the characteristic value of the index characteristic is the average value of the characteristic values of the simple characteristics under the attribution index characteristic.
304. In the index tree, a storage address of a characteristic value of an index characteristic corresponding to the index characteristic is recorded.
After the monitoring system stores the characteristic value of the index characteristic in the characteristic value memory, the monitoring system can receive the storage address of the characteristic value of the index characteristic fed back by the characteristic value memory so as to record the characteristic value.
305. And receiving the input condition sent by the user terminal.
306. When the index characteristics matching the input conditions are acquired, the index tree is searched for, and the storage address of the characteristic value of the index characteristic corresponding to the index characteristic is acquired.
In this case, the index tree may not be obtained according to the input condition, for example, when the input condition is { computer room C }, the index tree shown in fig. 2 is queried, and the index characteristic matching the input condition is not obtained. Therefore, in order to ensure the query efficiency, before step 306, the monitoring system may determine whether the absolute value of the difference between the number of dimensions in the input condition and the total number of dimensions is greater than 1, if the absolute value is greater than 1, it indicates that the characteristics of more than 2 dimensions in the input condition are summary characteristics, for example, when the input condition is { computer room C }, the characteristics of the area dimension and the operator dimension are summary characteristics, at this time, the monitoring system may split the input condition { computer room C }, and obtain { telecommunications, computer room C }, { communications, and computer room C }, and then query the index tree according to the split input condition, and obtain the index characteristics matched with the input condition.
307. The characteristic value of the index characteristic corresponding to the storage address of the characteristic value of the index characteristic is acquired from the characteristic value storage.
308. And sending the characteristic value of the index characteristic to the user terminal so that the user terminal can perform corresponding processing according to the characteristic value of the index characteristic.
In addition, the above steps 301 to 304 may be repeatedly performed at a preset cycle. The preset period may be every minute, every 5 minutes, every hour, every day, etc., and is not particularly limited herein. Taking every minute as an example, in the time period from 8:01 to 10:00, the monitoring system can perform convergence processing on simple characteristics in the relation tree from 8:01 to the current time in a cycle of every minute; in the time period of 10:01 to 12:00, the monitoring system can perform convergence processing on simple characteristics in the relation tree from 10:01 to the current time in a period of every minute; when the query time period in the input condition of the user is 9:30 to 13:10, the monitoring system may acquire the index characteristics in the 9:01 to 10:00 time period obtained by summarizing the time 10:01, the index characteristics in the relationship tree in the 10:01 to 12:00 time period obtained by summarizing the time 12:01, and the index characteristics in the 12:01 to 13:00 time period obtained by summarizing the time 13:11, and further perform convergence processing on the acquired index characteristics.
In this embodiment, by determining a convergence dimension, according to the convergence dimension, convergence processing is performed on each simple characteristic in the relationship tree, and an index tree is obtained, where the index tree includes: index characteristics and simple characteristics under the attribution index characteristics, characteristic values of the index characteristics corresponding to the index characteristics are obtained and stored in a characteristic value storage, storage addresses of the characteristic values of the index characteristics corresponding to the index characteristics are recorded in an index tree, so that when a monitoring system receives input conditions sent by a user terminal, the monitoring system can query the index tree according to the input conditions of the user terminal to obtain the index characteristics matched with the input conditions, obtain the storage addresses of the characteristic values of the index characteristics corresponding to the index characteristics, obtain the characteristic values of the index characteristics corresponding to the storage addresses of the characteristic values of the index characteristics from a characteristic value storage according to the storage addresses of the characteristic values of the index characteristics, send the characteristic values of the index characteristics to the user terminal, and reduce the number of the characteristic values called from the characteristic value storage, the calculated amount in the data gathering process is reduced, and the data gathering speed is improved.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Fig. 4 is a schematic structural diagram of an embodiment of a data aggregation device provided by the present invention, as shown in fig. 4, including:
a receiving module 41, configured to receive an input condition sent by a user terminal;
a query module 42, configured to query the index tree, and when acquiring the index characteristic matching the input condition, acquire a storage address of a characteristic value of the index characteristic corresponding to the index characteristic;
an obtaining module 43, configured to obtain, from the characteristic value storage, a characteristic value of the index characteristic corresponding to a storage address of the characteristic value of the index characteristic;
and a sending module 44, configured to send the characteristic value of the index characteristic to the user terminal, so that the user terminal performs corresponding processing according to the characteristic value of the index characteristic.
The execution main body of the invention is a data aggregation device, the data aggregation device can be a monitoring system, the monitoring system can collect the characteristic value and the characteristic information of each service, process and store the characteristic value and the characteristic information of each service, receive the input condition of the user terminal, inquire the pre-stored data according to the characteristic in the input condition of the user terminal, obtain the characteristic value matched with the input condition and provide the characteristic value for the user terminal, so that the user terminal can carry out fault analysis, alarm analysis, display and the like according to the characteristic value matched with the input condition.
The input conditions may include: characteristics of one or more dimensions. The dimension can be a service name, related software, a region where the service server is located, an operator to which the service server belongs, a task type of the service, a service interface, an error code, a machine room of the service server, a service convergence index and the like.
Assuming that the total number of dimensions is N, the index properties in the index tree include: the storage address of the characteristic value of the index characteristic, the characteristic information of the index characteristic and the corresponding characteristic information identifier, wherein the characteristic information of the index characteristic comprises: n-1 dimensions of features. Simple properties in the index tree include: the storage address of the characteristic value of the simple characteristic, the characteristic information of the simple characteristic and the corresponding characteristic information identification, wherein the characteristic information of the simple characteristic comprises the following components: n dimensions of properties. Each dimension in the characteristic information of the simple characteristic can be a service name, related software, a region where the service server is located, an operator to which the service server belongs, a task type of the service, a service interface, an error code, a machine room of the service server, a monitoring index of the service, and the like. The characteristics of the monitoring index dimension can be the number of service requests, the number of successful requests and the like in unit time. For example, if the input condition may be { telecommunications, room C }, the characteristic information of the index characteristic matching the input condition may be { telecommunications, room C }, and the monitoring system may obtain a storage address of a characteristic value of the index characteristic corresponding to the index characteristic. If the input condition is { computer room C }, the characteristic information of the index characteristic matched with the input condition is { telecommunication, computer room C } and { Unicom, computer room C }.
In addition, it should be noted that, if the number of the index characteristics matching the input condition is large, the monitoring system may perform aggregation calculation on the index characteristics first and then provide the result to the user terminal. The manner of convergence calculation may be determined according to the convergence index in the input condition. For example, if the aggregation indicator is the number of successful requests, the aggregation calculation is performed by summing the characteristic values of the index characteristics that match the input conditions. For example, if the aggregation index is an average number of requests, the aggregation calculation method is to average the characteristic values of the index characteristics matching the input conditions.
In this embodiment, by receiving an input condition sent by a user terminal, querying an index tree according to the input condition of the user terminal, obtaining an index characteristic matched with the input condition, obtaining a storage address of a characteristic value of the index characteristic corresponding to the index characteristic, obtaining a characteristic value of the index characteristic corresponding to the storage address of the characteristic value of the index characteristic from a characteristic value storage according to the storage address of the characteristic value of the index characteristic, and sending the characteristic value of the index characteristic to the user terminal, the number of the characteristic values retrieved from the characteristic value storage is reduced, the amount of calculation in a data aggregation process is reduced, and the data aggregation speed is increased.
Fig. 5 is a schematic structural diagram of another embodiment of the data aggregation device provided by the present invention, as shown in fig. 5, based on the embodiment shown in fig. 4, the device further includes: a determination module 45, a convergence module 46 and a recording module 47;
a determination module 45 for determining a convergence dimension;
a convergence module 46, configured to perform convergence processing on each simple characteristic in the relationship tree according to a convergence dimension, to obtain an index tree, where the index tree includes: an index property, and a simple property under an attribution index property;
the obtaining module 43 is further configured to obtain a characteristic value of the index characteristic corresponding to the index characteristic, and store the characteristic value of the index characteristic in the characteristic value storage;
and a recording module 47, configured to record, in the index tree, a storage address of a characteristic value of the index characteristic corresponding to the index characteristic.
The determining module is specifically configured to determine a plurality of preconfigured dimensions as convergence dimensions, and converge for each dimension; or, determining a plurality of pre-configured dimensions as convergence dimensions, and determining a final convergence dimension according to the convergence condition of each dimension for convergence processing. The convergence dimension is: any one or more of a region convergence dimension, an operator convergence dimension, an interface convergence dimension, an error code convergence dimension, a machine room convergence dimension, and a task convergence dimension.
In this embodiment, the dimension with the smallest convergence number is selected as the summary dimension, which can reduce the number of index features, so that when the monitoring system receives the input condition of the user, the number of index features obtained by matching is smaller, thereby reducing the number of feature values called from the feature value memory, and reducing the calculation amount in the data aggregation process.
Further, in order to facilitate storage and calculation of the characteristic value, the characteristic information of the simple characteristic may further include: a time period. The time period may be 1 minute, 1 hour, 2 hours, a day, etc. For example, if the monitoring system collects the characteristic value and the characteristic information of each service every minute within the previous minute and the time period is 2 hours, the characteristic value of each simple characteristic may be a characteristic number group, and the characteristic value group includes 120 characteristic values within 2 hours.
Further, the obtaining module 43 may include: an acquisition unit 431 and a calculation unit 432, as shown in fig. 6;
an acquisition unit 431 configured to acquire a characteristic value of the simple characteristic under the home index characteristic;
a calculating unit 432, configured to calculate a characteristic value of the index characteristic according to the characteristic value of the simple characteristic under the attribute index characteristic.
The calculation method of the calculation unit 432 may be determined according to the characteristic of the monitoring index dimension in the characteristic information of the index characteristic. For example, if the characteristic of the monitoring index dimension is the request number, the characteristic value of the index characteristic is the sum of the characteristic values of the simple characteristics under the attribution index characteristic; if the characteristic of the monitoring index dimension is the average request number, the characteristic value of the index characteristic is the average value of the characteristic values of the simple characteristics under the attribution index characteristic.
In addition, the index tree is directly queried according to the input condition, and the index characteristic matching the input condition may not be obtained, for example, when the input condition is { computer room C }, the index tree shown in fig. 2 is queried, and the index characteristic matching the input condition is not obtained. Therefore, in order to ensure the query efficiency, before step 306, the monitoring system may determine whether the absolute value of the difference between the number of dimensions in the input condition and the total number of dimensions is greater than 1, if the absolute value is greater than 1, it indicates that the characteristics of more than 2 dimensions in the input condition are summary characteristics, for example, when the input condition is { computer room C }, the characteristics of the area dimension and the operator dimension are summary characteristics, at this time, the monitoring system may split the input condition { computer room C }, and obtain { telecommunications, computer room C }, { communications, and computer room C }, and then query the index tree according to the split input condition, and obtain the index characteristics matched with the input condition.
In this embodiment, by determining a convergence dimension, according to the convergence dimension, convergence processing is performed on each simple characteristic in the relationship tree, and an index tree is obtained, where the index tree includes: index characteristics and simple characteristics under the attribution index characteristics, characteristic values of the index characteristics corresponding to the index characteristics are obtained and stored in a characteristic value storage, storage addresses of the characteristic values of the index characteristics corresponding to the index characteristics are recorded in an index tree, so that when a monitoring system receives input conditions sent by a user terminal, the monitoring system can query the index tree according to the input conditions of the user terminal to obtain the index characteristics matched with the input conditions, obtain the storage addresses of the characteristic values of the index characteristics corresponding to the index characteristics, obtain the characteristic values of the index characteristics corresponding to the storage addresses of the characteristic values of the index characteristics from a characteristic value storage according to the storage addresses of the characteristic values of the index characteristics, send the characteristic values of the index characteristics to the user terminal, and reduce the number of the characteristic values called from the characteristic value storage, the calculated amount in the data gathering process is reduced, and the data gathering speed is improved.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A data aggregation method, comprising:
receiving an input condition sent by a user terminal;
inquiring an index tree, and when acquiring the index characteristics matched with the input conditions, acquiring the storage address of the characteristic value of the index characteristics corresponding to the index characteristics;
acquiring a characteristic value of the index characteristic corresponding to a storage address of the characteristic value of the index characteristic from a characteristic value storage;
and sending the characteristic value of the index characteristic to the user terminal so that the user terminal can perform corresponding processing according to the characteristic value of the index characteristic.
2. The method of claim 1, further comprising:
determining a convergence dimension;
according to the convergence dimension, carrying out convergence processing on each simple characteristic in the relation tree to obtain the index tree, wherein the index tree comprises: an index property, and a simple property attributed to the index property;
acquiring a characteristic value of the index characteristic corresponding to the index characteristic, and storing the characteristic value of the index characteristic in the characteristic value memory;
in the index tree, a storage address of a characteristic value of an index characteristic corresponding to the index characteristic is recorded.
3. The method of claim 2, wherein determining the convergence dimension comprises:
a plurality of dimensions, which may be pre-configured, for each dimension to converge;
or,
the plurality of dimensions can be pre-configured, but according to the convergence condition of each dimension, a final convergence dimension is determined for convergence processing.
4. The method of claim 2, wherein obtaining the characteristic value of the index characteristic corresponding to the index characteristic comprises:
acquiring a characteristic value of a simple characteristic under the attribution of the index characteristic;
and calculating and determining the characteristic value of the index characteristic according to the characteristic value of the simple characteristic belonging to the index characteristic.
5. A method according to claim 2 or 3, wherein the convergence dimension is: any one or more of a region convergence dimension, an operator convergence dimension, an interface convergence dimension, an error code convergence dimension, a machine room convergence dimension, and a task convergence dimension.
6. A data convergence device, comprising:
the receiving module is used for receiving input conditions sent by the user terminal;
the query module is used for querying the index tree and acquiring the storage address of the characteristic value of the index characteristic corresponding to the index characteristic when the index characteristic matched with the input condition is acquired;
an obtaining module, configured to obtain, from a characteristic value storage, a characteristic value of an index characteristic corresponding to a storage address of the characteristic value of the index characteristic;
and the sending module is used for sending the characteristic value of the index characteristic to the user terminal so that the user terminal can correspondingly process according to the characteristic value of the index characteristic.
7. The apparatus of claim 6, further comprising: the device comprises a determining module, a convergence module and a recording module;
the determining module is used for determining a convergence dimension;
the convergence module is configured to perform convergence processing on each simple characteristic in the relationship tree according to the convergence dimension to obtain the index tree, where the index tree includes: an index property, and a simple property attributed to the index property;
the obtaining module is further configured to obtain a characteristic value of the index characteristic corresponding to the index characteristic, and store the characteristic value of the index characteristic in the characteristic value storage;
the recording module is used for recording the storage address of the characteristic value of the index characteristic corresponding to the index characteristic in the index tree.
8. The apparatus of claim 7, wherein the determining module is configured to determine the determined value
Determining a plurality of pre-configured dimensions as convergence dimensions, and converging for each dimension;
or,
and determining a plurality of pre-configured dimensions as convergence dimensions, but determining a final convergence dimension for convergence processing according to the convergence condition of each dimension.
9. The apparatus of claim 7, wherein the obtaining module comprises: an acquisition unit and a calculation unit;
the acquiring unit is used for acquiring a characteristic value of a simple characteristic under the index characteristic;
the calculation unit is used for calculating and determining the characteristic value of the index characteristic according to the characteristic value of the simple characteristic belonging to the index characteristic.
10. The apparatus of claim 7 or 8, wherein the convergence dimension is: any one or more of a region convergence dimension, an operator convergence dimension, an interface convergence dimension, an error code convergence dimension, a machine room convergence dimension, and a task convergence dimension.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510455006.2A CN106407205B (en) | 2015-07-29 | 2015-07-29 | Data aggregation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510455006.2A CN106407205B (en) | 2015-07-29 | 2015-07-29 | Data aggregation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106407205A true CN106407205A (en) | 2017-02-15 |
| CN106407205B CN106407205B (en) | 2019-12-20 |
Family
ID=58009025
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510455006.2A Active CN106407205B (en) | 2015-07-29 | 2015-07-29 | Data aggregation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106407205B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101262494A (en) * | 2008-01-23 | 2008-09-10 | 华为技术有限公司 | Method, client, server and system for processing distributed information |
| CN101398830A (en) * | 2007-09-27 | 2009-04-01 | 阿里巴巴集团控股有限公司 | Thesaurus fuzzy enquiry method and thesaurus fuzzy enquiry system |
| CN101957973A (en) * | 2010-10-29 | 2011-01-26 | 江苏瑞蚨通软件科技有限公司（中外合资） | System for recommending commodities |
| CN102209118A (en) * | 2011-06-10 | 2011-10-05 | 成都勤智数码科技有限公司 | Distributed mass data gathering method |
| CN103412897A (en) * | 2013-07-25 | 2013-11-27 | 中国科学院软件研究所 | Parallel data processing method based on distributed structure |
| US20140006411A1 (en) * | 2012-06-29 | 2014-01-02 | Nokia Corporation | Method and apparatus for multidimensional data storage and file system with a dynamic ordered tree structure |
-
2015
- 2015-07-29 CN CN201510455006.2A patent/CN106407205B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101398830A (en) * | 2007-09-27 | 2009-04-01 | 阿里巴巴集团控股有限公司 | Thesaurus fuzzy enquiry method and thesaurus fuzzy enquiry system |
| CN101262494A (en) * | 2008-01-23 | 2008-09-10 | 华为技术有限公司 | Method, client, server and system for processing distributed information |
| CN101957973A (en) * | 2010-10-29 | 2011-01-26 | 江苏瑞蚨通软件科技有限公司（中外合资） | System for recommending commodities |
| CN102209118A (en) * | 2011-06-10 | 2011-10-05 | 成都勤智数码科技有限公司 | Distributed mass data gathering method |
| US20140006411A1 (en) * | 2012-06-29 | 2014-01-02 | Nokia Corporation | Method and apparatus for multidimensional data storage and file system with a dynamic ordered tree structure |
| CN103412897A (en) * | 2013-07-25 | 2013-11-27 | 中国科学院软件研究所 | Parallel data processing method based on distributed structure |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106407205B (en) | 2019-12-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109117275B (en) | Account checking method and device based on data slicing, computer equipment and storage medium | |
| CN111258978B (en) | Data storage method | |
| CN107169094B (en) | Information aggregation method and device | |
| US11762879B2 (en) | Information traceability method and system based on blockchain | |
| CN106095789B (en) | Message subscription management device and method | |
| US11188443B2 (en) | Method, apparatus and system for processing log data | |
| CN109213758B (en) | Data access method, device, equipment and computer readable storage medium | |
| CN112052247A (en) | Index updating system, method and device of search engine, electronic equipment and storage medium | |
| CN106326280B (en) | Data processing method, device and system | |
| CN111538563A (en) | Event analysis method and device for Kubernetes | |
| CN108366012B (en) | Social relationship establishing method and device and electronic equipment | |
| CN108154024B (en) | Data retrieval method and device and electronic equipment | |
| CN112491609A (en) | System application architecture discovery method and system based on business service | |
| CN109218131B (en) | Network monitoring method and device, computer equipment and storage medium | |
| CN110198227B (en) | Log file processing method and device, storage medium and electronic device | |
| CN114510708A (en) | Real-time data warehouse construction and anomaly detection method, device, equipment and product | |
| CN112148920B (en) | Data management method | |
| CN113297253A (en) | Equipment identification method, device, equipment and readable storage medium | |
| CN112764988A (en) | Data segmentation acquisition method and device | |
| CN112463527A (en) | Data processing method, device, equipment, system and storage medium | |
| CN106407205B (en) | Data aggregation method and device | |
| CN112148925A (en) | User identification correlation query method, device, equipment and readable storage medium | |
| CN116955709A (en) | User data fusion processing method and system | |
| CN111291127B (en) | Data synchronization method, device, server and storage medium | |
| CN113486237A (en) | Flight information recommendation method and system, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |